If model quality is too good to be true (>95%), then what should you try?

When the model quality is too high, we usually suspect hindsight bias. Einstein detects and removes any fields susceptible to hindsight bias, but it’s always good to double-check your dataset for other fields that should be removed.