Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for evaluating and comparing multiple trained machine learning models. Methods can include generating, using a first and a second machine learning model, a respective predicted value for the target attribute. The methods compute a differential value for a model performance metric indicating a difference in the respective model performance attribute values and a corresponding confidence interval that indicates a probability that the differential value accurately reflects the difference in the respective model performance attribute values using a linear regression model and the respective predicted values. The methods then select based on the computed confidence interval a machine learning model. The methods obtain a set of actual data items encountered in a production environment, and use the selected machine learning model to generate a corresponding set of predicted values for the target attribute.