Abstract: An online system accesses a list of features used as input into a predictor to predict a performance metric for content presented to users. The online system computes importance scores for one or more of the features. A ranked list of categories is created, with each category having one or more sub-categories. For each feature having a computed importance score, the online system assigns, for each attribute in the ranked list of attributes for that feature, the feature to a sub-category in one of the categories in the ranked list of categories that has the same rank as the attribute in the ranked list of attributes for the feature, where the sub-category is associated with a label that corresponds with the attribute. For each sub-category in each category, a cumulative score is computed for the sub-category based on the importance scores of the features of that sub-category.

Abstract: An online system ranks content eligible for presentation to an online system user based on a prediction made by a general model or a specific model indicating a likelihood that the user will interact with a content item, in which the specific model has a higher latency than the general model. The online system determines which prediction to use for the ranking by balancing the benefit of a more accurate prediction made by the specific model against the higher latency of the specific model. The online system outputs the predicted likelihood from one of the models based on the determination, ranks content items eligible for presentation to the user based on the output, and selects content item(s) for presentation to the user based on the ranking. The online system may log the predicted likelihoods from both models, the outputted predicted likelihood, and information describing the performance of the content item.

Abstract: An online system accesses a model attribute store, which stores configuration information and model performance scores for a plurality of models, each model used to predict performance metrics regarding content from a third party system presented to users of the online system. The online system trains a meta-model classifier using the models in the model attribute store, the meta-model classifier trained to predict, for a candidate model, a predicted model performance score of that candidate model. The online system also generates a plurality of candidate models for input to the meta-model classifier, each of the plurality of candidate models including a distinct set of configuration information. The predicted model performance scores for a selected candidate model in the plurality of candidate models is computed using the meta-model classifier, and the online system transmits a report to the third party system indicating predicted model performance score for the selected candidate model.