Abstract: An online system determines a stability metric that indicates overlap between the set of entities associated with a particular entity when embeddings have been adjusted due to modifications in the input data of an embedding model. The online system generates a stability score for the embedding model by computing a statistic for one or more stability metrics. The online system determines a stability metric for a particular content provider by identifying a first cluster of content providers in a set of first embeddings, and a second cluster of content providers in a set of second embeddings. The second embeddings are generated after modifications have been made to input data. The online system determines the stability metric based on an overlap between the first cluster and the second cluster of content providers. The stability score can be an indicator of model performance that can be used to select embedding models.

Abstract: An online system maintains one or more models that determine likelihoods of a user performing various interactions after being presented with a content item. Additionally, the online system receives information identifying interactions by users with content, and generates embeddings for various users based on the interactions by the users with content. When determining whether to present a content item including an objective identifying an interaction to a user, the online system applies a maintained model to determine a likelihood of the user performing the interaction identified by the objective after being presented with the content item. Additionally, the online system determines a similarity of the embedding of the user to embeddings of users who performed the interaction identified by the objective. Based on a combination of the likelihood determined by the model and the similarity, the online system determines whether to present the content item to the user.

Abstract: An online system determines a stability metric that indicates overlap between the set of entities associated with a particular entity when embeddings have been adjusted due to modifications in the input data of an embedding model. The online system generates a stability score for the embedding model by computing a statistic for one or more stability metrics. The online system determines a stability metric for a particular content provider by identifying a first cluster of content providers in a set of first embeddings, and a second cluster of content providers in a set of second embeddings. The second embeddings are generated after modifications have been made to input data. The online system determines the stability metric based on an overlap between the first cluster and the second cluster of content providers. The stability score can be an indicator of model performance that can be used to select embedding models.