Abstract: In an example embodiment, a specialized machine learned model, called a look-alike model, is trained using a machine learning algorithm to predict future job engagement for a user. This look-alike model is then used to create new segments on top of the segments provided by a rules-based approach. Specifically, the look-alike model is designed to take users who have been segmented by a rule-based approach into an “inactive job seeker” categorization (such as those assigned to the resting users and dormant users segments) and calculate a predicted job engagement score for these users. Based on the predicted job engagement score, a user may then be reassigned from one of the inactive job seeker categorizations to one of one or more new job seeker categorizations (such as predicted open job seekers or predicted opportunistic job seekers).

Abstract: Sponsored and organic pieces of content are displayed in accordance with a blending model that is used to first identify a pattern of slots to which to assign either sponsored or organic pieces of content. This blending model is applied to a combination of both sponsored and non-sponsored pieces of content being considered for display to a user. This pattern only determines the slot assignments. The actual ranking of the pieces of content, and more particularly the actual ranking of the organic pieces of content, is determined by an ordering other than the ranking determined by the blending model, such as by using the original ordering of the second list. The pieces of content are then displayed in the order of this actual ranking, but in the slots indicated as having been assigned to be either sponsored or organic in the pattern determined by the blending model.

Abstract: Sponsored and organic pieces of content are displayed in accordance with a blending model that is used to first identify a pattern of slots to which to assign either sponsored or organic pieces of content. This blending model is applied to a combination of both sponsored and non-sponsored pieces of content being considered for display to a user. This pattern only determines the slot assignments. The actual ranking of the pieces of content, and more particularly the actual ranking of the organic pieces of content, is determined by an ordering other than the ranking determined by the blending model, such as by using the original ordering of the second list. The pieces of content are then displayed in the order of this actual ranking, but in the slots indicated as having been assigned to be either sponsored or organic in the pattern determined by the blending model.

Abstract: In an example embodiment, a fully automated process is provided for frequent model retraining and redeployment of a machine learned model trained to output a prediction of how likely it is that a candidate is qualified for a particular job posting. Model quality verification is provided by maintaining a snapshot of a baseline model and automatically comparing it to a proposed model by performing various metrics on the models by testing the models using a holdout data set that includes only data that was not used during the training process. Overlap between data in the holdout set used during retraining and the training set used during initial training is prevented by splitting each dataset using a hash on certain fields of the data.

Abstract: In an example embodiment, a specialized machine learned model, called a look-alike model, is trained using a machine learning algorithm to predict future job engagement for a user. This look-alike model is then used to create new segments on top of the segments provided by a rules-based approach. Specifically, the look-alike model is designed to take users who have been segmented by a rule-based approach into an “inactive job seeker” categorization (such as those assigned to the resting users and dormant users segments) and calculate a predicted job engagement score for these users. Based on the predicted job engagement score, a user may then be reassigned from one of the inactive job seeker categorizations to one of one or more new job seeker categorizations (such as predicted open job seekers or predicted opportunistic job seekers).