Abstract: A system receives a first plurality of impressions associated with a first set of features. Labels for the first plurality is generated based on the first set of features. A machine learning model is trained based on the first set of features and the labels. A second plurality of impressions associated with a second set of features is received. A first estimated probability measuring conversion likelihood when impression delivery occurs is generated based on applying the second plurality and the second set of features to the model. A plurality of holdout impressions associated with a third set of features is identified. A second estimated probability measuring conversion likelihood when impression delivery is withheld is generated based on applying the plurality of holdout impressions and the third set of features to the model. Increased valued (e.g., lift) is estimated based on subtracting the second estimated probability from the first estimated probability.

Abstract: An online system provides content items to target users who are identified to have high incremental likelihood of performing conversion actions when presented with content items. The incremental likelihood represents the difference between the response likelihood of performing conversion actions when a content item is presented to a user, and the baseline likelihood when a content item is not presented to the user. The baseline and response likelihood for a user are predicted by one or more machine-learned models. By targeting the content to users that are likely to have a high incremental likelihood, the online system provides content items to users whose conversion actions are more likely to be impacted by the presentation of content items, rather than users that may just be of interest for performing the action.