Abstract: Approximating a more complex multi-objective feed item scoring model using a less complex single objective feed item scoring model in a multistage feed ranking system of an online service. The disclosed techniques can facilitate multi-objective optimization for personalizing and ranking feeds including balancing personalizing a feed for viewer experience, downstream professional or social network effects, and upstream effects on content creators. The techniques can approximate the multi-objective model—that uses a rich set of machine learning features for scoring feed items at a second pass ranker in the ranking system—with the more lightweight, single objective model—that uses fewer machine learning features at a first pass ranker in the ranking system. The single objective model can more efficiently score a large set of feed items while maintaining much of the multi-objective model's richness and complexity and with high recall at the second pass ranking stage.

Abstract: Approximating a more complex multi-objective feed item scoring model using a less complex single objective feed item scoring model in a multistage feed ranking system of an online service. The disclosed techniques can facilitate multi-objective optimization for personalizing and ranking feeds including balancing personalizing a feed for viewer experience, downstream professional or social network effects, and upstream effects on content creators. The techniques can approximate the multi-objective model—that uses a rich set of machine learning features for scoring feed items at a second pass ranker in the ranking system—with the more lightweight, single objective model—that uses fewer machine learning features at a first pass ranker in the ranking system. The single objective model can more efficiently score a large set of feed items while maintaining much of the multi-objective model's richness and complexity and with high recall at the second pass ranking stage.

Abstract: A feature importance score for a target machine learning feature of a target machine learning model used in a multistage feed ranking system for scoring feed items is supplemented with a feature computing resource cost. The feature computing resource cost represents the cost of using the target feature in the target model in terms of computing resources such as CPU, memory, network resources, etc. A tradeoff between feature importance and feature computing resource cost can be made to decide whether to have the target machine learning model use or not use the target machine learning feature in production, thereby improving the production multistage feed item ranking system and solving the technical problem of determining which machine learning features of a machine learning model represent the best tradeoff between feature importance and feature computing resource cost.

Abstract: A feature importance score for a target machine learning feature of a target machine learning model used in a multistage feed ranking system for scoring feed items is supplemented with a feature computing resource cost. The feature computing resource cost represents the cost of using the target feature in the target model in terms of computing resources such as CPU, memory, network resources, etc. A tradeoff between feature importance and feature computing resource cost can be made to decide whether to have the target machine learning model use or not use the target machine learning feature in production, thereby improving the production multistage feed item ranking system and solving the technical problem of determining which machine learning features of a machine learning model represent the best tradeoff between feature importance and feature computing resource cost.

Abstract: Approximating a more complex multi-objective feed item scoring model using a less complex single objective feed item scoring model in a multistage feed ranking system of an online service. The disclosed techniques can facilitate multi-objective optimization for personalizing and ranking feeds including balancing personalizing a feed for viewer experience, downstream professional or social network effects, and upstream effects on content creators. The techniques can approximate the multi-objective model—that uses a rich set of machine learning features for scoring feed items at a second pass ranker in the ranking system—with the more lightweight, single objective model—that uses fewer machine learning features at a first pass ranker in the ranking system. The single objective model can more efficiently score a large set of feed items while maintaining much of the multi-objective model's richness and complexity and with high recall at the second pass ranking stage.

Abstract: Computer-implemented techniques for approximating recall of a first pass ranker at the second pass ranking stage in a scalable manner. The techniques are efficient in that they do not require the second pass ranker to score all feed items considered by the first pass ranker in order to approximate the recall. Instead, the recall is approximated with a fewer number of feed item for which scores are already logged. Because the first pass ranker scores and the second pass ranker scores are already logged and available at a time of recall approximation, the techniques are computationally efficient. At the same time, using the scores of the fewer number of feed items still gives a good approximation of the recall at the second pass ranking stage.

Abstract: Techniques for optimizing machine-learned models based on dwell time of network-transmitted content items are provided. In one technique, impression data and selection data are used train a selection prediction model. For each impression, a dwell time associated with that impression is determined and compared to a skip time. If the dwell time is less than the skip time, then a first training label that indicates that the impression is skipped is associated with the impression. If the dwell time is greater than the skip time, then a second training label that indicates that the impression is not skipped is associated with the impression. These training labels are used to train a skip prediction model. The selection prediction model and the skip prediction model are used in a content item selection event to generate a score for each candidate content item. The scores are used to select a content item.

Abstract: An online system generates one or more models that determine a likelihood of a user interacting with an application over a particular time interval after installing the application. To generate the one or more models, the online system obtains information describing a user's interaction with the application that occurred greater than a threshold time period prior to a time for which user interaction with the application is to be determined. Example user interactions with the application include: usage of the application, numbers of particular interactions with the application, an amount of compensation the application receives from the user, interactions with other users of the application via the application, and any other suitable interactions. Various engagement metrics may be predicted by the one or more models such as an amount of time spent using the application, particular actions taken in the application, and revenue generated by the user in the application.

Abstract: Methods and systems are described herein for predicting the quality of content items for display to a user of an online system. The method involves training a model to predict user values for content items based on ratings provided by a panel of professional raters for a set of content items. The trained model receives embeddings for a viewing user of the online system and for a page associated with a content item along with edge factors representing the viewing user's interactions on the online system and generates a user value representing the predicted quality of the content item for the viewing user. The method further involves combining the predicted user value with a user interaction score for the content item to generate a content item score used to determine whether to display the content item to the viewing user.

Abstract: An online system generates one or more models that determine a likelihood of a user interacting with an application over a particular time interval after installing the application. To generate the one or more models, the online system obtains information describing a user's interaction with the application that occurred greater than a threshold time period prior to a time for which user interaction with the application is to be determined. Example user interactions with the application include: usage of the application, numbers of particular interactions with the application, an amount of compensation the application receives from the user, interactions with other users of the application via the application, and any other suitable interactions. Various engagement metrics may be predicted by the one or more models such as an amount of time spent using the application, particular actions taken in the application, and revenue generated by the user in the application.