Abstract: Machine learning techniques to optimize user interface template selection are provided. In one technique, a first set of feature values pertaining to a first entity is identified. Multiple sets of feature values are also identified, each set of feature values pertaining to a different user interface (UI) template for rendering content items on a computer screen. For each set of feature values of the multiple sets, the set of feature values and the first set of feature values are inserted into a machine-learned model to generate a score, which is added to a set of scores, which set of scores is initially empty. Based on the set of scores, a particular UI template is selected for a content item. The content item is transmitted over a computer network to be presented on a screen of a computing device of the first entity according to the particular UI template.

Abstract: Techniques are provided for automatically tuning a parameter in a layered model framework. One or more machine learning techniques are used to train multiple versions of a first model that includes a first version and a second version. A second model is stored that includes a parameter and accepts, as input, output from the first model. Multiple parameter values of the parameter are tested when processing content requests using the first and second versions of the first model. A strict subset of the plurality of parameter values are selected for the parameter of the second model, such that processing a first subset of the content requests using the first version of the first model results in a first value of a particular metric that matches a second value of the particular metric resulting from processing a second subset of the content requests using the second version of the first model.

Abstract: Techniques for using machine-learned models to throttle content are provided. In one technique, based on multiple selection events, a distribution of relevance measures is computed, where the relevance measures are associated with the content item selection events. The relevance measures may be generated by one or more machine-learned models. Based on the computed distribution, a threshold relevance measure is computed. Thereafter, a request for content is received over a computer network. In response, a computer system performs, in real-time, multiple steps. For example, an identity of an entity that is associated with the request is identified and, based on that identity, multiple content delivery groups are identified. A relevance measure of one of the content delivery groups relative to the entity is determined and compared to the threshold relevance measure. The content delivery group is selected only after determining that the relevance measure is above the threshold relevance measure.

Abstract: Techniques for using a machine-learned model to personalize content item density. In one technique, an entity that is associated with a content request is identified. Multiple sets of content items are identified that includes content items of different types. A first position of a first slot is determined in a content item feed that comprises multiple slots. A second position of a previous content item is determined, in the content item feed, that is of a first type. A difference between the first position and the second position is determined. Based on the difference, a gap sensitivity value that is associated with the entity and is different than the difference is determined. Based on the gap sensitivity value, a content item from the multiple sets of content items is selected and inserted into the first slot. The content item feed is transmitted to a computing device to be presented thereon.

Abstract: Techniques for selectively transmitting electronic notifications using machine learning techniques based on entity selection history are provided. In one technique, a candidate notification is identified for a target entity. An entity selection rate of the candidate notification by the target entity is determined. Based on the candidate notification, determining a probability of the target entity visiting a target online system. Based on online history of the target entity, a measure of downstream interaction by the target entity relative to one or more online systems is determined. Based on the entity selection rate, the probability, and the measure of downstream interaction by the target entity, a score for the candidate notification is generated. Based on the score, it is determined whether data about the candidate notification is to be transmitted over a computer network to a computing device of the target entity.

Abstract: Techniques for controlling item frequency using machine learning are provides. In one technique, two prediction models are trained: one based on interaction history of multiple content items by multiple entities and the other based on predicted interaction rates and an impression count for each of multiple content items. In response to a request, a particular entity associated with the request is identified and multiple candidate content items are identified. For each identified candidate content item, the first prediction model is used to determine a predicted interaction rate, an impression count of the candidate content item is determined with respect to the particular entity, the second prediction model is used to generate an adjustment based on the impression count, and an adjusted entity interaction rate is generated based on the predicted interaction rate and the adjustment. A particular candidate content item is selected based on the generated adjusted entity interaction rates.

Abstract: Techniques for using a machine-learned model to personalize content item density. In one technique, an entity that is associated with a content request is identified. Multiple sets of content items are identified that includes content items of different types. A first position of a first slot is determined in a content item feed that comprises multiple slots. A second position of a previous content item is determined, in the content item feed, that is of a first type. A difference between the first position and the second position is determined. Based on the difference, a gap sensitivity value that is associated with the entity and is different than the difference is determined. Based on the gap sensitivity value, a content item from the multiple sets of content items is selected and inserted into the first slot. The content item feed is transmitted to a computing device to be presented thereon.

Abstract: Techniques for varying content item density are provided. A first minimum gap value is stored that dictates how close two content items of a first type may appear in a content item feed that contains content items of multiple types that includes the first type and a second type. The first minimum gap value is used to place content items in a first set of content item feeds. For each content item feed of the first set of content item feeds, performance data that indicates how well content items of the first type perform in the content item feed is generated. Based on the performance data and the first minimum gap value, a second minimum gap value that is different than the first minimum gap value is generated. The second minimum gap value is used to place content items in a second plurality of content item feeds.

Abstract: Techniques are provided for automatically tuning a parameter in a layered model framework. One or more machine learning techniques are used to train multiple versions of a first model that includes a first version and a second version. A second model is stored that includes a parameter and accepts, as input, output from the first model. Multiple parameter values of the parameter are tested when processing content requests using the first and second versions of the first model. A strict subset of the plurality of parameter values are selected for the parameter of the second model, such that processing a first subset of the content requests using the first version of the first model results in a first value of a particular metric that matches a second value of the particular metric resulting from processing a second subset of the content requests using the second version of the first model.

Abstract: A method can include determining, based on learned parameter values, an intrinsic interest and an affinity for the user to be influenced to visit the website, determining, using the learned parameter values, intrinsic interest, and affinity for the user to be influenced to visit the website, a first probability indicating a likelihood that the user will, in response to viewing a badge notification, turn off notifications or delete an app and a second probability indicating a likelihood that the user will, in response to viewing the badge notification on the app, visit a website, in response to determining the second probability is greater than a threshold larger than the first probability, causing the app to include the badge notification when displayed on the user device.

Abstract: Technologies for optimizing content delivery to end-users are provided. Disclosed techniques include storing results of an online experiment with respect to a set of users and determining a plurality of distinct subsets of users based upon the results of the experiment. Users within each of the plurality of distinct subsets may be identified based on metric impacts of the online experiment. For each distinct subset and each associated model parameter, a utility value that represents effectiveness of the model parameter, with respect to an objective, may be determined. An objective optimization model may be used to automatically determine probabilities for each of the model parameters associated with each distinct subset. Users of a second set of users may be assigned to a distinct subset and associated model parameters may be applied to a content delivery strategies of the second set of users.

Abstract: Techniques for controlling item frequency using machine learning are provides. In one technique, two prediction models are trained: one based on interaction history of multiple content items by multiple entities and the other based on predicted interaction rates and an impression count for each of multiple content items. In response to a request, a particular entity associated with the request is identified and multiple candidate content items are identified. For each identified candidate content item, the first prediction model is used to determine a predicted interaction rate, an impression count of the candidate content item is determined with respect to the particular entity, the second prediction model is used to generate an adjustment based on the impression count, and an adjusted entity interaction rate is generated based on the predicted interaction rate and the adjustment. A particular candidate content item is selected based on the generated adjusted entity interaction rates.

Abstract: Techniques for accounting for position-specific differences in user interaction while conducting content item selection events are provided. In one technique, a position-specific factor is determined. The position-specific factor may be based on a ratio of an observed interaction and a predicted interaction. Different positions in a content item feed or on a web page may be associated with different position-specific factors. Eventually, multiple content items are identified for presentation on a screen of a computing device. The content items include a first content item for which a predicted interaction rate is calculated and a second content item for which no predicted interaction rate is calculated. An order of the content items is determined based on the position-specific factor. For example, the predicted interaction rate of the first content item is modified based on the position-specific factor. The content items are presented on the screen based on the order.

Abstract: Techniques for accounting for position-specific differences in user interaction while conducting content item selection events are provided. In one technique, a position-specific factor is determined. The position-specific factor may be based on a ratio of an observed interaction and a predicted interaction. Different positions in a content item feed or on a web page may be associated with different position-specific factors. Eventually, multiple content items are identified for presentation on a screen of a computing device. The content items include a first content item for which a predicted interaction rate is calculated and a second content item for which no predicted interaction rate is calculated. An order of the content items is determined based on the position-specific factor. For example, the predicted interaction rate of the first content item is modified based on the position-specific factor. The content items are presented on the screen based on the order.

Abstract: A system and method for content generation and targeting using machine learning are provided. In example embodiments, a probability that a user will visit a webpage based on historical data is calculated. A probability that the user will engage with a particular content category based on past user engagement is calculated. In response to the probability of the user engaging with the particular content category being equal to or greater than a first threshold, the content is generated. Further, in response to the probability of the user not visiting a webpage meeting or exceeding a second threshold, the generated content is sent to the user.

Abstract: Techniques are provided for automatically adjusting a resource reduction amount based on resource availability and other factors. The following are determined for a content delivery campaign: a winning distribution for a target audience of the content delivery campaign, a through rate distribution of the content delivery campaign, a resource allocation of the content delivery campaign, and an estimated number of content item selection events in which the content delivery campaign will participate in a future time period. Based on these values, a resource reduction amount is determined. An example of a resource reduction amount is an effective cost per impression. The resource reduction amount is used in one or more subsequent content item selection events in which the content delivery campaign participates.

Abstract: Techniques for automatically merging multiple content item queues are provided. In one technique, a first set of content items of a first type is identified. A second set of content items of a second type that is different than first type is identified. The first set of content items and the second set of content items are merged in a content item feed. Such merging involves, for a particular slot in the content item feed: determining a previous slot that contains a first content item from the first set; determining a number of slots between the previous slot and the particular slot; based on the number of slots, generating a score for a second content item from the second set; and based on the score, determining whether to insert, into the particular slot, the second content item or a third content item from the first set of content items.

Abstract: Techniques for automatically merging multiple content item queues are provided. In one technique, a first set of content items of a first type is identified. A second set of content items of a second type that is different than first type is identified. The the first set of content items and the second set of content items are merged in a content item feed. Such merging involves, for a particular slot in the content item feed: determining a previous slot that contains a first content item from the first set; determining a number of slots between the previous slot and the particular slot; based on the number of slots, generating a score for a second content item from the second set; and based on the score, determining whether to insert, into the particular slot, the second content item or a third content item from the first set of content items.

Abstract: Techniques for varying content item density are provided. A first minimum gap value is stored that dictates how close two content items of a first type may appear in a content item feed that contains content items of multiple types that includes the first type and a second type. The first minimum gap value is used to place content items in a first set of content item feeds. For each content item feed of the first set of content item feeds, performance data that indicates how well content items of the first type perform in the content item feed is generated. Based on the performance data and the first minimum gap value, a second minimum gap value that is different than the first minimum gap value is generated. The second minimum gap value is used to place content items in a second plurality of content item feeds.

Abstract: A notification platform for distribution of notification content in an on-line social network system addresses the technical problem of optimizing the volume of quality notifications that are being delivered to a given member. A notification delivery system is designed as a stream processing system that can fetch, store, and process data in a near-line fashion. It can perform feature generation, processing and scoring of notifications, as well as ranking of the notifications based on their respective relevance scores that are calculated using machine learning techniques. The notification delivery system is positioned centrally with respect to different producers of notifications, such that it can consume centrally-stored information about members' holistic notification experiences.