Abstract: In an example, a method is performed by sending query input and a first instruction to a generative machine learning model (GMLM). The first instruction is to cause the GMLM to generate and output a query input summary. The query input summary and a second instruction are sent to the GMLM. The second instruction is to cause the GMLM to provide query results using multiple different queries and the query input summary. The query results and a third instruction are to cause the GMLM to summarize a comparison of a query result with the query input summary. A query result evaluation summary is generated and output by the GMLM. A signal is received from a position within a presentation of the query result evaluation summary. The first instruction is updated to include the signal in the query input summary.

Abstract: An example sends query input and a first instruction to a generative machine learning model (GMLM). The first instruction is to cause the GMLM to generate and output a query input summary. The query input summary and a second instruction are sent to the GMLM. The second instruction is to cause the GMLM to provide query results using multiple different queries and the query input summary. The query results and a third instruction are to cause the GMLM to summarize a comparison of a query result with the query input summary. A query result evaluation summary is generated and output by the GMLM. A signal is received from a position within a presentation of the query result evaluation summary. The first instruction is updated to include the signal in the query input summary.

Abstract: An example provides a multi-agent system. Via an orchestrator agent, a query input is received. Via the orchestrator agent, the query input and a first instruction are provided to a generative machine learning model (GMLM). The first instruction is to cause the GMLM to determine queries using the query input. Via a query execution agent, the queries execute in parallel. Via the orchestrator agent, it is determined whether the queries are executing. Via a query evaluation agent, query results of execution of the queries and a second instruction are provided to the GMLM. The second instruction is to cause the GMLM to generate, for each query result, a query result summary. Via the orchestrator agent, the query result summaries are used to determine a subset of the query results for presentation via a device.

Abstract: In an example embodiment, a machine-learned model is trained to forecast a performance-based metric for a piece of content based on a budget applied to the piece of content. The increase in the performance-based metric that is due to a corresponding increase in budget may be termed “incrementality.” The machine-learned model is trained in such a way that incrementality is built into the model. More particularly, in an example embodiment, an asymmetric budget split process is used to create two groups of training data, one for high budget and one for low budget. Rather than relying on historical data, the asymmetric budget split process applies a high budget to a piece of content in a first subgroup (e.g., group of users) and a low budget to that same piece of content in a second subgroup, and then the performance results in each subgroup are compared.

Abstract: Described herein is a technique for generating data for a computer system. An apparatus comprises a high-speed communication bus, a memory unit communicatively coupled to the high-speed communication bus, an integrated circuit communicatively coupled to the high-speed communication bus, the integrated circuit to execute a machine learned model trained to receive input data and generate a set of implicit keywords from the input data to support a network service, and an implicit facet mapper to map the input data to an implicit facet of a knowledge graph, a database communicatively coupled to the high-speed communication bus, the database to store the set of implicit keywords and implicit facet in a data structure, and a network interface communicatively coupled to the high-speed communication bus to access a wireless network. Other embodiments are described and claimed.

Abstract: Sponsored and organic pieces of content are displayed in accordance with a blending model that is used to first identify a pattern of slots to which to assign either sponsored or organic pieces of content. This blending model is applied to a combination of both sponsored and non-sponsored pieces of content being considered for display to a user. This pattern only determines the slot assignments. The actual ranking of the pieces of content, and more particularly the actual ranking of the organic pieces of content, is determined by an ordering other than the ranking determined by the blending model, such as by using the original ordering of the second list. The pieces of content are then displayed in the order of this actual ranking, but in the slots indicated as having been assigned to be either sponsored or organic in the pattern determined by the blending model.

Abstract: Described herein is a technique for deriving and using a member profile-to-job posting graph that includes a plurality of implicit facets that map member profiles to job postings. Each implicit facet is a combination of member profile attributes linked with job profile attributes, such that a member profile is linked to the implicit facet when the member profile has member profile attribute values that match those of the implicit facet, and a job posting is linked to the implicit facet when the job posting has job attribute values that match those of the implicit facet. Using the graph, information about the potential applicant pool for a new job posting can be derived and presented to the person who is creating the new job posting.

Abstract: In an example embodiment, a machine-learned model is trained to forecast a performance-based metric for a piece of content based on a budget applied to the piece of content. The increase in the performance-based metric that is due to a corresponding increase in budget may be termed “incrementality.” The machine-learned model is trained in such a way that incrementality is built into the model. More particularly, in an example embodiment, an asymmetric budget split process is used to create two groups of training data, one for high budget and one for low budget. Rather than relying on historical data, the asymmetric budget split process applies a high budget to a piece of content in a first subgroup (e.g., group of users) and a low budget to that same piece of content in a second subgroup, and then the performance results in each subgroup are compared.

Abstract: Sponsored and organic pieces of content are displayed in accordance with a blending model that is used to first identify a pattern of slots to which to assign either sponsored or organic pieces of content. This blending model is applied to a combination of both sponsored and non-sponsored pieces of content being considered for display to a user. This pattern only determines the slot assignments. The actual ranking of the pieces of content, and more particularly the actual ranking of the organic pieces of content, is determined by an ordering other than the ranking determined by the blending model, such as by using the original ordering of the second list. The pieces of content are then displayed in the order of this actual ranking, but in the slots indicated as having been assigned to be either sponsored or organic in the pattern determined by the blending model.

Abstract: Techniques for the dynamic slotting of content items within electronic content are provided. In response to receiving a request, a first set of content items is identified and a first score is generated for each based on a first objective. Based on the first scores, a first ranking of the first set of content items is generated. A subset of the first set is selected based on the first ranking. A second set of content items that includes the subset is identified and a second score is generated for each item in the second set based on a second objective that is different than the first objective. Based on the second scores, a second ranking of the second set is generated. A subset of the second set is selected based on the second ranking. The second subset is presented on a computing device that is associated with the request.

Abstract: Technologies for determining performance gains for content item delivery based on modifications to content item selection parameters are provided. The disclosed techniques include implementing a multi-objective optimization model for content item selection using a value for a parameter. The model generates a first plurality of scores and a first ranking for content items. Subset of content items is selected for delivery based on the first ranking. New values for the parameter are identified and for each new value, the content item selection event is replayed. A second plurality of scores and a second ranking is generated for the content items, where the second ranking is different from the first ranking. A third plurality of scores and a third ranking is generated where the third ranking matches the second ranking. A set of gains is calculated for each new value, where each gain corresponds to a different objective of the model.

Abstract: Techniques for leveraging existing statistical prediction models are provided. A first statistical prediction model is generated for a content item. An instruction is received to create a clone from the content item. In response to receiving the instruction, the clone is created based on attributes of the content item. A second statistical prediction model that is different than the first statistical prediction model is generated for the clone. In response to receiving a request for content, the clone is identified as relevant to the first request. A similarity between (1) first content of the content item and (2) second content of the clone is determined. If the similarity exceeds a similarity threshold, then the first statistical prediction model is used to generate a prediction of an entity user selection rate associated with the clone. Otherwise, the second statistical prediction model is used to generate the prediction.

Abstract: The disclosed embodiments provide a system for performing dynamic job bidding optimization. During operation, the system obtains historical data containing a time series of interactions with a job. Next, the system uses the historical data to calculate an initial price of a job based on a predicted number of interactions with the job. The system then determines a first dynamic adjustment to the initial price that improves utilization of a budget for the job and a second dynamic adjustment to the initial price that improves a performance of the job. Finally, the system applies the first and second adjustments to the initial price to produce an updated price for the job and delivers the job within an online system based on the updated price.

Abstract: Techniques for the dynamic slotting of content items within electronic content are provided. In response to receiving a request, a first set of content items is identified and a first score is generated for each based on a first objective. Based on the first scores, a first ranking of the first set of content items is generated. A subset of the first set is selected based on the first ranking. A second set of content items that includes the subset is identified and a second score is generated for each item in the second set based on a second objective that is different than the first objective. Based on the second scores, a second ranking of the second set is generated. A subset of the second set is selected based on the second ranking. The second subset is presented on a computing device that is associated with the request.

Abstract: Instead of a fixed fee for a particular job application, discussed in some examples are methods, systems, and machine readable mediums which provide for a job posting service that utilizes a pay-per-click model. That is, job posters pay a fee for each time the member selects the job posting for further inquiry when that posting is shown to a member (called an impression). The fee that is paid is determined by the job poster. Selecting a job posting may comprise clicking on or otherwise entering an input signifying an intention to view the job.

Abstract: Technologies for determining performance gains for content item delivery based on modifications to content item selection parameters are provided. The disclosed techniques include implementing a multi-objective optimization model for content item selection using a value for a parameter. The model generates a first plurality of scores and a first ranking for content items. Subset of content items is selected for delivery based on the first ranking. New values for the parameter are identified and for each new value, the content item selection event is replayed. A second plurality of scores and a second ranking is generated for the content items, where the second ranking is different from the first ranking. A third plurality of scores and a third ranking is generated where the third ranking matches the second ranking. A set of gains is calculated for each new value, where each gain corresponds to a different objective of the model.

Abstract: The disclosed embodiments provide a system for performing calibration of response rates. During operation, the system obtains a position of a content item in a ranking of content items generated for delivery to a member of an online system and a predicted response rate by the member to the content item. Next, the system determines an updated response rate by the member to the content item based on the position of the content item in the ranking and dimensions associated with the predicted response rate and the ranking. The system then outputs the updated response rate for use in managing delivery of the content item.

Abstract: The disclosed embodiments provide a system that performs pacing for balanced delivery. During operation, the system obtains predicted response rates associated with impressions of a content item delivered within an online system and a cost per action (CPA) for the content item. Next, the system determines an impression-based spending for the content item based on the predicted response rates and the CPA. The system then calculates a pacing score for the content item based on the impression-based spending. Finally, the system adjusts subsequent interactions with the content item based on the pacing score.

Abstract: The disclosed embodiments provide a system for performing an experiment strategy for member-based ramping. During operation, the system divides members of an online system into a control group and a treatment group. Next, the system configures delivery of content to the control group over a first set of channels that charge for actions related to the content and delivery of the content to the treatment group over a second set of channels that charge for the actions related to the content, wherein the second set of channels is smaller than the first set of channels. The system then performs a first experiment that selects an adjustment factor for content delivered over the second set of channels to the treatment group to achieve revenue neutrality and/or engagement neutrality between the treatment and control groups.

Abstract: Methods, systems, and computer programs are presented for determining presentation options, associated with a job posting, related to how well the member's profile matches the job posting characteristics. One method includes performing a job search for a member of an online service that returns jobs for presentation to the member, and identifying a job posting from the job postings. The method further includes determining a job-matching score that indicates a degree of matching between a profile of the member and characteristics of the identified job. Further yet, the method checks if the job-matching score is above a matching threshold and the identified job is presented, in a user interface, with a message based on the checking. The message is based on the degree of matching between the profile of the member and the characteristics of the job.