Abstract: An embedding model is trained to learn latent representations of users describing information related to conditional treatment effect for users relative to different potential treatments. The user embeddings may be used to determine the types of situations in which a user responds differently to different conditions or situations. To train this model, a plurality of experiments with users may be performed to determine user responses to different treatment conditions in the experiments. The conditional treatment effect for users in the experiments may be determined, e.g., with counterfactual predictions of a treatment not experienced by a user in the experiment. The embedding model may be trained with decoders that each predict the conditional treatment effect with respect to one of the experiments, enabling a loss for each experiment with respect to the conditional treatment effect to jointly train the embedding model.

Abstract: A computer system allowing users to search for items of interest provides a search query interface. The system receives characters of a search query in the search interface as the user enters the characters and interactively calculates, ranks, and displays a set of possible search query options from which the user can select. To rank the set of possible search query options, the system modifies rankings of candidate search queries based on factors associated with third parties. More specifically, contextual relevance scores are computed for the candidate search queries based on the context, such as a user to whom the search results are provided. These contextual relevance scores are in turn adjusted using factors associated with third parties, such as values calculated based on consideration offered by third parties. Users are shown the search query options, ranked in order of the adjusted relevance scores, as possible query selections.

Abstract: An online system uses a trained machine-learning model for dynamically modifying an authorization buffer amount to cover additional expenses occurring during fulfillment of an online order. Upon receiving a signal indicating that a user entered an online checkout stage of the order, the online system applies the machine-learning model to generate a set of values of a metric for a set of authorization buffer amounts, each value of the metric resulting from charging the user a respective authorization buffer amount over an expected value of the order if a value of the order at delivery is greater than the expected value. The online system selects an authorization buffer amount resulting in the largest value of the metric, and generates an authorization signal that authorizes charging the user the authorization buffer amount over the expected value if the value of the order is greater than the expected value.

Abstract: A computing system automatically selects treatments for users by generating a propensity vector for a set of treatments and selecting a treatment based on the propensity vector. The propensity vector is determined based on one or more computer models that predict user actions responsive to the treatments and the propensity vector is determined based on the value of a treatment parameter. The treatment parameter is perturbed to determine an adjusted propensity vector. Treatments are applied and outcomes determined with the propensities determined by the current value of the treatment parameter, and counterfactuals for the adjusted treatment vector are determined to evaluate the effect of modifying the treatment parameter. When the perturbed treatment parameter value yields improved results in the counterfactual, the current value is modified to improve performance of the model as a whole without requiring retraining of underlying predictive models.

Abstract: An online system sends content items for display to client devices associated with users and detects actions associated with the content items performed by the users. The system accesses and applies machine-learning models to predict metrics for a set of users and generates a set of optimal values for the set of users based on the metrics, one or more objectives, and a set of constraints, in which each optimal value indicates whether a user is eligible to be presented with a content item. Responsive to identifying an opportunity to present the content item to a user of the set of users, the system determines whether the user is eligible to be presented with the content item based on an optimal value determined for the user and sends the content item for display to a client device associated with the user if the user is eligible.

Abstract: An embedding model is trained to learn latent representations of users describing information related to conditional treatment effect for users relative to different potential treatments. The user embeddings may be used to determine the types of situations in which a user responds differently to different conditions or situations. To train this model, a plurality of experiments with users may be performed to determine user responses to different treatment conditions in the experiments. The conditional treatment effect for users in the experiments may be determined, e.g., with counterfactual predictions of a treatment not experienced by a user in the experiment. The embedding model may be trained with decoders that each predict the conditional treatment effect with respect to one of the experiments, enabling a loss for each experiment with respect to the conditional treatment effect to jointly train the embedding model.

Abstract: A system artificial intelligence (AI) agent is trained to act on behalf of an online system. The system AI agent comprises a large language model that has been pre-trained using a set of system constraints and a set of system objectives. The system AI agent is trained adversarially using training service requests from a plurality of different user AI agents of different types to determine resolutions to the training service requests. Once trained, the system AI agent may determine resolutions to service requests of users of the online system. In some embodiments, the system agent may determine the resolutions via messaging with user AI agents that represent the users. The online system may further train the system AI agent (and in some embodiments the user AI agents) based in part on the resolutions to the service requests.

Abstract: An online system displays items to a user in search results based on appeasement scores for the items, adjusted according to how specific the search query is. The online system receives a search query from a user of an online system. The online system computes a query specificity score, a measure of the specificity of the search query. The online system accesses candidate items from a database that potentially match the search query. For each candidate item, the online system may compute or predict an appeasement score. The online system adjusts the appeasement score based on the query specificity score such that a more specific query weights the appeasement score lower than a less specific query. The online system may then compute a ranking score based on the adjusted appeasement score and display the candidate items to the user based on their ranking scores.

Abstract: A system generates item images using an item image generation model. The system receives a prompt for the model. The prompt is configured to request the model generate item images for an item. The system executes the model using the prompt to generate a set of item images. The system evaluates each of the set of item images to determine performance data of each of the set of item images. The system iteratively improves the set of item images by performing the following steps. The system updates the prompt based on the performance data of each of the set of item images to obtain a new prompt. The system executes, using the new prompt, the model to generate a new set of item images, and the system evaluates the new set of item images to determine performance data of each of the new set of item images.

Abstract: An online system displays items to a user in search results based on appeasement scores for the items, adjusted according to how specific the search query is. The online system receives a search query from a user of an online system. The online system computes a query specificity score, a measure of the specificity of the search query. The online system accesses candidate items from a database that potentially match the search query. For each candidate item, the online system may compute or predict an appeasement score. The online system adjusts the appeasement score based on the query specificity score such that a more specific query weights the appeasement score lower than a less specific query. The online system may then compute a ranking score based on the adjusted appeasement score and display the candidate items to the user based on their ranking scores.

Abstract: An online system maintains a shared cache for storing actions assigned to users in different experiment groups. The system receives an indication that a user interacted with an online system, and data associated with the user. The system generates a set of propensities for a set of actions by identifying a first set of features for the user, accessing a first machine learning model, and applying the first machine learning model to the first set of features. The system selects an action based on the set of propensities and presents the action to the user. The system updates a cache of a set of user data and includes the transmitted action. The system receives a second indication and accesses a database to determine a selected action stored in association to the user. The system presents the selected action for a second time to the user.

Abstract: A method for predicting customer long-term behavior using LLM-based modeling is described. The online system receives a representation of a stimulus or treatment that is presented to a user and generates a summary of a simulated user profile. The online system performs an inference task in conjunction with the model serving system or interface system to infer one or more actions that will likely be performed in response to the representation of the stimulus based on the simulated user profile. The online system computes a surrogate measure based on the response received from the model serving system and computes a correlation coefficient between the surrogate measure and a true metric of interest from collected experiment data. Responsive to determining a correlation coefficient greater than a threshold value, the online system predicts the true metric of interest based on the surrogate measure.

Abstract: An online system sends content items for display to client devices associated with users and detects actions associated with the content items performed by the users. The system accesses and applies machine-learning models to predict metrics for a set of users and generates a set of optimal values for the set of users based on the metrics, one or more objectives, and a set of constraints, in which each optimal value indicates whether a user is eligible to be presented with a content item. Responsive to identifying an opportunity to present the content item to a user of the set of users, the system determines whether the user is eligible to be presented with the content item based on an optimal value determined for the user and sends the content item for display to a client device associated with the user if the user is eligible.

Abstract: An online system uses Large Language Models (LLM's) to simulate the behavior of real customers in such tests as a low-cost way to simulate A/B tests. A prompt is constructed for a given customer. The prompt is sent to the LLM with a request to infer the predicted outcome of a treatment. The online system collects the output. Statistical analyses are run based on the output of the previous step to determine which treatment to select for the user.

Abstract: A ranking computer model is trained based on grouping a collection of users of an online system into different buckets based on intended likelihoods of presenting a set of content items to the collection of users, wherein a contextual bandit model is employed to compute the intended likelihoods. The online system applies the ranking computer model to generate, based on user data for a user of the online system and contextual data associated with a current session of the user, a ranking score for each content item in a set of content items. The online system selects, based on the ranking score for each content item, one or more content items from the set of content items. The online system causes a device associated with the user to display a user interface with the one or more content items for recommendation to the user.

Abstract: A system generates a set of embeddings for known treatments by applying a machine-learned embedding model to descriptions of the known treatments, where these embeddings form a vector space. The system generates an embedding for a new treatment and mapping it within the vector space, and identifies one or more known treatments with embeddings that exceed a similarity threshold with the new treatment embedding. The system accesses performance data for the selected known treatments to assess user response, and identifies a subset of users for the new treatment based on this performance data. The system also creates a content item that incorporates the new treatment, and transmits instructions to client devices of the targeted users to cause the client devices to display the content item.

Abstract: A system generates item images using an item image generation model. The system receives a prompt for the model. The prompt is configured to request the model generate item images for an item. The system executes the model using the prompt to generate a set of item images. The system evaluates each of the set of item images to determine performance data of each of the set of item images. The system iteratively improves the set of item images by performing the following steps. The system updates the prompt based on the performance data of each of the set of item images to obtain a new prompt. The system executes, using the new prompt, the model to generate a new set of item images, and the system evaluates the new set of item images to determine performance data of each of the new set of item images.

Abstract: A computer system allowing users to search for items of interest provides a search query interface. The system receives characters of a search query in the search interface as the user enters the characters and interactively calculates, ranks, and displays a set of possible search query options from which the user can select. To rank the set of possible search query options, the system modifies rankings of candidate search queries based on factors associated with third parties. More specifically, contextual relevance scores are computed for the candidate search queries based on the context, such as a user to whom the search results are provided. These contextual relevance scores are in turn adjusted using factors associated with third parties, such as values calculated based on consideration offered by third parties. Users are shown the search query options, ranked in order of the adjusted relevance scores, as possible query selections.

Abstract: An online system displays items to a user in search results based on appeasement scores for the items, adjusted according to how specific the search query is. The online system receives a search query from a user of an online system. The online system computes a query specificity score, a measure of the specificity of the search query. The online system accesses candidate items from a database that potentially match the search query. For each candidate item, the online system may compute or predict an appeasement score. The online system adjusts the appeasement score based on the query specificity score such that a more specific query weights the appeasement score lower than a less specific query. The online system may then compute a ranking score based on the adjusted appeasement score and display the candidate items to the user based on their ranking scores.

Abstract: A computing system automatically selects treatments for users by generating a propensity vector for a set of treatments and selecting a treatment based on the propensity vector. The propensity vector is determined based on one or more computer models that predict user actions responsive to the treatments and the propensity vector is determined based on the value of a treatment parameter. The treatment parameter is perturbed to determine an adjusted propensity vector. Treatments are applied and outcomes determined with the propensities determined by the current value of the treatment parameter, and counterfactuals for the adjusted treatment vector are determined to evaluate the effect of modifying the treatment parameter. When the perturbed treatment parameter value yields improved results in the counterfactual, the current value is modified to improve performance of the model as a whole without requiring retraining of underlying predictive models.