Abstract: An online system generates text-based representations of various types of data for processing using a large language model. The online system extracts location data from a map of a source location and converts the location data into a text-based representation of the location data. The online system receives a set of item identifiers from a client device of a user and generates an LLM prompt based on the set of item identifiers and the text-based representations of the location data. The online system receives a response from the LLM and parses the response for a text-based description of related items. The online system maps the text-based description of the related items to item identifiers and transmits a notification to the client device that includes item data associated with the related items.

Abstract: An online system receives, from a computing device associated with a servicing user, an indication that items are collected for an order. The system identifies a runover by determining that a total value of the items collected for the order is greater than an expected value of the order. The system performs runover verification by: identifying features from the order, the items collected, and the runover, applying a remediation time prediction model to the features to predict an amount of time to remediate the runover, applying an uplift model to the features to predict a differential loss between a loss associated with rejecting the runover and a loss associated with verifying the runover, and verifying or rejecting the runover based on the predicted amount of time to remediate and the predicted differential loss. The online system transmits a notification indicating the verification results.

Abstract: An online concierge system receives location information associated with pickers and actual orders associated with a geographical zone. A model trained to predict a likelihood an actual order associated with the zone will be available for servicing within a timeframe is accessed and applied to forecasted orders. Each picker is matched to an order for servicing by minimizing a value of a function that is based on a difference between a location associated with each picker matched to an actual order and an associated retailer location, a difference between the location associated with each picker matched to a forecasted order and an associated retailer location, and the predicted likelihood. Recommendations for accepting an actual order, moving to a retailer location associated with a forecasted order, or checking back later with the system are generated based on the matches and sent for display to a client device associated with each picker.

Abstract: An online concierge system receives location information associated with pickers and actual orders associated with a geographical zone. A model trained to predict a likelihood an actual order associated with the zone will be available for servicing within a timeframe is accessed and applied to forecasted orders. Each picker is matched to an order for servicing by minimizing a value of a function that is based on a difference between a location associated with each picker matched to an actual order and an associated retailer location, a difference between the location associated with each picker matched to a forecasted order and an associated retailer location, and the predicted likelihood. Recommendations for accepting an actual order, moving to a retailer location associated with a forecasted order, or checking back later with the system are generated based on the matches and sent for display to a client device associated with each picker.

Abstract: An online system predicts time to park at a fulfillment location in fulfillment of an order by a fulfillment user. The online system receives an order from a requesting user, and applies a timeliness prediction model to the order, the parking configuration of the corresponding fulfillment location, to other contextual factors, or some combination thereof to predict the time to park at the fulfillment location. The timeliness prediction model is trained on historical orders with their associated completion times and known parking configurations of the respective fulfillment locations. The online system may batch orders together to optimize fulfillment efficiency in consideration of the predicted lag time for the order. The online system assigns and transmits the batches to fulfillment users to fulfill at the fulfillment locations.

Abstract: An online system uses a computer-vision item identification model to identify items and physical containers storing those items to detect sorting errors of the physical containers. The online system receives a first image from a client device that depicts a set of physical containers that contain items for a batch of orders that the online system has received. The online system identifies items in those physical containers by applying a contained-item identification model to the first image. The online system uses the output of this model to determine which visible items are in each physical container and uses that information plus order data for the batch of orders to determine which physical containers are associated with each order. The online system compares this first image to a subsequently received image to determine whether the correct physical containers were delivered by the user.

Abstract: An online concierge system trains a computer model to map receipt item labels to order item identifiers, enabling the online concierge system to identify discrepancies between receipt items and customer order items. The online concierge system identifies a training set of data comprising quantities of receipt item labels and corresponding orders having quantities of order item identifiers and trains the computer model to predict quantities of order item identifiers based on the set of training data. The online concierge system applies a one-hot encoding for a receipt item label to determine predicted order item identifiers and maps the receipt item label to order item identifiers based on the predictions.

Abstract: A trained model is used to predict and prevent a failed delivery of an order placed by a user of an online system. The online system accesses a delivery prediction model trained to predict a likelihood of a delivery for the order ending up as a failed delivery as the order would not be delivered at a location associated with the user. The online system applies the delivery prediction model to predict, based on order data, user data and fulfillment data, the likelihood of the failed delivery for the order. Responsive to the predicted likelihood of the failed delivery being greater than a threshold value, the online system identifies one or more actions associated with the order to prevent an occurrence of the failed delivery for the order. The online system applies the one or more actions to prevent the occurrence of the failed delivery for the order.

Abstract: An online system includes an interface which facilitates communication between customers and pickers who are servicing the user's order. The customer may request a modification to their order through the interface. The online system performs an inference task in conjunction with the model serving system or the interface system to continuously monitor conversations between users and pickers to infer whether a customer requested to modify their order to maintain an updated order and an updated in-store transaction estimate for the order. The online system determines if the order has been updated to account for the requested changes. If the order has not been updated, the online system automatically updates the customer's order and computes an updated in-store transaction estimate based on the changes made.

Abstract: An online system receives information describing a set of items requested by a user and an indication via a chat interface that a particular item needs replacement. The online system generates one or more prompts configured to request a machine learned language model to identify the particular item that needs replacement and to identify one or more replacement items for the particular item. The online system receives a set of item identifiers from the machine learned language model and selects a replacement item from a database based on the set of item identifiers. The online system may also receive an order and a communication history associated with a user including a message with a request to modify the a. The online uses the machine-learning language model to map the request type to the set of API requests for updating the order to reflect the request from the user.

Abstract: An online system generates text-based representations of various types of data for processing using a large language model. The online system extracts location data from a map of a source location and converts the location data into a text-based representation of the location data. The online system receives a set of item identifiers from a client device of a user and generates an LLM prompt based on the set of item identifiers and the text-based representations of the location data. The online system receives a response from the LLM and parses the response for a text-based description of related items. The online system maps the text-based description of the related items to item identifiers and transmits a notification to the client device that includes item data associated with the related items.

Abstract: An online system may prompt a shopper to capture one or more images of items on a checkout belt of a retailer, wherein the items are for fulfilling orders for one or more users of an online service. An online system may provide the one or more images to a machine learning model configured to classify an item as a product. An online system may classify the items to one or more products by applying the machine learning model to the images. An online system may for each user, matching the classified products to the user's order. An online system may obtain an annotated image of the items highlighting classified products which do not match the user's order. An online system may provide to the shopper the annotated image with a notification of a potential discrepancy.

Abstract: An online system detects an anomaly associated with an item selection made by a picker for fulfilling an order of a user of an online system. The system generates a prompt for execution by a machine-learned model trained as a large language model. The prompt comprises a chat log between the picker and the user. The system provides the prompt to the machine-learned model for execution. The system receives, as output from the machine-learned model and based on the chat log, a description indicating whether the anomaly is attributable to the user. The system determines, based on the output from the machine-learned model, that the item selection is not attributable to the user. Responsive to determining that the item selection is not attributable to the user, the system provides a notification to a client device of the user to confirm whether the item selection is approved by the user.

Abstract: An online system receives a request to confirm a transaction that is associated with an order. The system accepts or declines the transaction based on whether an amount associated with the pending transaction is likely to exceed an expected amount of the order by more than a threshold value. To determine the threshold, the system trains a first model to predict an overspend for an order and then trains a second model to predict an amount of error associated with the predictions from the first model. The outputs of the first model and the second model provide a mean and a variance for an expected distribution of the overspend. If the actual overspend amount for the transaction exists in too high of a percentile of the distribution, the transaction may be flagged for review or declined.

Abstract: An online concierge system receives location information associated with pickers and actual orders associated with a geographical zone. A model trained to predict a likelihood an actual order associated with the zone will be available for servicing within a timeframe is accessed and applied to forecasted orders. Each picker is matched to an order for servicing by minimizing a value of a function that is based on a difference between a location associated with each picker matched to an actual order and an associated retailer location, a difference between the location associated with each picker matched to a forecasted order and an associated retailer location, and the predicted likelihood. Recommendations for accepting an actual order, moving to a retailer location associated with a forecasted order, or checking back later with the system are generated based on the matches and sent for display to a client device associated with each picker.