Abstract: A smart shopping cart may utilize cameras and/or load sensors to provide capacity-informed recommendations. The cameras are positioned facing at least a first basket of the smart shopping cart and configured to capture image data during a visit at a retailer location. The load sensors are configured to measure load data during a visit at the retailer location. The cart detects obtained items entering the first basket based on the image data and the load data. The cart identified remaining capacity in the first basket based on the image data and the load data. The cart applies a capacity-informed model to the one or more obtained items and the remaining capacity in the first basket to identify one or more recommended items. The cart displays, via an electronic display, the one or more recommended items.

Abstract: A system or a method for intelligently routing user inquiries to knowledgeable retail shoppers using machine learning. Upon receiving an inquiry from a client device that includes both text and image content, the system applies machine learning models to identify item categories referenced in the text and shown in the image. The system uses an item availability model—trained on historical retailer inventory data—to identify a retailer likely to carry items in the identified categories and transmits suggestion information to the user's device, prompting a user interface that recommends the retailer. The system selects a shopper associated with the retailer who has subject matter expertise in the relevant item categories. Expertise is determined using a machine-learned model trained on labeled data from historical shopper orders. The system then forwards the user inquiry to the expert shopper's device, enabling direct communication and facilitating more accurate, personalized retail assistance.

Abstract: A system for taking actions based on user interactions receives, from a computer system, image data of a document depicting one or more items. The system may use computer vision models to identify a user interaction with the document in the image data and determine an item depicted by the document that corresponds to the user interaction. The system detects a gesture that occurred based on the user interaction and the item. The system correlates the gesture with the identified item and causes the computer system to invoke a function corresponding to the gesture and identified item.

Abstract: An online concierge system uses a findability machine-learning model to predict the findability of items within a physical area. The findability model is a machine-learning model that is trained to compute findability scores, which are scores that represent the ease or difficulty of finding items within a physical area. The findability model computes findability scores for items based on an item map describing the locations of items within a physical area. The findability model is trained based on data describing pickers that collect items to service orders for the online concierge system. The online concierge system aggregates this information across a set of pickers to generate training examples to train the findability model. These training examples include item data for an item, an item map data describing an item map for the physical area, and a label that indicates a findability score for that item/item map pair.

Abstract: A cart management system generates an error priority assignment for smart cart systems based on device error predictions for those smart cart systems. An error priority assignment is an assignment of the relative priority of servicing or providing maintenance to a set of smart cart systems. To generate the error priority assignment, the cart management system applies an error detection model to cart data received from the set of smart cart systems. The cart data has measurements captured by sensors coupled to the smart cart systems, and the error detection model uses the cart data to generate device error predictions. Each of these predictions represents a likelihood that a smart cart system will experience a device error within some time period. The cart management system uses the device error predictions to generate the error priority assignment and selects which smart cart system to service based on the error priority assignment.

Abstract: An online system trains a ceiling prediction model to determine a user's ceiling for one or more item categories. The user's ceiling for an item category is a maximum amount of an item within the item category the user is likely to include in an order. Based on previously fulfilled orders for the user, information describing a current order from the user, and contextual information about the order, the ceiling prediction model determines the user's ceiling for an item category. The online system leverages the user's ceiling for an item category to refine content about different items that is selected for presentation to a user. For example, the online system determines whether the order includes a quantity of items from an item category that equals the user's ceiling for the item category when determining which items to present to the user.

Abstract: An online system integrates an artificial intelligence (AI) agent to communicate with users. Upon obtaining a set of candidate items (e.g., promotional items), the online system applies a machine-learning model to generate a score for each candidate item that is indicative of a likelihood of a user converting on each candidate item. Based on the scores, the online system selects an item from the candidate items for user's recommendation. The online system prompts the AI agent with information about the item, user's features, content of a current order, and inputs from an online platform (e.g., bid for the item) to generate a pitch for the item. The online system then generates a user interface signal causing a user's device to display a user interface with a description of the item, provide the pitch for the item, and display a user interface element for adding the item to the current order.

Abstract: A smart shopping cart may utilize cameras to identify fulfillment instructions to maximize efficiency. A fulfillment user may be tasked to fulfill a batch of orders at a retailer location. Each order includes one or more items to be obtained. The cart captures image data via one or more cameras in view of the cart's baskets. From the image data, the cart can detect obtained items placed in the baskets and can generate an occupancy state of the baskets indicating a configuration of each obtained item in the baskets. The cart can apply a fulfillment optimization model to the occupancy state to identify a next item to be obtained in the batch of orders and an optimal packing configuration for the next item. The cart can display to the fulfillment user the next item and the optimal packing configuration.

Abstract: An online system integrates an artificial intelligence (AI) agent to communicate with users. Upon obtaining a set of candidate items (e.g., promotional items), the online system applies a machine-learning model to generate a score for each candidate item that is indicative of a likelihood of a user converting on each candidate item. Based on the scores, the online system selects an item from the candidate items for user's recommendation. The online system prompts the AI agent with information about the item, user's features, content of a current order, and inputs from an online platform (e.g., bid for the item) to generate a pitch for the item. The online system then generates a user interface signal causing a user's device to display a user interface with a description of the item, provide the pitch for the item, and display a user interface element for adding the item to the current order.

Abstract: An online concierge system selectively replaces default static item displays with dynamic item displays to represent items. The dynamic item displays encourage a viewing user of the online concierge system to purchase the items and may be selected based on item or user preferences or characteristics. The online concierge system applies a machine learning model to determine display scores describing the expected benefit of dynamic item displays and bandwidth scores describing resource usage of dynamic item displays. The online concierge system selectively replaces default static item displays with dynamic item displays based on the display and bandwidth scores so as to maximize benefit while ensuring that performance of the online concierge system is not negatively impacted by the resource usage.

Abstract: An online system leverages a cart issue prediction model trained as a machine-learning model to identify predicted issues with the operation of a smart cart based on sensor data captured by sensors on the smart cart. The machine-learning model is trained on historical data related to the operation of a fleet of smart carts. In response to identifying any cart issues, the online system can trigger a remedial workflow to remedy the predicted issues. The online system may transmit command signals to the smart cart to calibrate the sensors, may transmit remedial tasks to a client device of the user for prompting the user to aid in remedying the predicted issues, or may schedule a service appointment with a technician, including an issue report indicating the predicted issues.

Abstract: An online concierge system receives, from a client device associated with a user, a request to access a user interface including a listing of sources associated with the system, in which each source is associated with a catalog of items. The system retrieves user data describing interactions by the user with items available at the sources and accesses and applies a machine-learning model to predict a user engagement score for each item-source pair associated with the sources based on the user data, in which the score indicates a likelihood of an interaction by the user with an item available at a source. The system selects a set of item-source pairs based on the scores and generates the user interface including the listing and a selectable option to add an item associated with each selected pair to a shopping list. The system then sends the user interface to the client device.

Abstract: The online system suggests replacement items to a user when an ordered item is unavailable. To improve the replacement suggestions for a given user, the online system obtains a set of generalized parameters for the user, where a generalized parameter provides guidance for replacing multiple different types of items. When the online system is fulfilling an order and identifies that an item in the order is unavailable, the online system uses a machine-learning model to score various candidate replacement items. The candidate replacement items can be obtained by prompting a generative model (e.g., language model) with one or more generalized parameters for the user that are relevant to the item. The generative model can be further prompted to adjust the scores of the candidate replacement items. Based on the adjusted scores, the online system selects a replacement item, and then presents the selected replacement item to the user.

Abstract: The methods and systems of the present disclosure may: receive data indicating: one or more items to be purchased by a customer from a retailer associated with an online shopping concierge platform, and one or more items purchased by the customer from the retailer; identify one or more items offered by the retailer for subsequent consideration by the customer; determine, based at least in part on one or more machine learning (ML) models, one or more values for each item representing a likelihood that the customer will purchase the item if subsequently presented with the opportunity under specified conditions; generate data describing a graphical user interface (GUI) comprising a ranked listing of at least a portion of the item(s); and communicate the data describing the GUI such that a computing device associated with the customer renders and displays the listing.

Abstract: An online concierge system receives, from a client device associated with a user, a request to access a user interface including a listing of sources associated with the system, in which each source is associated with a catalog of items. The system retrieves user data describing interactions by the user with items available at the sources and accesses and applies a machine-learning model to predict a user engagement score for each item-source pair associated with the sources based on the user data, in which the score indicates a likelihood of an interaction by the user with an item available at a source. The system selects a set of item-source pairs based on the scores and generates the user interface including the listing and a selectable option to add an item associated with each selected pair to a shopping list. The system then sends the user interface to the client device.

Abstract: A cart management system generates an error priority assignment for smart cart systems based on device error predictions for those smart cart systems. An error priority assignment is an assignment of the relative priority of servicing or providing maintenance to a set of smart cart systems. To generate the error priority assignment, the cart management system applies an error detection model to cart data received from the set of smart cart systems. The cart data has measurements captured by sensors coupled to the smart cart systems, and the error detection model uses the cart data to generate device error predictions. Each of these predictions represents a likelihood that a smart cart system will experience a device error within some time period. The cart management system uses the device error predictions to generate the error priority assignment and selects which smart cart system to service based on the error priority assignment.

Abstract: An online system trains a ceiling prediction model to determine a user's ceiling for one or more item categories. The user's ceiling for an item category is a maximum amount of an item within the item category the user is likely to include in an order. Based on previously fulfilled orders for the user, information describing a current order from the user, and contextual information about the order, the ceiling prediction model determines the user's ceiling for an item category. The online system leverages the user's ceiling for an item category to refine content about different items that is selected for presentation to a user. For example, the online system determines whether the order includes a quantity of items from an item category that equals the user's ceiling for the item category when determining which items to present to the user.

Abstract: A trained model is used to generate market adjustment recommendations for a retailer associated with an online system. Upon displaying an item to a user of the online system for replacing an originally requested item and collecting user's engagement data in relation to the replacement item, the online system accesses a market adjustment model that is trained to generate a score for the user indicative of an affinity of the user in relation to the replacement item and generate one or more market adjustment recommendations for the retailer. The online system applies the market adjustment model to generate, based on the engagement data, behavioral information of the user and/or contextual information associated with the user, the score for the user and the one or more market adjustment recommendations for the retailer. The online system provides the one or more market adjustment recommendations to a computing system associated with the retailer.

Abstract: An online system leverages a cart issue prediction model trained as a machine-learning model to identify predicted issues with the operation of a smart cart based on sensor data captured by sensors on the smart cart and/or feedback from a user of the smart cart. The machine-learning model is trained on historical data related to the operation of a fleet of smart carts. In response to identifying any cart issues, the online system can trigger a remedial workflow to remedy the predicted issues. The online system may transmit command signals to the smart cart to calibrate the sensors, may transmit remedial tasks to a client device of the user for prompting the user to aid in remedying the predicted issues, or may schedule a service appointment with a technician, including an issue report indicating the predicted issues.

Abstract: An online system uses a trained machine-learning model to predict a perception of a user about an expiration date of an item. Upon receiving an item signal including information about the expiration date, the online system applies the machine-learning model to the item signal, information about the user, and information about the item to generate a perception score indicative of a likelihood that the user will perceive the expiration date as unacceptable. Based on the perception score and the information about the item, the online system identifies a second item for replacing the item, the second item having a second expiration date that is later than the expiration date. The online system generates, using information about the item and the second item, a user interface signal that causes a user interface to display a notification about the expiration date and a recommendation for replacing the item with the second item.