Abstract: A shopping cart's tracking system determines a first baseline location of the shopping cart at a first timestamp with a wireless device located on the shopping cart detecting one or more external wireless devices (e.g., RFID tags) in the indoor environment. The shopping cart's tracking system receives wheel motion data from one or more wheel sensors coupled to one or more wheels of the shopping cart, wherein the wheel motion data describes rotation of the one or more wheels. The shopping cart's tracking system calculates a translation traveled by the shopping cart from the first baseline location based on the wheel motion data. The shopping cart's tracking system determines an estimated location of the shopping cart at a second timestamp based on the first baseline location and the translation. With the estimated location, the shopping cart can update a map with the estimated location of the shopping cart.

Abstract: A warehouse from which shoppers fulfill orders for an online concierge system maintains an online concierge system-specific portion for which the online concierge system specifies placement of items in regions. To place items in the online concierge system-specific portion, the online concierge system accounts for co-occurrences of different items in orders and measures of similarity between different items. From the co-occurrences of items, the online concierge system generates an affinity graph. The online concierge system also generates a colocation graph based on distances between different regions in the online concierge system-specific portion. Using an optimization function with the affinity graph and the colocation graph, the online concierge system selects regions within the online concierge system-specific portion for different items to minimize an amount of time for shoppers to obtain items in the online concierge-system specific portion.

Abstract: An online concierge system maintains various items and an item embedding for each item. When the online concierge system receives a query for retrieving one or more items, the online concierge system generates an embedding for the query. The online concierge system trains a machine-learned model to determine a measure of relevance of an embedding for a query to item embeddings by generating training data of examples including queries and items with which users performed a specific interaction. The online concierge system generates a subset of the training data including examples satisfying one or more criteria and further trains the machine-learned model by application to the examples of the subset of the training data and stores parameters resulting from the further training as parameters of the machine-learned model.

Abstract: For each retailer in the geographic region, an online system predicts a number of orders placed at the retailer and a capacity to fulfill orders during a forecast time period. The capacity of the retailer is predicted based on a number of pickers expected to be available to the retailer during the forecast time period. The online system determines demand for the services of a picker at the retailer based on a comparison of the predicted number of orders and the predicted capacity to fulfill those orders. The online system displays a user interactive map of the geographic region to the picker. The map displays a pin at the location of each retailer in the geographic region, which describes the categorization determined for the retailer. The picker selects a pin, which causes the user interactive map to display a notification characterizing the demand for services at the retailer.

Abstract: An item recognition system uses a top camera and one or more peripheral cameras to identify items. The item recognition system may use image embeddings generated based on images captured by the cameras to generate a concatenated embedding that describes an item depicted in the image. The item recognition system may compare the concatenated embedding to reference embeddings to identify the item. Furthermore, the item recognition system may detect when items are overlapping in an image. For example, the item recognition system may apply an overlap detection model to a top image and a pixel-wise mask for the top image to detect whether an item is overlapping with another in the top image. The item recognition system notifies a user of the overlap if detected.

Abstract: An online concierge system receives a delivery order containing a list of items, generates a suggested picking sequence for picking the delivery order in a warehouse, and transmits the suggested picking sequence to a mobile device of the shopper. Generating the suggested sequence includes applying a trained item sequence model to the delivery order. Training the item sequence model includes accessing data about a set of historical orders, determining a pairwise distance between each pair of aisles in the warehouse based on the data about the set of historical orders, and generating a distance graph based on the pairwise distance between each pair of aisles in the warehouse. The plurality of nodes represent a plurality of aisles in the warehouse, and the plurality of edges represent pairwise distances between pairs of aisles.

Abstract: An online concierge system receives unstructured data describing items offered for purchase by various warehouses. To generate attributes for products from the unstructured data, the online concierge system extracts candidate values for attributes from the unstructured data through natural language processing. One or more users associate a subset candidate values with corresponding attributes, and the online concierge system clusters the remaining candidate values with the candidate values of the subset associated with attributes. One or more users provide input on the accuracy of the generated clusters. The candidate values are applied as labels to items by the online concierge system, which uses the labeled items as training data for an attribute extraction model to predict values for one or more attributes from unstructured data about an item.

Abstract: A smart shopping cart includes internally facing cameras and an integrated scale to identify objects that are placed in the cart. To avoid unnecessary processing of images that are irrelevant, and thereby save battery life, the cart uses the scale to detect when an object is placed in the cart. The cart obtains images from a cache and sends those to an object detection machine learning model. The cart captures and sends a load curve as input to the trained model for object detection. Labeled load data and labeled image data are used by a model training system to train the machine learning model to identify an item when it is added to the shopping cart. The shopping cart also uses weight data and the image data from a timeframe associated with the addition of the item to the cart as inputs.

Abstract: An online concierge system trains a classification model as a domain adversarial neural network from training data labeled with source classes from a source domain that do not overlap with target classes from a target domain output by the classification model. The online concierge system maps one or more source classes to a target class. The classification model extracts features from an image, classifies whether an image is from the source domain or the target domain, and predicts a target class for an image from the extracted features. The classification model includes a gradient reversal layer between feature extraction layers and the domain classifier that is used during training, so the feature extraction layers extract domain invariant features from an image.

Abstract: A barcode decoding system decodes item identifiers from images of barcodes. The barcode decoding system receives an image of a barcode and rotates the image to a pre-determined orientation. The barcode decoding system also may segment the barcode image to emphasize the portions of the image that correspond to the barcode. The barcode decoding system generates a binary sequence representation of the item identifier encoded in the barcode by applying a barcode classifier model to the barcode image, and decodes the item identifier from the barcode based on the binary sequence representation.

Abstract: An online concierge system receives two types of orders, one of which requires fulfillment in a specific time interval, while the other can be fulfilled anytime up to a specific time interval. A machine learning model, trained on historical data about available shoppers in discrete time intervals, is used to predict how many shoppers will be available to fulfill orders in each time interval. For each time interval, the system retrieves the relevant orders of both types and creates candidate groups including orders of both types. For each group, the system determines a fulfillment cost based on items in the orders. The candidate group with the lowest cost is selected, and the orders in the selected group are sent to devices of available shoppers in that interval, prompting the shoppers to view and fulfill the orders.

Abstract: A user treatment engine uses user data describing characteristics of a user to evaluate a set of treatments that the user treatment engine may apply to the user. The user treatment engine generates treatment cost predictions for the treatments and generates treatment scores for the set of treatments based on the treatment cost predictions for the treatments and the user data for the user. The user treatment engine selects and applies a treatment from the set of treatments based on the generated treatment scores. The user treatment engine determines a reward to the online concierge system for the application of the treatment to the user and updates treatment selection parameters for the applied treatment based on the determined reward.

Abstract: An online concierge system trains a user interaction model to predict a probability of a user performing an interaction after one or more content items are displayed to the user. This provides a measure of an effect of displaying content items to the user on the user performing one or more interactions. The user interaction model is trained from displaying content items to certain users of the online concierge system and withholding display of the content items to other users of the online concierge system. To train the user interaction model, the user interaction model is applied to labeled examples identifying a user and value based on interactions the user performed after one or more content items were displayed to the user and interactions the user performed when one or more content items were not used.

Abstract: An online shopping concierge system sorts a list of items to be picked in a warehouse by receiving data identifying a warehouse and items to be picked by a picker in the warehouse. The system retrieves a machine-learned model that predicts a next item of a picking sequence of items. The model was trained, using machine-learning, based on sets of data that each include a list of picked items, an identification of a warehouse from which the items were picked, and a sequence in which the items were picked. The system identifies an item to pick first and a plurality of remaining items. The system predicts, using the model, a next item to be picked based on the remaining items, the first item, and the warehouse. The system transmits data identifying the first item and the predicted next item to be picked to the picker in the warehouse.

Abstract: An online concierge system includes sponsored content items in an interface including different slots for displaying content items. A sponsored content item may be displayed in a single slot or in multiple adjacent slots. The online concierge system determines a content score for various sponsored content items indicating a likelihood of a user interacting with a sponsored content item and a position bias for slots in the interface indicating a likelihood of the user interacting with a slot independent of content in the slot. Position biases are different dependent on a number of slots in which a content item is displayed. The online concierge system generates a graph identifying potential placements of sponsored content items in slots by selecting content items in an order according to their content scores. Sponsored content items are positioned in slots according to a path through the graph that has the highest overall expected value.

Abstract: An online concierge system suggests subsequent search queries based on previous search queries and whether the previous search queries resulted in conversions. The online concierge system trains a machine learning model using previous delivery orders and whether initial and subsequent search queries in the previous delivery orders resulted in conversions. When the online concierge system receives a search query to identify one or more items from a customer, the online concierge system parses the search query into combinations of terms and identifies items related to the search query. In response to the search query resulting in a conversion, the online concierge system retrieves a conversion graph and presents a suggested subsequent search query based on the conversion graph. In response to the search query not resulting in a conversion, the online concierge system retrieves a non-conversion graph and presents a suggested subsequent search query based on the non-conversion graph.

Abstract: An online concierge system allows users to order items within discrete time intervals later than a time when an order was received or for short-term fulfillment when the order was received. To account for a number of shoppers available to fulfill orders during different discrete time intervals and numbers of orders for fulfillment during different discrete time intervals, the online concierge system specifies a target rate for orders fulfilled later than a specified discrete time interval and a threshold from the target rate. A trained machine learning model periodically predicts a percentage of orders being fulfilled late, with an order associated with a predicted percentage when the order was received. The online concierge system increases a price of orders associated with predicted percentages greater than the threshold from the target rate. The increased price of an order is determined from a price elasticity curve and the predicted percentage.

Abstract: An online system may receive, from a user device, a request to view, at a graphical user interface, available entries of a third-party system. The third-party system may operate multiple physical locations. The operation of each physical location is documented by a time-sensitive dataset which includes multiple dynamic item entries. The online system may retrieve a geographical location associated with the user device and determine a subset of physical locations operated by the third-party system that are eligible for further selection based on distances of the physical locations from the geographical location associated with the user device. The online system may determine a metric measuring a size of the dynamic item entries available in the time-sensitive dataset and select one of the physical locations based on the metric. The online system may cause for display the dynamic item entries in the time-sensitive dataset associated with the selected physical location.

Abstract: An online concierge system may determine recommended search terms for a user. The online concierge system may receive a request from a user to view a user interface configured to receive a search query. The online concierge system retrieves long-term activity data including previous search terms entered by the user while searching for items to add to an online shopping cart. For each previous search term, the online concierge system retrieves categorical search terms corresponding to one or more categories to which the previous search term was mapped. The online concierge system determines a set of nearby categorical search terms and sends, for display via a client device, the set of nearby categorical search terms as recommended search terms.

Abstract: An online shopping concierge system identifies a set of delivery orders and a set of delivery agents associated with a location. The system allocates the orders among the agents, each agent being allocated at least one order. The system obtains agent progress data describing travel progress of the agents to the location, and order preparation progress data describing progress of preparing the orders for delivery. The system periodically updates the allocation of the orders among the agents based on the agent progress data and the order preparation progress data. This involves re-allocating at least one order to a different delivery agent. When a first agent arrives at the location, the system assigns to the first agent the orders allocated to the first agent. The system then removes the first agent from the set of available delivery agents, and removes the assigned delivery orders from the set of delivery orders.