Abstract: Systems and methods for a contract-based offer generator is provided. A contract for a promotional offer on a product is received. Data is extracted from the contract. An offer band is accessed, and a plurality of test offers are selected from the offer bank by scoring each offer in the offer bank against the extracted data. The promotional offer and the selected plurality of test offers are deployed in a plurality of retail locations. This is done by maximizing orthogonality between the following variables: store sales, store out of stock rates, number of relevant SKUs carried in each store, temporal effects, discount depth, buy quantity and offer structure.

Abstract: An online concierge system receives multiple images of an item from a first client device associated with a shopper associated with the online concierge system, in which each of the images of the item is captured from a different angle and/or position and the item is included among an inventory of a warehouse associated with a retailer associated with the online concierge system. Based in part on the images of the item, the online concierge system generates a three-dimensional image of the item, in which the three-dimensional image of the item includes a dimension of the item and/or a color of the item. The online concierge system then sends the three-dimensional image of the item to a second client device associated with a customer of the online concierge system, in which a perspective of the three-dimensional image is modifiable within a display area of the second client device.

Abstract: An online concierge system detects acquired items included among an inventory of a customer and identifies one or more candidate available items from the acquired items based on a predicted perishability of each item and a predicted amount of each item that was used. The system retrieves recipes, matches the item(s) likely to be available to a set of recipes based on their ingredients, and identifies any remaining items for each matched recipe not likely to be available. The system retrieves a set of attributes associated with the customer and the set of recipes and computes a suggestion score for each recipe based on the attributes. The system ranks the recipes based on their scores, identifies one or more recipes for suggesting to the customer based on the ranking, and sends the recipe(s) and any remaining items for each recipe to a client device associated with the customer.

Abstract: A shopping cart's tracking system determines a first baseline location of the shopping cart at a first timestamp with a wireless device located on the shopping cart detecting one or more external wireless devices (e.g., RFID tags) in the indoor environment. The shopping cart's tracking system receives wheel motion data from one or more wheel sensors coupled to one or more wheels of the shopping cart, wherein the wheel motion data describes rotation of the one or more wheels. The shopping cart's tracking system calculates a translation traveled by the shopping cart from the first baseline location based on the wheel motion data. The shopping cart's tracking system determines an estimated location of the shopping cart at a second timestamp based on the first baseline location and the translation. With the estimated location, the shopping cart can update a map with the estimated location of the shopping cart.

Abstract: A warehouse from which shoppers fulfill orders for an online concierge system maintains an online concierge system-specific portion for which the online concierge system specifies placement of items in regions. To place items in the online concierge system-specific portion, the online concierge system accounts for co-occurrences of different items in orders and measures of similarity between different items. From the co-occurrences of items, the online concierge system generates an affinity graph. The online concierge system also generates a colocation graph based on distances between different regions in the online concierge system-specific portion. Using an optimization function with the affinity graph and the colocation graph, the online concierge system selects regions within the online concierge system-specific portion for different items to minimize an amount of time for shoppers to obtain items in the online concierge-system specific portion.

Abstract: For each retailer in the geographic region, an online system predicts a number of orders placed at the retailer and a capacity to fulfill orders during a forecast time period. The capacity of the retailer is predicted based on a number of pickers expected to be available to the retailer during the forecast time period. The online system determines demand for the services of a picker at the retailer based on a comparison of the predicted number of orders and the predicted capacity to fulfill those orders. The online system displays a user interactive map of the geographic region to the picker. The map displays a pin at the location of each retailer in the geographic region, which describes the categorization determined for the retailer. The picker selects a pin, which causes the user interactive map to display a notification characterizing the demand for services at the retailer.

Abstract: An online concierge system maintains various items and an item embedding for each item. When the online concierge system receives a query for retrieving one or more items, the online concierge system generates an embedding for the query. The online concierge system trains a machine-learned model to determine a measure of relevance of an embedding for a query to item embeddings by generating training data of examples including queries and items with which users performed a specific interaction. The online concierge system generates a subset of the training data including examples satisfying one or more criteria and further trains the machine-learned model by application to the examples of the subset of the training data and stores parameters resulting from the further training as parameters of the machine-learned model.

Abstract: An online concierge system receives a delivery order containing a list of items, generates a suggested picking sequence for picking the delivery order in a warehouse, and transmits the suggested picking sequence to a mobile device of the shopper. Generating the suggested sequence includes applying a trained item sequence model to the delivery order. Training the item sequence model includes accessing data about a set of historical orders, determining a pairwise distance between each pair of aisles in the warehouse based on the data about the set of historical orders, and generating a distance graph based on the pairwise distance between each pair of aisles in the warehouse. The plurality of nodes represent a plurality of aisles in the warehouse, and the plurality of edges represent pairwise distances between pairs of aisles.

Abstract: An item recognition system uses a top camera and one or more peripheral cameras to identify items. The item recognition system may use image embeddings generated based on images captured by the cameras to generate a concatenated embedding that describes an item depicted in the image. The item recognition system may compare the concatenated embedding to reference embeddings to identify the item. Furthermore, the item recognition system may detect when items are overlapping in an image. For example, the item recognition system may apply an overlap detection model to a top image and a pixel-wise mask for the top image to detect whether an item is overlapping with another in the top image. The item recognition system notifies a user of the overlap if detected.

Abstract: An online concierge system receives unstructured data describing items offered for purchase by various warehouses. To generate attributes for products from the unstructured data, the online concierge system extracts candidate values for attributes from the unstructured data through natural language processing. One or more users associate a subset candidate values with corresponding attributes, and the online concierge system clusters the remaining candidate values with the candidate values of the subset associated with attributes. One or more users provide input on the accuracy of the generated clusters. The candidate values are applied as labels to items by the online concierge system, which uses the labeled items as training data for an attribute extraction model to predict values for one or more attributes from unstructured data about an item.

Abstract: An online concierge system trains a classification model as a domain adversarial neural network from training data labeled with source classes from a source domain that do not overlap with target classes from a target domain output by the classification model. The online concierge system maps one or more source classes to a target class. The classification model extracts features from an image, classifies whether an image is from the source domain or the target domain, and predicts a target class for an image from the extracted features. The classification model includes a gradient reversal layer between feature extraction layers and the domain classifier that is used during training, so the feature extraction layers extract domain invariant features from an image.

Abstract: A smart shopping cart includes internally facing cameras and an integrated scale to identify objects that are placed in the cart. To avoid unnecessary processing of images that are irrelevant, and thereby save battery life, the cart uses the scale to detect when an object is placed in the cart. The cart obtains images from a cache and sends those to an object detection machine learning model. The cart captures and sends a load curve as input to the trained model for object detection. Labeled load data and labeled image data are used by a model training system to train the machine learning model to identify an item when it is added to the shopping cart. The shopping cart also uses weight data and the image data from a timeframe associated with the addition of the item to the cart as inputs.

Abstract: An online concierge system receives two types of orders, one of which requires fulfillment in a specific time interval, while the other can be fulfilled anytime up to a specific time interval. A machine learning model, trained on historical data about available shoppers in discrete time intervals, is used to predict how many shoppers will be available to fulfill orders in each time interval. For each time interval, the system retrieves the relevant orders of both types and creates candidate groups including orders of both types. For each group, the system determines a fulfillment cost based on items in the orders. The candidate group with the lowest cost is selected, and the orders in the selected group are sent to devices of available shoppers in that interval, prompting the shoppers to view and fulfill the orders.

Abstract: A barcode decoding system decodes item identifiers from images of barcodes. The barcode decoding system receives an image of a barcode and rotates the image to a pre-determined orientation. The barcode decoding system also may segment the barcode image to emphasize the portions of the image that correspond to the barcode. The barcode decoding system generates a binary sequence representation of the item identifier encoded in the barcode by applying a barcode classifier model to the barcode image, and decodes the item identifier from the barcode based on the binary sequence representation.

Abstract: A user treatment engine uses user data describing characteristics of a user to evaluate a set of treatments that the user treatment engine may apply to the user. The user treatment engine generates treatment cost predictions for the treatments and generates treatment scores for the set of treatments based on the treatment cost predictions for the treatments and the user data for the user. The user treatment engine selects and applies a treatment from the set of treatments based on the generated treatment scores. The user treatment engine determines a reward to the online concierge system for the application of the treatment to the user and updates treatment selection parameters for the applied treatment based on the determined reward.

Abstract: An online concierge system trains a user interaction model to predict a probability of a user performing an interaction after one or more content items are displayed to the user. This provides a measure of an effect of displaying content items to the user on the user performing one or more interactions. The user interaction model is trained from displaying content items to certain users of the online concierge system and withholding display of the content items to other users of the online concierge system. To train the user interaction model, the user interaction model is applied to labeled examples identifying a user and value based on interactions the user performed after one or more content items were displayed to the user and interactions the user performed when one or more content items were not used.

Abstract: An online shopping concierge system sorts a list of items to be picked in a warehouse by receiving data identifying a warehouse and items to be picked by a picker in the warehouse. The system retrieves a machine-learned model that predicts a next item of a picking sequence of items. The model was trained, using machine-learning, based on sets of data that each include a list of picked items, an identification of a warehouse from which the items were picked, and a sequence in which the items were picked. The system identifies an item to pick first and a plurality of remaining items. The system predicts, using the model, a next item to be picked based on the remaining items, the first item, and the warehouse. The system transmits data identifying the first item and the predicted next item to be picked to the picker in the warehouse.

Abstract: An online concierge system includes sponsored content items in an interface including different slots for displaying content items. A sponsored content item may be displayed in a single slot or in multiple adjacent slots. The online concierge system determines a content score for various sponsored content items indicating a likelihood of a user interacting with a sponsored content item and a position bias for slots in the interface indicating a likelihood of the user interacting with a slot independent of content in the slot. Position biases are different dependent on a number of slots in which a content item is displayed. The online concierge system generates a graph identifying potential placements of sponsored content items in slots by selecting content items in an order according to their content scores. Sponsored content items are positioned in slots according to a path through the graph that has the highest overall expected value.