Abstract: A system may generate a prompt based in part on a search query from a customer client device. The prompt instructs a machine learned model to provide item predictions. And the model was trained by: converting structured data describing items of an online catalog to annotated text data (unstructured data), generating training examples based in part on the annotated text data, and training the model using the training examples. The system may receive item predictions generated by the prompt being applied to the machine learned model, the item predictions may have corresponding item identifiers. The item predictions are processed to identify a recommended item from the item predictions. The processing includes determining item information for the recommended item using an item identifier associated with the recommended item. The item information is provided to the customer client device.

Abstract: An online concierge shopping system fulfills orders using workers who pick items at a warehouse to complete an order and workers to deliver the orders to a customer's location. To optimize the staffing of workers for each task, the system uses a trained model to predict the number of workers needed to achieve an optimal outcome based on an input set of contextual information. The system also schedules specific workers to various shifts using the predicted number of workers needed and then searching a feasibility space for an optimal solution. The trained model may be updated based on performance observations.

Abstract: An online system receives information describing a physical retail store, in which the information includes attributes of physical elements within the store and their arrangement. A request is received from a user to generate a rendering of the store in a virtual reality environment. A profile of the user describing the user's geographic location and a set of historical actions performed by the user are accessed, in which the set of historical actions is associated with one or more of the physical elements. Based on the information describing the store and the profile, the rendering is generated to include virtual reality elements representing a set of the physical elements arranged based on the arrangement of the physical elements, and the rendering is sent for display to the user. When an update to the information describing the store is received, the rendering is updated and sent for display to the user.

Abstract: Disclosed are visual recognition and sensor fusion weight detection system and method. An example method includes: tracking, by a sensor system, objects and motions within a selected area of a store; activating, by the sensor system, a first computing device positioned in the selected area in response to detecting a presence of a customer within the selected area: identifying, by the sensor system, the customer and at least one item carried by the customer; transmitting, by the sensor system, identifying information of the customer and the at least one item to a computing server system via a communication network; measuring, by the first computing device, a weight of the at least one item; transmitting, by the first computing device, the weight to the computing server system via the communication network; and generating, by the computing server system, via the communication network, transaction information of the at least one item.

Abstract: An online concierge system receives orders from users identifying items and a warehouses from which the items are obtained. The online concierge system displays groups of one or more orders to shoppers, allowing a shopper to select a group of orders for fulfillment. When selecting groups of orders to display to shoppers, the online concierge system accounts for costs for fulfilling different groups and displays groups having costs satisfying one or more criteria, while maintaining one or more restrictions on times to fulfill orders. The online concierge system trains a selection prediction model to predict an amount of time for a shopper to select a group of orders and determines an estimated fulfillment time for the group from the predicted amount of time. Accounting for the predicted selection time allows the online concierge system to identify a larger number of groups for which costs of fulfillment are determined.

Abstract: An online concierge system includes a marketplace automation engine for setting various control parameters affecting marketplace operation. The marketplace automation engine applies a hyperparameter learning model to the marketplace state data to predict a set of hyperparameters affecting a set of respective parameterized control decision models. The hyperparameter learning model is trained on historical marketplace state data and a configured outcome objective for the online concierge system. The marketplace automation engine independently applies the set of parameterized control decision models to the marketplace state data using the hyperparameters to generate a respective set of control parameters affecting marketplace operation of the online concierge system. The marketplace automation engine applies the respective set of control parameters to operation of the online concierge system.

Abstract: An online system receives information describing an order placed by a user of the online system and a set of contextual features associated with servicing the order. The online system also retrieves a set of user features associated with the user. The online system accesses a machine learning model trained to predict a tip amount the user is likely to provide for servicing the order and applies the machine learning model to a set of inputs, in which the set of inputs includes the information describing the order, the set of user features, and the set of contextual features. The online system then determines a suggested tip amount for servicing the order based on the predicted tip amount.

Abstract: An online concierge system suggests replacement items when an ordered item may be unavailable. To promote similarity of sources between the replacement item with the ordered item, candidate replacement items are scored, in part, based on a source similarity score based on a source of the candidate replacement item and a source of the ordered item. The source similarity score may be determined by a computer model based on user interactions with item sources. The similarity score may be based on source embeddings that may be determined based on respective item embeddings or may be determined by training source embeddings directly from user-source interactions. The similarity score for a candidate replacement item may be combined with a replacement score indicating the user's likelihood of selecting the candidate replacement item as a replacement to yield a total score for selection as suggestion as a replacement for the ordered item.

Abstract: An online system displays search results in response to a query by receiving a query from a customer. An online system accesses a set of candidate items and computes a relevance score and personalization score for each item. The online system computes the relevance score based on query data and item data and may normalize the relevance score. The online system computes the personalization score based on item data, such as an item embedding, and user data, such as a user embedding. The online system computes a query specificity score and adjusts the personalization score with the query specificity score such that generic queries have high personalization scores and specific queries have low personalization scores. The online system combines the relevance and personalization scores for each candidate item into a ranking score and displays the candidate items to the customer based on their ranking scores.

Abstract: An online concierge system assigns shoppers to fulfill orders from users. To allocate shoppers, the online concierge system predicts future supply and demand for the shoppers' services for different time windows. To forecast a supply of shoppers, the online concierge system trains a machine learning model that estimates future supply based on access to a shopper mobile application through which the shoppers obtain new assignments by shoppers. The online concierge system also forecasts future orders. The online concierge system estimates a supply gap in a future time period by selecting a target time to accept for shoppers to accept orders and determining a corresponding ratio of number of shoppers and number of orders. The online concierge system may adjust a number of shoppers allocated to the future time period to achieve the determined ratio number of shoppers and number of orders.

Abstract: A receipt capture device can collect transaction information from transactions conducted at a point of sale system by capturing receipt data transmitted from the point of sale system for the purpose of printing receipts at an external receipt printer. The receipt capture device can then send the collected receipt data to an online system for analysis. At the online system, received receipt data can be decoded from the printer-readable format it is transmitted in and used to enhance the online system's understanding of transactions occurring at a retailer associated with the point of sale system. For example, the online system can determine an approximate inventory of items available at purchase at the retailer by aggregating items recently purchased in transactions at the point of sale system.

Abstract: An online system maintains various items and maintains values for different attributes of the items, as well as an item embedding for each item. When the online system receives a query for retrieving one or more items, the online system generates an embedding for the query. Based on measures of similarity between the embedding for the query and item embeddings, the online system selects a set of items. The online system identifies a specific attribute of items and generates a whitelist of values for the specific attribute based on measures of similarity between item embeddings for items in the selected set and the embedding for the query. The online system removes items having values for the selected attribute outside of the whitelist of values from the selected set of items to identify items more likely to be relevant to the query.

Abstract: An online concierge system requests an image of a receipt of an order from a picker after the picker fulfills the order at a store. The online concierge system performs image processing on the image of the receipt and uses machine learning and optical character recognition to determine a tax amount paid for the order and a confidence score associated with the tax amount. The online concierge system may use the machine learning model for segmenting extracted text in the image of the receipt into tokens. The online concierge system may then determine at least one token associated with a tax item and the tax amount associated with the tax item. The online concierge system communicates the tax amount to the store for reimbursement based on the tax amount and the confidence score.

Abstract: Systems and methods for a contract-based offer generator is provided. A contract for a promotional offer on a product is received. Data is extracted from the contract. An offer band is accessed, and a plurality of test offers are selected from the offer bank by scoring each offer in the offer bank against the extracted data. The promotional offer and the selected plurality of test offers are deployed in a plurality of retail locations. This is done by maximizing orthogonality between the following variables: store sales, store out of stock rates, number of relevant SKUs carried in each store, temporal effects, discount depth, buy quantity and offer structure.

Abstract: An online concierge system receives multiple images of an item from a first client device associated with a shopper associated with the online concierge system, in which each of the images of the item is captured from a different angle and/or position and the item is included among an inventory of a warehouse associated with a retailer associated with the online concierge system. Based in part on the images of the item, the online concierge system generates a three-dimensional image of the item, in which the three-dimensional image of the item includes a dimension of the item and/or a color of the item. The online concierge system then sends the three-dimensional image of the item to a second client device associated with a customer of the online concierge system, in which a perspective of the three-dimensional image is modifiable within a display area of the second client device.

Abstract: An online concierge system detects acquired items included among an inventory of a customer and identifies one or more candidate available items from the acquired items based on a predicted perishability of each item and a predicted amount of each item that was used. The system retrieves recipes, matches the item(s) likely to be available to a set of recipes based on their ingredients, and identifies any remaining items for each matched recipe not likely to be available. The system retrieves a set of attributes associated with the customer and the set of recipes and computes a suggestion score for each recipe based on the attributes. The system ranks the recipes based on their scores, identifies one or more recipes for suggesting to the customer based on the ranking, and sends the recipe(s) and any remaining items for each recipe to a client device associated with the customer.

Abstract: A shopping cart's tracking system determines a first baseline location of the shopping cart at a first timestamp with a wireless device located on the shopping cart detecting one or more external wireless devices (e.g., RFID tags) in the indoor environment. The shopping cart's tracking system receives wheel motion data from one or more wheel sensors coupled to one or more wheels of the shopping cart, wherein the wheel motion data describes rotation of the one or more wheels. The shopping cart's tracking system calculates a translation traveled by the shopping cart from the first baseline location based on the wheel motion data. The shopping cart's tracking system determines an estimated location of the shopping cart at a second timestamp based on the first baseline location and the translation. With the estimated location, the shopping cart can update a map with the estimated location of the shopping cart.

Abstract: A warehouse from which shoppers fulfill orders for an online concierge system maintains an online concierge system-specific portion for which the online concierge system specifies placement of items in regions. To place items in the online concierge system-specific portion, the online concierge system accounts for co-occurrences of different items in orders and measures of similarity between different items. From the co-occurrences of items, the online concierge system generates an affinity graph. The online concierge system also generates a colocation graph based on distances between different regions in the online concierge system-specific portion. Using an optimization function with the affinity graph and the colocation graph, the online concierge system selects regions within the online concierge system-specific portion for different items to minimize an amount of time for shoppers to obtain items in the online concierge-system specific portion.