Abstract: In some implementations, a system for verifying items in a retail environment includes a physical shopping cart including a first set of sensors, and an automated checkout station including a second, different set of sensors. The physical shopping cart receives item verification data for verifying an item, detects the item as it enters the physical shopping cart, and performs a primary verification of the item. The automated checkout station obtains a virtual shopping cart that corresponds to the physical shopping cart. The virtual shopping cart includes a list of items that have been placed in the physical shopping cart, and a verification status of each item. The second, different set of sensors generate station sensor data that represents the physical shopping cart and the items in the physical shopping cart. A secondary verification of the physical shopping cart and its contents is performed by the automated checkout station.

Abstract: Physical shopping carts can have product detection systems and associate physical shopping carts with mobile computing devices (e.g., smartphones, tablet computing devices, smart watches, wearable computing devices). For example, physical shopping carts can be equipped with one or more product detection systems (e.g., scanners, sensors, cameras) that can electronically tally products that are placed in physical shopping carts. Mobile computing devices can be associated with and mounted on the physical shopping carts to provide a variety of enhanced shopping cart features not possible with conventional physical shopping carts, such as electronically tracking the contents of a shopping cart, checking-out from the mobile computing device (instead of at conventional check-out areas, such as point of sale terminals), and others.

Abstract: In some implementations, a computer-implemented method verifies item placed in physical shopping carts, using location data. An item verification engine receives an item scanned notification indicating that a mobile computing device has scanned an item, an item sensed notification indicating that a cart computing system has sensed the item being added to a shopping cart, item verification data that specifies an expected location of the scanned item, and actual location data that represents an actual location of the item in the retail environment. After receiving the item scanned notification, the item sensed notification, the item verification data, and the actual location data, a verification of the item is performed based at least in part on the item verification data and the actual location data. The item verification engine provides verification results based on performing the verification of the item, for receipt by the mobile computing device.

Abstract: Physical shopping carts can have product detection systems and associate physical shopping carts with mobile computing devices (e.g., smartphones, tablet computing devices, smart watches, wearable computing devices). For example, physical shopping carts can be equipped with one or more product detection systems (e.g., scanners, sensors, cameras) that can electronically tally products that are placed in physical shopping carts. Mobile computing devices can be associated with and mounted on the physical shopping carts to provide a variety of enhanced shopping cart features not possible with conventional physical shopping carts, such as electronically tracking the contents of a shopping cart, checking-out from the mobile computing device (instead of at conventional check-out areas, such as point of sale terminals), and others.

Abstract: Disclosed are systems and techniques for determining out of stock conditions on shelves. The techniques can include receiving, by a computing system, image data from a camera having pixel locations that each uniquely address and store a pixel value, generating a backing map having cell locations that each uniquely address and share a unique address with a corresponding pixel location in the image data, each cell location storing a backing value being an empty value if the pixel value is classified as showing the backing of a shelf and the backing value being a nonempty value if the pixel value is classified as not showing the backing of the shelf, determining, in the backing map, a shelf area representing a location of the captured shelf, and identifying an empty area by finding an area above the shelf area containing a threshold number of cell locations storing the empty value.

Abstract: Described herein are systems and techniques for imaging inventory objects in an environment. A system can include a cart, a first fixed camera fixedly mounted on the cart at a first angle, a pan-tilt-zoom (PTZ) camera controllably mounted on the cart, a PTZ controller, and a cart controller. The PTZ controller can receive PTZ instructions from the cart controller and send engagement instructions to the PTZ camera. The cart controller can receive, from the first fixed camera, first image data that captures a first inventory object, determine, from the first image data, a spatial location of a first inventory object, generate PTZ instructions to cause the PTZ camera to capture the first inventory object, transmit the PTZ instructions to the PTZ controller, and receive PTZ image data that captures the first inventory object.

Abstract: Disclosed are systems and techniques for identifying product identifiers in images. A technique can include receiving, by an edge computing device, x and y coordinates for a location of an out of stock shelf section, determining a frame of reference (FOR) based on the coordinates, the FOR including a location of a product identifier for the out of stock shelf section, determining incremental adjustments to a camera based on the FOR, instructing the camera to adjust by the incremental adjustments and capture an image of the location of the product identifier, performing image analysis on the image to identify the product identifier, determining second incremental adjustments to the camera to focus the camera on the location of the product identifier if the product identifier is not identifiable in the image, and instructing the camera to reset to original camera settings if the product identifier is identifiable in the image.

Abstract: In some implementations, a method performed by data processing apparatuses includes receiving, by an item verification engine, a scan notification indicating that a mobile computing device has scanned an item. In response to receiving the scan notification, the item verification engine provides an instruction for a cart computing system to activate item sensors on a shopping cart. The item verification engine receives sensor data collected by the item sensors as a result of the item having been placed in the shopping cart. The item verification engine receives item verification data for verifying the scanned item from the mobile computing device, performs a verification of the item based on the sensor data and the item verification data, and provides verification results for presentation by the mobile computing device.

Abstract: The disclosed technology provides for reconstructing products that are added to a shopping cart in three-dimensional space. The recontructed products can be used for product validation. A shopping cart can include product validation hardware including light emitters and light sensors along a top perimeter of the shopping cart. The shopping cart can activate the light emitters to emit light across a top horizontal plane of the shopping cart, receive, from light sensors on a side of the shopping cart opposite a side where the activated light emitter is located, light intensity data as a product passes through the top horizontal plane and obstructs at least a portion of the emitted light from being detected by the light sensors, identify, from the light intensity data, slices of the product, and reconstruct the product in 3D space based on stitching together the slices.

Abstract: In some implementations, a system for verifying items in a retail environment includes a physical shopping cart including a first set of sensors, and an automated checkout station including a second, different set of sensors. The physical shopping cart receives item verification data for verifying an item, detects the item as it enters the physical shopping cart, and performs a primary verification of the item. The automated checkout station obtains a virtual shopping cart that corresponds to the physical shopping cart. The virtual shopping cart includes a list of items that have been placed in the physical shopping cart, and a verification status of each item. The second, different set of sensors generate station sensor data that represents the physical shopping cart and the items in the physical shopping cart. A secondary verification of the physical shopping cart and its contents is performed by the automated checkout station.

Abstract: Physical shopping carts can have product detection systems and associate physical shopping carts with mobile computing devices (e.g., smartphones, tablet computing devices, smart watches, wearable computing devices). For example, physical shopping carts can be equipped with one or more product detection systems (e.g., scanners, sensors, cameras) that can electronically tally products that are placed in physical shopping carts. Mobile computing devices can be associated with and mounted on the physical shopping carts to provide a variety of enhanced shopping cart features not possible with conventional physical shopping carts, such as electronically tracking the contents of a shopping cart, checking-out from the mobile computing device (instead of at conventional check-out areas, such as point of sale terminals), and others.

Abstract: Physical shopping carts can have product detection systems and associate physical shopping carts with mobile computing devices (e.g., smartphones, tablet computing devices, smart watches, wearable computing devices). For example, physical shopping carts can be equipped with one or more product detection systems (e.g., scanners, sensors, cameras) that can electronically tally products that are placed in physical shopping carts. Mobile computing devices can be associated with and mounted on the physical shopping carts to provide a variety of enhanced shopping cart features not possible with conventional physical shopping carts, such as electronically tracking the contents of a shopping cart, checking-out from the mobile computing device (instead of at conventional check-out areas, such as point of sale terminals), and others.

Abstract: A physical shopping cart comprises a controller and a sensor array comprising an initial sensor configured to detect possible-items placed into the physical shopping cart and a secondary sensor configured to determine if possible-items detected by the initial sensor is are items-of-interest. The physical shopping cart is configured to maintain the secondary sensor in a low-power state; sense a possible-item placed into the physical shopping cart; determine if the possible-item is an item-of-interest; and transmit a sense-message that identifies the item-of-interest.

Abstract: A physical shopping cart comprises a controller and a sensor array comprising an initial sensor configured to detect possible-items placed into the physical shopping cart and a secondary sensor configured to determine if possible-items detected by the initial sensor is are items-of-interest. The physical shopping cart is configured to maintain the secondary sensor in a low-power state; sense a possible-item placed into the physical shopping cart; determine if the possible-item is an item-of-interest; and transmit a sense-message that identifies the item-of-interest.

Abstract: Physical shopping carts can have product detection systems and associate physical shopping carts with mobile computing devices (e.g., smartphones, tablet computing devices, smart watches, wearable computing devices). For example, physical shopping carts can be equipped with one or more product detection systems (e.g., scanners, sensors, cameras) that can electronically tally products that are placed in physical shopping carts. Mobile computing devices can be associated with and mounted on the physical shopping carts to provide a variety of enhanced shopping cart features not possible with conventional physical shopping carts, such as electronically tracking the contents of a shopping cart, checking-out from the mobile computing device (instead of at conventional check-out areas, such as point of sale terminals), and others.