Abstract: A projected image item tracking system that analyzes projected camera images to determine items taken from, placed on, or moved on a shelf or other area in an autonomous store. The items and actions performed on them may then be attributed to a shopper near the area. Projected images may be combined to generate a 3D volume difference between the state of the area before and after shopper interaction. The volume difference may be calculated using plane-sweep stereo, or using convolutional neural networks. Because these methods may be computationally intensive, the system may first localize a change volume where items appear to have been displaced, and then generate a volume difference only within that change volume. This optimization results in significant savings in power consumption and in more rapid identification of items. The 3D volume difference may also indicate the quantity of items displaced, for example from a vertical stack.

Abstract: An autonomous store that tracks shopper movements and actions, and performs store cleaning actions based on analysis of shopper activity. Cleaning actions may include disinfecting the store or regions within the store using radiation, fogging or spraying, or ventilation. Cleaning actions may be targeted; for example, zones where shoppers linger or congregate may be cleaned more frequently or intensively, or shelves or items that shoppers touch may be cleaned after these interactions. Shopper activity information may be used to limit the number of shoppers in a store at once, for example by denying entry when the store is at capacity. The density of shoppers in regions of the store may be communicated to shoppers so that they can limit their interactions with other shoppers. Shopper activity history may be used for contact tracing by identifying other shoppers that may have been exposed to an individual who was in the store.

Abstract: A projected image item tracking system that analyzes projected camera images to determine items taken from, placed on, or moved on a shelf or other area in an autonomous store. The items and actions performed on them may then be attributed to a shopper near the area. Projected images may be combined to generate a 3D volume difference between the state of the area before and after shopper interaction. The volume difference may be calculated using plane-sweep stereo, or using convolutional neural networks. Because these methods may be computationally intensive, the system may first localize a change volume where items appear to have been displaced, and then generate a volume difference only within that change volume. This optimization results in significant savings in power consumption and in more rapid identification of items. The 3D volume difference may also indicate the quantity of items displaced, for example from a vertical stack.

Abstract: An automated store attached to or integrated into a site where vehicles park, such as a gas station, charging station, or parking lot. The store may obtain the identity of the vehicle automatically, for example in a message sent over a charging cable, or by scanning a license plate. An authorization linked to the vehicle identity may be extended to passengers who exit the vehicle, so that these passengers may take items from the store and have them automatically charged to the vehicle's account. Locked cases containing products may be unlocked automatically when a shopper who exited an authorized vehicle arrives at the case. As passengers move around the site and obtain items from the store, messages may be transmitted back to the vehicle, or to a mobile device of the vehicle owner or driver, showing the items that have been taken.

Abstract: A sensor bar shelf monitor, for example that may be added to an existing shelving system to convert a store to autonomous operation. The sensor bar may contain distance sensor to detect shoppers reaching towards items on a shelf, and cameras to determine which items shoppers have taken. It may be installed into shelf supports such as gondola shelving uprights. The sensor bar may be located at the front edge of a shelf, and may monitor the shelf below. Placing the sensor bar along the front edge prevents damage to electronics from spills or shelf cleaning, and prevents heat from the sensor bar electronics from damaging items on the shelf. The sensor bar may have a local sensor bar processor that collects sensor data; images may be analyzed locally or transferred to more powerful store processors. A sensor bar may also have controllable lights and controllable electronic labels.

Abstract: An autonomous store that tracks shopper movements and actions, and performs store cleaning actions based on analysis of shopper activity. Cleaning actions may include disinfecting the store or regions within the store using radiation, fogging or spraying, or ventilation. Cleaning actions may be targeted; for example, zones where shoppers linger or congregate may be cleaned more frequently or intensively, or shelves or items that shoppers touch may be cleaned after these interactions. Shopper activity information may be used to limit the number of shoppers in a store at once, for example by denying entry when the store is at capacity. The density of shoppers in regions of the store may be communicated to shoppers so that they can limit their interactions with other shoppers. Shopper activity history may be used for contact tracing by identifying other shoppers that may have been exposed to an individual who was in the store.

Abstract: A sensor bar shelf monitor, for example that may be added to an existing shelving system to convert a store to autonomous operation. The sensor bar may contain distance sensor to detect shoppers reaching towards items on a shelf, and cameras to determine which items shoppers have taken. It may be installed into shelf supports such as gondola shelving uprights. The sensor bar may be located at the front edge of a shelf, and may monitor the shelf below. Placing the sensor bar along the front edge prevents damage to electronics from spills or shelf cleaning, and prevents heat from the sensor bar electronics from damaging items on the shelf. The sensor bar may have a local sensor bar processor that collects sensor data; images may be analyzed locally or transferred to more powerful store processors. A sensor bar may also have controllable lights and controllable electronic labels.

Abstract: A system that integrates camera images and quantity sensors to determine items taken from, placed on, or moved on a shelf or other area in an autonomous store. The items and actions performed may then be attributed to a shopper near the area. Shelves may be divided into storage zones, such as bins or lanes, and a quantity sensor may measure the item quantity in each zone. Quantity changes indicate that a shopper has taken or placed items in the zone. Distance sensors, such as LIDAR, may be used for shelves that push items towards the front. Strain gauges may be used for bins or hanging rods. Quantity changes may trigger analysis of camera images of the shelf to identify the items taken or replaced. Images from multiple cameras that view a shelf may be projected to a vertical plane at the front of the shelf to simplify analysis.

Abstract: A system that integrates camera images and quantity sensors to determine items taken from, placed on, or moved on a shelf or other area in an autonomous store. The items and actions performed may then be attributed to a shopper near the area. Shelves may be divided into storage zones, such as bins or lanes, and a quantity sensor may measure the item quantity in each zone. Quantity changes indicate that a shopper has taken or placed items in the zone. Distance sensors, such as LIDAR, may be used for shelves that push items towards the front. Strain gauges may be used for bins or hanging rods. Quantity changes may trigger analysis of camera images of the shelf to identify the items taken or replaced. Images from multiple cameras that view a shelf may be projected to a vertical plane at the front of the shelf to simplify analysis.

Abstract: A projected image item tracking system that analyzes projected camera images to determine items taken from, placed on, or moved on a shelf or other area in an autonomous store. The items and actions performed on them may then be attributed to a shopper near the area. Projected images may be combined to generate a 3D volume difference between the state of the area before and after shopper interaction. The volume difference may be calculated using plane-sweep stereo, or using convolutional neural networks. Because these methods may be computationally intensive, the system may first localize a change volume where items appear to have been displaced, and then generate a volume difference only within that change volume. This optimization results in significant savings in power consumption and in more rapid identification of items. The 3D volume difference may also indicate the quantity of items displaced, for example from a vertical stack.

Abstract: A projected image item tracking system that analyzes projected camera images to determine items taken from, placed on, or moved on a shelf or other area in an autonomous store. The items and actions performed on them may then be attributed to a shopper near the area. Projected images may be combined to generate a 3D volume difference between the state of the area before and after shopper interaction. The volume difference may be calculated using plane-sweep stereo, or using convolutional neural networks. Because these methods may be computationally intensive, the system may first localize a change volume where items appear to have been displaced, and then generate a volume difference only within that change volume. This optimization results in significant savings in power consumption and in more rapid identification of items. The 3D volume difference may also indicate the quantity of items displaced, for example from a vertical stack.

Abstract: A projected image item tracking system that analyzes projected camera images to determine items taken from, placed on, or moved on a shelf or other area in an autonomous store. The items and actions performed on them may then be attributed to a shopper near the area. Projected images may be combined to generate a 3D volume difference between the state of the area before and after shopper interaction. The volume difference may be calculated using plane-sweep stereo, or using convolutional neural networks. Because these methods may be computationally intensive, the system may first localize a change volume where items appear to have been displaced, and then generate a volume difference only within that change volume. This optimization results in significant savings in power consumption and in more rapid identification of items. The 3D volume difference may also indicate the quantity of items displaced, for example from a vertical stack.

Abstract: A projected image item tracking system that analyzes projected camera images to determine items taken from, placed on, or moved on a shelf or other area in an autonomous store. The items and actions performed on them may then be attributed to a shopper near the area. Projected images may be combined to generate a 3D volume difference between the state of the area before and after shopper interaction. The volume difference may be calculated using plane-sweep stereo, or using convolutional neural networks. Because these methods may be computationally intensive, the system may first localize a change volume where items appear to have been displaced, and then generate a volume difference only within that change volume. This optimization results in significant savings in power consumption and in more rapid identification of items. The 3D volume difference may also indicate the quantity of items displaced, for example from a vertical stack.

Abstract: A system that analyzes camera images to track a person from a point where the person obtains an authorization to a different point where the authorization is used. The authorization may be extended in time and space from the point where it was initially obtained. Scenarios enabled by embodiments include automatically opening a locked door or gate for an authorized person and automatically charging items taken by a person to that person's account. Supports automated stores that allow users to enter, take products and exit without explicitly paying. An illustrative application is an automated, unmanned gas station that allows a user to pay at the pump and then enter a locked on-site convenience store or a locked case with products the user can take for automatic purchase. Embodiments may also extend authorization to other people, such as occupants of the same vehicle.

Abstract: A system that analyzes camera images to track a person in an autonomous store, and to determine when a tracked person takes or moves items in the store. The system may associate a field of influence volume around a person's location; intersection of this volume with an item storage area, such as a shelf, may trigger the system to look for changes in the items on the shelf. Items that are taken from, placed on, or moved on a shelf may be determined by a neural network that processes before and after images of the shelf. Person tracking may be performed by analyzing images from fisheye ceiling cameras projected onto a plane horizontal to the floor. Projected ceiling camera images may be analyzed using a neural network trained to recognize shopper locations. The autonomous store may include modular ceiling and shelving fixtures that contain cameras, lights, processors, and networking.