Abstract: An apparatus includes a memory and a processor. The memory stores a set of inputs, an algorithmic shopping cart, and a machine learning algorithm. The set of inputs includes information collected from sensors located in a physical store during a shopping session of a person. The algorithmic shopping cart includes items determined by an algorithm, based on the set of inputs, to have been selected during the shopping session. The machine learning algorithm is configured to use the set of inputs to select between using the algorithmic shopping cart and using a virtual shopping cart to process a transaction associated with the shopping session. The processor uses the machine learning algorithm to determine, based on the set of inputs, to use the algorithmic shopping cart to process the transaction.

Abstract: An apparatus includes an interface, display, memory, and processor. The interface receives a video feed including first and second camera feeds, each feed corresponding to a camera located in a store. The processor stores a video segment in memory, assigned to a person and capturing a portion of a shopping session. The video segment includes first and second camera feed segments, each segment corresponding to a recording of the corresponding camera feed from a starting to an ending timestamp. Playback of the first and second camera feed segments is synchronized, and a slider bar controls a playback progress of the camera feed segments. The processor displays the camera feed segments and copies of the slider bar on the display. The processor receives an instruction from at least one of the copies of the slider bar to adjust the playback progress of the camera feed segments and adjusts the playback progress.

Abstract: An image sensor is positioned such that a field-of-view of the image sensor encompasses at least a portion of a rack storing items. The image sensor generates images of the items stored on the rack. Over a period of time, a tracking subsystem tracks a pixel position of the wrist of a person interacting with items stored on the rack. receives image frames of the angled-view images. The tracking subsystem determines whether an item was interacted with by a person and, if so, the identified item is assigned to the person.

Abstract: An image sensor is positioned such that a field-of-view of the image sensor encompasses at least a portion of a rack storing items. The image sensor generates angled-view images of the items stored on the rack. A tracking subsystem determines that a person is within a threshold distance of the rack and receives image frames of the angled-view images. A pixel position of a wrist of the person is determined in at least a subset of the received image frames, thereby determining a set of pixel positions of the wrist. An aggregated wrist position is determined based on the set of pixel positions. If the aggregated wrist position is determined to correspond to a position on a shelf of the rack, a trigger signal is provided indicating a shelf-interaction event has occurred.

Abstract: An apparatus includes a display, interface, and processor. The interface receives a live camera feed from a camera directed at a first physical structure located in a physical space. The processor receives an indication of an event associated with the first physical structure and accordingly displays a first virtual structure corresponding to the first physical structure. The first virtual structure includes virtual items that emulate physical items located on the first physical structure. The processor additionally displays a recording of the live camera feed, which depicts the event associated with the first physical structure.

Abstract: An apparatus includes a display and a processor. The processor displays a virtual shopping cart. The processor also receives information indicating that an algorithm determined that a physical item was selected by a person during a shopping session in a physical store, based on a set of inputs received from sensors located within the store. In response, the processor displays a virtual item, which includes a graphical representation of the physical item. The processor additionally displays a rack video captured during the shopping session by a rack camera located in the store. The rack camera is directed at a physical rack located in the store, which includes the physical item. In response to displaying the rack video, the processor receives information identifying the virtual item, where the rack video depicts that the person selected the physical item. The processor then stores the virtual item in the virtual shopping cart.

Abstract: A system includes a weight sensor and a weight server. The weight sensor generates a signal indicative of a weight of at least one of an item. The weight server detects an event corresponding to a weight change on the weight sensor when a quantity of the item is removed from the weight sensor. The weight server determines the item quantity by calculating a result of dividing the weight change over a unit weight of the item. If the result is within a threshold range from an integer that is closest to the result, the weight server determines that a quantity of the item with the amount of the integer is removed from the weight sensor. If the result is not within the threshold range from the integer, the weight server uses weight change patterns of historically observed signals to determine item quantity that was removed from the weight sensor.

Abstract: An object tracking system that includes a sensor and a tracking system. The sensor configured to capture a frame of at least a portion of a rack within a global plane for a space. The tracking system is configured to detect an item was removed from the rack. The tracking system is further configured to receive the frame of the rack, to identify a marker on an item within a predefined zone in the frame, and to identify the item associated with the identified marker. The tracking system is further configured to determine a pixel location for a person, to determine the person is within the predefined zone associated with the, and to add the identified item to a digital cart associated with the person.

Abstract: An object tracking system that includes a sensor, a weight sensor, and a tracking system. The sensor configured to capture a frame of at least a portion of a rack within a global plane for a space. The tracking system is configured to detect a weight decrease on the weight sensor. The tracking system is further configured to receive the frame of the rack, to identify a marker on an item within a predefined zone in the frame, and to identify the item associated with the identified marker. The tracking system is further configured to determine a pixel location for a person, to determine the person is within the predefined zone associated with the, and to add the identified item to a digital cart associated with the person.

Abstract: An object tracking system includes a sensor and a tracking system. The sensor is configured to capture a first frame of a global plane for at least a portion of a marker grid in a space. The tracking system is configured to receive a first coordinate in the global plane for a first corner of a marker grid, to determine a second coordinate in the global plane for the first marker on the marker grid, and to determine a third coordinate in the global plane where the second marker on the marker grid. The tracking system is further configured to determine a first pixel location for the first marker, to determine a second pixel location for the second marker, and to generate a homography based on the second coordinate for the first marker, the third coordinate for the second marker, the first pixel location, and the second pixel location.

Abstract: An object tracking system that includes a sensor and a tracking system. The sensor is configured to capture a frame of at least a portion of a rack within a global plane for a space. The tracking system is further configured to receive the frame, to determine a pixel location for the first person, and to determine a person is within the predefined zone associated with the rack. The tracking system is further configured to identify the plurality of items in a digital cart associated with the person, to identify an item from the digital cart associated with the first person, and to remove the identified item from the digital cart associated with the person.

Abstract: An image sensor is positioned such that a field-of-view of the image sensor encompasses at least a portion of a rack storing items. The image sensor generates angled-view images of the items stored on the rack. A tracking subsystem determines that a person is within a threshold distance of the rack and receives image frames of the angled-view images. A pixel position of a wrist of the person is determined in at least a subset of the received image frames, thereby determining a set of pixel positions of the wrist. An aggregated wrist position is determined based on the set of pixel positions. If the aggregated wrist position is determined to correspond to a position on a shelf of the rack, a trigger signal is provided indicating a shelf-interaction event has occurred.

Abstract: A system includes sensors and a tracking subsystem. The subsystem tracks a first object and one or more other objects in a space. After determining that re-identification of the first object is needed, candidate identifiers are determined for the first object. The candidate identifiers include a subset of the identifiers of all tracked objects in the space. The subset includes possible identifiers of the first object based on a history of movements of the first object and interactions of the first object with the other objects in the space. Based on a top-view image, a first descriptor is determined for the first object. The first descriptor is associated with a characteristic of the first object. Based on results of comparing the first descriptor to a set of predetermined descriptors, an updated identifier is determined for the first object.

Abstract: An image sensor is positioned such that a field-of-view of the image sensor encompasses at least a portion of a rack storing items. The image sensor generates angled-view images of the items stored on the rack. A tracking subsystem receives image frames of the angled-view images. The tracking subsystem detects that a trigger event has occurred. A set of one or more image frames from the image feed are determined that are associated with the detected trigger event. A region-of-interest of the image frame is determined based on the pixel position of the wrist of the person. The region-of-interest includes a subset of the pixels of the image frame. A first item in the determined region-of-interest using an object detection algorithm. The identified first item is assigned to the person.

Abstract: An apparatus includes a display, interface, and processor. The interface receives a camera feed from a camera directed at a first physical rack located in a physical store. The processor displays virtual racks assigned to physical racks. The processor receives an indication of an event associated with the first physical rack and accordingly displays a first virtual rack, which includes first and second virtual shelves. The first and second virtual shelves include virtual items that emulate physical items located on physical shelves of the first physical rack. The processor additionally displays a recording of the camera feed, which depicts the event associated with the first physical rack.

Abstract: An apparatus includes a display and a processor. The processor receives a refund request, which includes a request for a refund of a price of an item charged to an account belonging to a person, and information identifying a shopping session of the person in a physical store. In response to receiving the refund request, the processor locates a video segment captured during the shopping session and stored in a database. The processor displays the video segment, which depicts a scenario indicating that the person did not select the item for purchase during the shopping session. The processor further receives information indicating that the person did not select the item for purchase during the shopping session. The information is based at least in part on the video segment. In response to receiving the information, the processor credits the account belonging to the person with the price of the item.

Abstract: An apparatus includes a display and a processor. The processor displays a virtual shopping cart. The processor also receives information indicating that an algorithm determined that a physical item was selected by a person during a shopping session in a physical store, based on a set of inputs received from sensors located within the store. In response, the processor displays a virtual item, which includes a graphical representation of the physical item. The processor additionally displays a rack video captured during the shopping session by a rack camera located in the store. The rack camera is directed at a physical rack located in the store, which includes the physical item. In response to displaying the rack video, the processor receives information identifying the virtual item, where the rack video depicts that the person selected the physical item. The processor then stores the virtual item in the virtual shopping cart.

Abstract: An object tracking system that includes a sensor, a weight sensor, and a tracking system. The sensor is configured to capture a frame of at least a portion of a rack within a global plane for a space. The tracking system is configured to detect a weight increase on the weight sensor and to determine a weight increase amount on the weight sensor. The tracking system is further configured to receive the frame, to determine a pixel location for the first person, and to determine a person is within the predefined zone associated with the rack. The tracking system is further configured to identify the plurality of items in a digital cart associated with the person, to identify an item from the digital cart with an item weight that is closest to the weight increase amount, and to remove the identified item from the digital cart associated with the person.

Abstract: An apparatus includes a display, interface, and processor. The interface receives video from a camera located in a physical store and directed at a first physical rack. The camera captures video of the rack during a shopping session. The processor displays a first virtual rack that emulates the first physical rack and includes first and second virtual shelves. The virtual shelves include virtual items, which include graphical representations of physical items located on the physical rack. The processor displays the rack video, which depicts an event including the person interacting with the first physical rack. The processor also displays a virtual shopping cart. The processor receives information associated with the event, identifying the first virtual item. The rack video depicts that the person selected the first physical item while interacting with the first physical rack. The processor then stores the first virtual item in the virtual shopping cart.

Abstract: A scalable tracking system processes video of a space to track the positions of people within a space. The tracking system determines local coordinates for the people within frames of the video and then assigns these coordinates to time windows based on when the frames were received. The tracking system then combines or clusters certain local coordinates that have been assigned to the same time window to determine a combined coordinate for a person during that time window.