Abstract: A tracking system includes a camera subsystem that includes cameras that capture vide of a space. Each camera is coupled with a camera client that determines local coordinates of people in the captured video. The camera clients generate frames that include color frames and depth frames labeled with an identifier number of the camera and their corresponding timestamps. The camera clients generate tracks that include metadata describing historical people detections, tracking identifications, timestamps, and the identifier number of the camera. The camera clients send the frames and tracks to cluster servers that maintain the frames and tracks such that they are retrievable using their corresponding labels. A camera server queries the cluster servers to receive the frames and tracks using their corresponding labels. The camera server determines the physical positions of people in the space based on the determined local coordinates.

Abstract: A sensor calibration system configured to receive a first frame of one or more markers on a repositionable platform at a first location within a space from a sensor. The system is further configured to determine pixel locations in the first frame for a first marker and a second marker from among the one or more markers. The system is further configured to receive distance information that corresponds with a distance between the platform and distance measuring devices. The system is further configured to determine (x,y) coordinates for the first marker and the second marker based on the distance information. The system is further configured to generate a homography based on the (x,y) coordinates and pixel locations of the first marker and the second marker. The homography includes coefficients that translate between pixel locations in the first frame of the sensor and (x,y) coordinates in the global plane.

Abstract: A tracking system includes a set of cameras, a kiosk, and a tracking server. The kiosk receives a payment amount from a person. The tracking server extracts features of the person from an image feed received from the set of cameras. The tracking server generates a session identifier that is associated with the payment amount and a unique code. The unique code represents at least one of the payment amount and features of the person. The tracking server sends a message to the kiosk to provide a ticket corresponding to the payment amount and the unique code to the person. The tracking server receives a digital cart associated with the person comprising items and a total cash value of the items. The tracking server concludes a transaction by deducting the total cash value from the payment amount.

Abstract: An apparatus includes a memory and processor. The memory stores a machine learning algorithm configured to decide between using an algorithmic and a virtual cart to process a transaction. The processor receives feedback for a decision made by the algorithm, indicating whether the algorithmic and virtual carts match. The processor assigns a reward to the feedback. A first positive reward is assigned when the algorithmic cart is selected, and the feedback indicates that the carts match. A second positive reward is assigned when the virtual cart is selected, and the feedback indicates that the carts do not match. A first negative reward is assigned when the algorithmic cart is selected, and the feedback indicates that the carts do not match. A second negative reward is assigned when the virtual cart is selected, and the feedback indicates that the carts match. The processor uses the reward to update the algorithm.

Abstract: An apparatus includes a memory and a processor. The memory stores a set of inputs, an algorithmic shopping cart, and a machine learning algorithm. The set of inputs includes information collected from sensors located in a physical store during a shopping session of a person. The algorithmic shopping cart includes items determined by an algorithm, based on the set of inputs, to have been selected during the shopping session. The machine learning algorithm is configured to use the set of inputs to select between using the algorithmic shopping cart and using a virtual shopping cart to process a transaction associated with the shopping session. The processor uses the machine learning algorithm to determine, based on the set of inputs, to use the algorithmic shopping cart to process the transaction. In response, the processor generates a receipt based on the algorithmic shopping cart. The processor sends the receipt to the person.

Abstract: An image sensor is positioned such that a field-of-view of the image sensor encompasses at least a portion of a rack storing items. The image sensor generates angled-view images of the items stored on the rack. A tracking subsystem determines that a person is within a threshold distance of the rack and receives image frames of the angled-view images. A pixel position of a wrist of the person is determined in at least a subset of the received image frames, thereby determining a set of pixel positions of the wrist. An aggregated wrist position is determined based on the set of pixel positions. If the aggregated wrist position is determined to correspond to a position on a shelf of the rack, a trigger signal is provided indicating a shelf-interaction event has occurred.

Abstract: A system includes sensors and a tracking subsystem. The subsystem tracks a first object and one or more other objects in a space. After determining that re-identification of the first object is needed, candidate identifiers are determined for the first object The candidate identifiers include a subset of the identifiers of all tracked objects in the space. The subset includes possible identifiers of the first object based on a history of movements of the first object and interactions of the first object with the other objects in the space. Based on a top-view image, a first descriptor is determined for the first object. The first descriptor is associated with a characteristic of the first object. Based on results of comparing the first descriptor to a set of predetermined descriptors, an updated identifier is determined for the first object.

Abstract: An object tracking system includes a first sensor, a second sensor, and a tracking system. The tracking system is configured to determine that the first current pixel location for the shelf marker does not match a first expected pixel location for the shelf marker. The tracking system is further configured to determine a second current pixel location for the shelf marker within a second frame from the second sensor, to recalibrate the first sensor when the second current pixel location for the shelf marker matches the second pixel location for the shelf marker and to update the first pixel location with the first current pixel location and the second pixel location with the second current pixel location when the second current pixel location for the shelf marker does not match the second pixel location for the shelf marker.

Abstract: An object tracking system includes a sensor, a weight sensor, and a tracking system. The sensor is configured to capture a frame of at least a portion of a rack within a global plane for a space. The tracking system is configured to detect a weight decrease on the weight sensor. The tracking system is further configured to receive the frame of the rack, to determine a pixel location for a person, to determine the person is within a predefined zone associated with the rack. The tracking system is further configured to identify the item associated with the weight sensor and to add the identified item to a digital cart associated with the person.

Abstract: An apparatus includes an interface, display, memory, and processor. The interface receives a video feed including first and second camera feeds, each feed corresponding to a camera located in a store. The processor stores a video segment in memory, assigned to a person and capturing a portion of a shopping session. The video segment includes first and second camera feed segments, each segment corresponding to a recording of the corresponding camera feed from a starting to an ending timestamp. Playback of the first and second camera feed segments is synchronized, and a slider bar controls a playback progress of the camera feed segments. The processor displays the camera feed segments and copies of the slider bar on the display. The processor receives an instruction from at least one of the copies of the slider bar to adjust the playback progress of the camera feed segments and adjusts the playback progress.

Abstract: A scalable tracking system includes a camera subsystem, a weight subsystem, and a central server. The camera subsystem includes cameras that capture video of a space, camera clients that determine local coordinates of people in the captured videos, and a camera server that determines the physical positions of people in the space based on the determined local coordinates. The weight subsystem determines when items were removed from shelves. The central server determines which person in the space removed the items based on the physical positions of the people in the space and the determination of when items were removed.

Abstract: A camera array for a scalable tracking system includes cameras that are communicatively coupled to camera clients. The cameras are arranged in a grid such that no camera is directly adjacent in the same row or column of the grid to another camera that is communicatively coupled to the same camera client. Cameras that are arranged along a diagonal of the grid are communicatively coupled to the same camera client.

Abstract: An apparatus includes a display, interface, and processor. The interface receives video from a camera located in a physical store and directed at a first physical rack. The camera captures video of the rack during a shopping session. The processor displays a first virtual rack that emulates the first physical rack and includes first and second virtual shelves. The virtual shelves include virtual items, which include graphical representations of physical items located on the physical rack. The processor displays the rack video, which depicts an event including the person interacting with the first physical rack. The processor also displays a virtual shopping cart. The processor receives information associated with the event, identifying the first virtual item. The rack video depicts that the person selected the first physical item while interacting with the first physical rack. The processor then stores the first virtual item in the virtual shopping cart.

Abstract: A system includes a sensor, a weight sensor, and a tracking subsystem. The tracking subsystem receives an image feed of top-view images generated by the sensor and weight measurements from the weight sensor. The tracking subsystem detects an event associated with an item being removed from a rack in which the weight sensor is installed. The tracking subsystem determines that a first person or a second person may be associated with the event. In response to determining that the first or second person may be associated with the event, buffer frames are stored of top-view images generated by the sensor during a time period associated with the event. The tracking subsystem then determines, using at least one of the stored buffer frames and a first action-detection algorithm, whether an action associated with the event was performed by the first person or the second person.

Abstract: An apparatus includes a processor. The processor receives an algorithmic shopping cart that includes a first set of items determined by an algorithm to have been selected by a person during a shopping session in a physical store, based on a set of inputs received from sensors located within the physical store. The processor also receives a virtual shopping cart that includes a second set of items. Video of the shopping session was captured by a set of cameras located in the physical store and depicts the person selecting the second set of items. The processor compares the algorithmic cart to the virtual cart and determines that a discrepancy exists between the algorithmic cart and the virtual cart. The processor determines a subset of the set of inputs associated with the discrepancy and attaches metadata explaining the discrepancy to the subset. The processor uses the subset to train the algorithm.

Abstract: A system includes sensors and a tracking subsystem. The subsystem receives a first image feed from a first sensor and a second image feed from a second sensor. The field-of view of the second sensor at least partially overlaps with that of the first sensor. The subsystem detects, in a frame from the first feed, a first contour associated with an object. The subsystem determines, based on pixel coordinates of the first contour, a first pixel position of the object. The subsystem detects, in a frame from the second feed, a second contour associated with the same object. The subsystem determines, based on pixel coordinates of the second contour, a second pixel position of the object. Based on the first pixel position and the second pixel position, a global position for the object is determined in a space.

Abstract: A rack for a scalable tracking system includes weight sensors disposed on shelves that hold items. The weight sensors detect the weight of the items and communicates signals indicating that weight to a circuit board positioned in the rack. The circuit board communicates these detected weights to a weight server that determines, based on these weights, whether items were removed from the shelves.

Abstract: A scalable tracking system processes video of a space to track the positions of people within a space. The tracking system determines local coordinates for the people within frames of the video and then assigns these coordinates to time windows based on when the frames were received. The tracking system then combines or clusters certain local coordinates that have been assigned to the same time window to determine a combined coordinate for a person during that time window.

Abstract: An object tracking system includes a sensor and a tracking system. The sensor is configured to capture a first frame of a global plane for at least a portion of a marker grid in a space. The tracking system is configured to receive a first coordinate in the global plane for a first corner of a marker grid, to determine a second coordinate in the global plane for the first marker on the marker grid, and to determine a third coordinate in the global plane where the second marker on the marker grid. The tracking system is further configured to determine a first pixel location for the first marker, to determine a second pixel location for the second marker, and to generate a homography based on the second coordinate for the first marker, the third coordinate for the second marker, the first pixel location, and the second pixel location.