Abstract: A counting device bears a visually distinct pattern or marking, and is coupled to a pusher or other movable system of a shelf in a manner that causes the counting device to rotate in response to changes in position of the pusher. When items are placed on a shelf, or removed from the shelf, causing the pusher to move, the pattern on the counting device rotates at an angle defined by an extent of the movement of the pusher. Images captured of the counting device at various times may be processed to determine orientations of the pattern at such times. Where a change in an orientation of a pattern is detected, a number of items added to or removed from a shelf may be calculated based on the change in orientation and an event involving the number of items may be determined to have occurred.

Abstract: Actors may be detected and tracked within a scene using multiple imaging devices provided in a network that are aligned with fields of view that overlap at least in part. Processors operating on the imaging devices may evaluate the images using one or more classifiers to recognize body parts within the images, and to associate the body parts with a common actor within the scene. Each of the imaging devices may generate records of the positions of the body parts and provide such records to a central server, that may correlate body parts appearing within images captured by two or more of the imaging devices and generate a three-dimensional model of an actor based on positions of the body parts. Motion of the body parts may be tracked in subsequent images, and the model of the actor may be updated based on the motion.

Abstract: This disclosure describes techniques for determining product volumes associated with products in a facility. These product volumes may be used to update planogram data associated with a facility, with the planogram data indicating inventory locations within the facility for various types of items supported by product fixtures. The planogram data may be used, in some instances, to update virtual carts of users interacting with the items in the facility.

Abstract: This disclosure describes techniques for updating planogram data associated with a facility. The planogram may indicate inventory locations within the facility for various types of items supported by product fixtures. In particular an image of a product fixture is analyzed to identify image segments corresponding to product groups, where each product group consists of instances of the same product and each image segment corresponds to a group of image points. Image data is further analyzed to determine coordinates of the points of each image segment. A product space corresponding to the product group is then defined based on the coordinates of the points of the product group. In some cases, for example, a product space may be defined in terms of the coordinates of the corners of a rectangular bounding box or volume.

Abstract: Poses or gestures of actors within a scene may be detected and tracked using multiple imaging devices aligned with fields of view that overlap at least in part. Images captured by the imaging devices may be synchronized and provided to a classifier to recognize body parts within the images, and score maps indicative of locations of peak probabilities that the images include the respective body parts may be generated. Locations of peak values within the score maps may be correlated with one another to confirm that a given body part is depicted in two or more fields of view, and vectors indicative of distances to or ranges of motion of body parts, with respect to the given body part, may be generated. Motion of the body parts may be tracked in subsequent images, and a virtual model of the body parts may be generated and updated based on the motion.

Abstract: Devices and techniques are generally described for articulated three-dimensional pose tracking. In some examples, a plurality of frames of image data captured by one or more cameras may be received. First feature data representing the plurality of frames of image data may be determined using a backbone network. The first feature data may be projected into three-dimensional (3D) space. In some examples, 3D location data describing respective 3D locations of one or more persons represented by the first feature data projected in the 3D space may be determined. The first feature data and the 3D location data may be sent to a four-dimensional (4D) convolutional neural network (CNN). The 4D CNN may generate second feature data comprising respective 3D representations of the one or more persons. Three dimensional pose data representing articulated 3D pose information for the one or more persons may be generated.

Abstract: Described herein is a method for enabling human-to-computer three-dimensional hand gesture-based natural interactions from depth images provided by a range finding imaging system. The method enables recognition of simultaneous gestures from detection, tracking and analysis of singular points of interests on a single hand of a user and provides contextual feedback information to the user. The singular points of interest of the hand: include hand tip(s), fingertip(s), palm center and center of mass of the hand, and are used for defining at least one representation of a pointer. The point(s) of interest is/are tracked over time and are analyzed to enable the determination of sequential and/or simultaneous “pointing” and “activation” gestures performed by a single hand.

Abstract: In a materials handling facility, events may be associated with users based on imaging data captured from multiple fields of view. When an event is detected at a location within the fields of view of multiple cameras, two or more of the cameras may be identified as having captured images of the location at a time of the event. Users within the materials handling facility may be identified from images captured prior to, during or after the event, and visual representations of the respective actors may be generated from the images. The event may be associated with one of the users based on distances between the users' hands and the location of the event, as determined from the visual representations, or based on imaging data captured from the users' hands, which may be processed to determine which, if any, of such hands includes an item associated with the event.

Abstract: Actors may be detected and tracked within a scene using multiple imaging devices provided in a network that are aligned with fields of view that overlap at least in part. Processors operating on the imaging devices may evaluate the images using one or more classifiers to recognize body parts within the images, and to associate the body parts with a common actor within the scene. Each of the imaging devices may generate records of the positions of the body parts and provide such records to a central server, that may correlate body parts appearing within images captured by two or more of the imaging devices and generate a three-dimensional model of an actor based on positions of the body parts. Motion of the body parts may be tracked in subsequent images, and the model of the actor may be updated based on the motion.

Abstract: Described herein is a method for detecting keypoints in three-dimensional images in which a three-dimensional image of a scene captured by a depth sensing imaging system is processed using a distance-independent keypoint filter. Keypoints are derived from the three-dimensional image by determining a mean shift field and using x- and y-components of the mean shift field to derive intersections of 0-isolines thereof. Positive and negative keypoints or nodes are connected to one another, positive to positive and negative to negative, to form a keygraph structure.

Abstract: Described herein is a method for image processing in which a captured image is processed to provide a background image which is updated in accordance with relative movement between foreground and background in successive captured images.

Abstract: In a materials handling facility, events may be associated with users based on imaging data captured from multiple fields of view. When an event is detected at a location within the fields of view of multiple cameras, two or more of the cameras may be identified as having captured images of the location at a time of the event. Users within the materials handling facility may be identified from images captured prior to, during or after the event, and visual representations of the respective actors may be generated from the images. The event may be associated with one of the users based on distances between the users' hands and the location of the event, as determined from the visual representations, or based on imaging data captured from the users' hands, which may be processed to determine which, if any, of such hands includes an item associated with the event.

Abstract: Described herein is method for applying noise reduction to multi-dimensional input data by applying a multi-dimensional approximation function to each point in the input data whilst maintaining definition for each discontinuity (620) present in the input data. A control lattice including a plurality of control points (630) is superposed on the input data such that, for each data point, a number of control points surround the data point (640). For each control point (630), a value which is a function of neighbouring data points is computed, and from, at least values of the neighbouring control points, determining an approximation function, which when applied to the input data provides filtered output data. Weighting is applied to at least control point associated with a data point in the vicinity of a discontinuity to maintain the sharpness of the discontinuity during the noise reduction process.

Abstract: Poses or gestures of actors within a scene may be detected and tracked using multiple imaging devices aligned with fields of view that overlap at least in part. Images captured by the imaging devices may be synchronized and provided to a classifier to recognize body parts within the images, and score maps indicative of locations of peak probabilities that the images include the respective body parts may be generated. Locations of peak values within the score maps may be correlated with one another to confirm that a given body part is depicted in two or more fields of view, and vectors indicative of distances to or ranges of motion of body parts, with respect to the given body part, may be generated. Motion of the body parts may be tracked in subsequent images, and a virtual model of the body parts may be generated and updated based on the motion.

Abstract: Described herein is a method for detecting keypoints in three-dimensional images in which a three-dimensional image of a scene captured by a depth sensing imaging system is processed using a distance-independent keypoint filter. Keypoints are derived from the three-dimensional image by determining a mean shift field and using x- and y-components of the mean shift field to derive intersections of 0-isolines thereof. Positive and negative keypoints or nodes are connected to one another, positive to positive and negative to negative, to form a keygraph structure.

Abstract: Described herein is a method and a system for providing efficient and complementary natural multi-modal gesture based interaction with a computerized system which displays visual feedback information on a graphical user interface on an interaction surface. The interaction surface is within the frustum of an imaging device comprising a single sensing system. The system uses the single sensing system for detecting both touch gesture interactions with the interaction surface (120) and three-dimensional touch-less gesture interactions in areas or volumes above the interaction surface performed by hands of a user. Both types of interaction are associated contextually with an interaction command controlling the computerized system when the gesture has been detected.

Abstract: Described herein is a method for signal processing in which noise in an input signal comprising an intensity which is a function of at least a first coordinate and a second coordinate. A noise reduction or de-noising process is applied to the input signal in respect of the first coordinate to generate an intermediate de-noised signal. A second noise reduction or de-noising process is applied to the intermediate de-noised signal in respect of the second coordinate to generate an output de-noised signal. For each coordinate, noise reduction processes are applied in two directions, the results of these processes being averaged to provide the de-noised signals. Weighting is applied in accordance with the detection of an edge within the signal used as input in the de-noising process.

Abstract: Described herein is a method for image processing in which a captured image is processed to provide a background image which is updated in accordance with relative movement between foreground and background in successive captured images.

Abstract: Described herein is method for applying noise reduction to multi-dimensional input data by applying a multi-dimensional approximation function to each point in the input data whilst maintaining definition for each discontinuity (620) present in the input data. A control lattice including a plurality of control points (630) is superposed on the input data such that, for each data point, a number of control points surround the data point (640). For each control point (630), a value which is a function of neighbouring data points is computed, and from, at least values of the neighbouring control points, determining an approximation function, which when applied to the input data provides filtered output data. Weighting is applied to at least control point associated with a data point in the vicinity of a discontinuity to maintain the sharpness of the discontinuity during the noise reduction process.

Abstract: Described herein is a method for signal processing in which noise in an input signal comprising an intensity which is a function of at least a first coordinate and a second coordinate. A noise reduction or de-noising process is applied to the input signal in respect of the first coordinate to generate an intermediate de-noised signal. A second noise reduction or de-noising process is applied to the intermediate de-noised signal in respect of the second coordinate to generate an output de-noised signal. For each coordinate, noise reduction processes are applied in two directions, the results of these processes being averaged to provide the de-noised signals. Weighting is applied in accordance with the detection of an edge within the signal used as input in the de-noising process.