Abstract: In a method for training a federated learning model, a server obtains a target split mode corresponding to a training node in response to determining that the training node satisfies a preset splitting condition. The server notifies a client to perform, based on the target split mode, node splitting. The server performs a next round of training by taking a left subtree node generated by performing the node splitting as a new training node until an updated training node does not satisfy the preset splitting condition. The server performs a next round of training by taking another non-leaf node of the boosting tree as a new training node. The server stops training and generates a target federated learning model in response to determining that a node dataset of the plurality of boosting trees is empty.

Abstract: An event at a shelf, such as a user interacting with items on the shelf, may be detected using sensors associated with the shelf. Hypotheses to describe the event are generated based on the collected sensor data. However, hypotheses determined based on only one type of sensor data may not have a confidence value that is sufficiently reliable. Hypotheses derived from different sensor types may be combined to generate new hypotheses that have higher confidence values. For example, hypotheses based on image data may be combined with hypotheses based on weight data to produce hypotheses with higher confidence values that are sufficiently reliable. A hypothesis, from the set of combined hypotheses, having the highest confidence value may then be used to describe the interaction at the shelf, such as identifying the item and quantity of item the user interacted with during the event.

Abstract: A method and a system for detecting rough handling. The system has a camera and a computing device. The computing device is configured to: receive a video stream captured by the camera; detect persons and packages from video frames of the video stream; construct person and package trajectories based on the detected persons and packages; recognize an action between one person trajectory and one package trajectory, where the action includes the one person picks up, holds, and drops off the one package; and determine existence of a rough handling when, within a predetermined frames after drop-off of the one package, a motion distance of the one package is greater than a threshold distance, and a motion speed of the one package is greater than a threshold speed.

Abstract: System and method for training a federated learning model asynchronously. The system includes a master and K number of institutional clients. The master is configured to execute an aggregation thread and K number of client management threads. The executed k-th client management thread includes: during initialization, instructing a k-th client to initiate a first iteration of training of its local federated learning model to obtain a training event Ek; or upon receiving an aggregation event Eaggr from the aggregation thread, instructing the k-th client to perform a t-th iteration of training of the local federated learning model to obtain the training event Ek; and sending the Ek to the aggregation thread. The executed aggregation thread includes: upon receiving Ek, updating a global federated learning model in the master to obtain the aggregation event Eaggr; and sending the Eaggr to the k-th client management thread.

Abstract: Described is a system for counting stacked items using image analysis. In one implementation, an image of an inventory location with stacked items is obtained and processed to determine the number of items stacked at the inventory location. In some instances, the item closest to the camera that obtains the image may be the only item viewable in the image. Using image analysis, such as depth mapping or Histogram of Oriented Gradients (HOG) algorithms, the distance of the item from the camera and the shelf of the inventory location can be determined. Using this information, and known dimension information for the item, a count of the number of items stacked at an inventory location may be determined.

Abstract: A method and a system for detecting rough handling. The system has a camera and a computing device. The computing device is configured to: receive a video stream captured by the camera; detect persons and packages from video frames of the video stream; construct person and package trajectories based on the detected persons and packages; recognize an action between one person trajectory and one package trajectory, where the action includes the one person picks up, holds, and drops off the one package; and determine existence of a rough handling when, within a predetermined frames after drop-off of the one package, a motion distance of the one package is greater than a threshold distance, and a motion speed of the one package is greater than a threshold speed.

Abstract: A system and a method for object detection using augmented training dataset. The system includes a computing device, which is configured to: provide a two-dimensional (2D) image, extract feature vectors and estimate depths of 2D image pixels, generate a point cloud using the feature vectors and the depths, and project the point cloud using a new camera pose to obtain a projected image. The 2D image has a bounding box enclosing an object and labeled with the object. Each pixel within the bounding box is named bounding box pixel, each point in the point cloud corresponding to the bounding box pixel is named bounding box point, each image pixel in the projected image corresponding to the bounding box point is named projected bounding box pixel, and a projected bounding box is defined using the projected bounding box pixels and labeled with the object.

Abstract: One or more load cells measure the weight of items at a fixture. Weight changes occur as items are picked from or placed to the fixture and may be used to determine when the item was picked or placed, quantity and so forth. Individual weights for a type of item may vary. A set of data comprising weight changes associated with interactions involving a single one of a particular type of item is gathered. These may be weight changes due to picks, places, or both. A model, such as a probability distribution, may be created that relates a particular weight of that type of item to a probability. The model may then be used to process other weight changes and attempt to determine what type of item was involved in an interaction.

Abstract: One or more load cells measure the weight of items on a shelf or other fixture. Weight changes occur as items are picked from or placed to the fixture. Output from the load cells is processed to produce denoised data. The denoised data is processed to determine event data representative of a pick or a place of an item. Hypotheses are generated using information about where particular types of items are stowed, the weights of those particular types of items, and the event data. A high scoring hypothesis is used to determine interaction data indicative of the type and quantity of an item that was added to or removed from the fixture. If ambiguity exists between hypotheses, additional techniques such as data about locations of weight changes and fine grained analysis may be used to determine the interaction data.

Abstract: A user undertakes an event, such as adding, removing, or otherwise interacting with an item stowed at a fixture. Using successive samples of weight data that occur during an event, a plurality of vectors are generated that are indicative of a weight change and a location associated with the fixture. The vectors are processed to determine where within the fixture the event took place. Hypotheses are generated that describe predicted interactions involving predicted locations that correspond to those indicated by the vectors. The hypotheses are ranked and then one is selected as a solution. The predicted values associated with the selected hypothesis are then used to generate interaction data that indicates one or more types of item and quantities of the items that were added, removed, or otherwise handled by the user.

Abstract: Sensor data from load cells at a shelf is processed using a first time window to produce first event data describing coarse and sub-events. Location data is determined that indicates where on the shelf weight changes occurred at particular times. Hypotheses are generated using information about where items are stowed, weights of those of items, type of event, and the location data. If confidence values of these hypotheses are below a threshold value, second event data is determined by merging adjacent sub-events. This second event data is then used to determine second hypotheses which are then assessed. A hypothesis with a high confidence value is used to generate interaction data indicative of picks or places of particular quantities of particular types of items from the shelf.

Abstract: This disclosure describes systems and techniques for detecting certain activity in image data, such as frames of video data. For example, the systems and techniques may create and utilize an activity classifier for detecting and classifying certain human activity in video data of a facility. In some instances, the classifier may be trained to identify, from the video data, certain predefined activity such as a user picking an item from a shelf, a user returning an item to a shelf, a first user passing an item to a second user, or the like. In some instances, the techniques enable activity detection using only video data, rather than in addition to data acquired by other sensors.

Abstract: This disclosure describes a system for utilizing multiple image processing techniques to identify an item represented in an image. In some implementations, one or more image processing algorithms may be utilized to process a received image to generate item image information and compare the item image information with stored item image information to identify the item. When a similarity score identifying the similarity between the item image information and at least one of the stored item image information is returned, a determination may be made as to whether the similarity score is high enough to confidently identify the item. If it is determined that the similarity score is high enough to confidently identify the item, the other algorithms may be terminated and the determined identity of the item returned.

Abstract: Load cells measure the weight of items on a shelf. Weight changes occur as items are picked from or placed to the fixture. Information about these weight changes is used to determine an estimated location on the shelf of a weight change. Hypotheses are generated using information about where particular types of items are stowed, the weights of those particular types of items, information about the weight changes, and the estimated locations of the weight changes. A model is used to produce confidence values in the hypotheses based on a change in weight measured at a first side and a change in weight measured at a second side of the shelf. A hypothesis with a confidence value that exceeds the threshold may be selected and used to determine interaction data indicative of a quantity picked or placed, type of item, and location on the shelf.

Abstract: A method and a system for counting pigs in a pig house. The system includes a computing device and an imaging device. The computing device has a processor and a storage device storing computer executable code. The computer executable code is configured to: receive images captured from one end to the other end of the house; detect keypoints in the images using a neural network; identify pig skeletons by associating several related keypoints; track the skeletons in the images to obtain trajectories; divide each image into an activated zone and a deactivated zone; designate a spatial value of 0 for the skeletons in the activated zone and a spatial value of 1 for the skeletons in the deactivated zone; summating first order difference of the spatial values for each trajectory to obtain a trajectory count; and add the trajectory counts to obtain pig count.

Abstract: A method and a system for self-checkout. The system includes a scanner, an imaging device, and a computing device. The computing device is configured to: initiate a self-checkout event; instruct the imaging device to capture video frames of a region of interest (ROI); track the product; record scanning status and location status of the product; in response to receive a scanning signal from the scanner, record scanning status of the product as scanned; calculate a shoplifting risk score based on a number of the product having the scanning status of unscanned and disappears from the table region or ROI; and provide a shoplifting warning when the shoplifting score is large.

Abstract: This disclosure describes techniques for removing noise from a signal to generate a modified signal, with the modified signal preserving any transitions of interest (e.g., sharp-edge discontinuities) present within the initial signal. In one example, the signal comprises a time-series signal with the time series representing a sequence of weight measurements from a scale device. In some examples, the scale device includes a platform that supports one or more physical items that may be selectively removed or added to. Here, the signal may include a sequence of step functions corresponding to changes in weight on the scale device (based on the removal or addition of items on the platform), plus corrupting noise from vibration. The techniques described herein may remove the corrupting noise, while preserving the sharp edges representing sudden changes in weight.

Abstract: A user may pick an item from a first inventory location, such as in a lane on a shelf, and may return it another location that is assigned to another type of item. Described are techniques to generate tidiness data that is indicative of whether an item has been returned to an inventory location assigned to that type of item. As items are taken, information about the type of item taken and its weight are stored. When an increase in weight at a lane indicates a return of an item to the lane, the weight of the return is compared to the stored weight of the items previously taken by a user. If the weights correspond to within a threshold value, the type of item associated with the stored weight is deemed to be returned and tidiness data indicative of a tidy return of the item to its appointed lane may be generated.

Abstract: A method and a system for counting pigs in a pig house. The system includes a computing device and an imaging device. The computing device has a processor and a storage device storing computer executable code. The computer executable code is configured to: receive images captured from one end to the other end of the house; detect keypoints in the images using a neural network; identify pig skeletons by associating several related keypoints; track the skeletons in the images to obtain trajectories; divide each image into an activated zone and a deactivated zone; designate a spatial value of 0 for the skeletons in the activated zone and a spatial value of 1 for the skeletons in the deactivated zone; summating first order difference of the spatial values for each trajectory to obtain a trajectory count; and add the trajectory counts to obtain pig count.

Abstract: A user may place an item at an inventory location in an incorrect location, such as the wrong lane on the shelf. An untidy score is generated that is indicative of whether an item is where it is supposed to be. A correct item verification (CIV) score is determined that is indicative of how similar the returned item is to the type of item that is supposed to be stored at that location. An invalid item recognition (IIR) score is determined that is indicative of similarity of the returned item to a set of items associated with the user, that set excluding the item that is supposed to be stored at that location. The CIV and the IIR are combined to generate an untidy score. Based on the untidy score, someone may be dispatched to clean up the inventory location.