Abstract: As agents move about a materials handling facility, tracklets representative of the position of each agent are maintained along with a confidence score indicating a confidence that the position of the agent is known. If the confidence score falls below a threshold level, image data of the agent associated with the low confidence score is obtained and processed to generate one or more embedding vectors representative of the agent at a current position. Those embedding vectors are then compared with embedding vectors of other candidate agents to determine a set of embedding vectors having a highest similarity. The candidate agent represented by the set of embedding vectors having the highest similarity score is determined to be the agent and the position of that candidate agent is updated to the current position, thereby re-identifying the agent.

Abstract: This disclosure describes techniques for determining product volumes associated with products in a facility. These product volumes may be used to update planogram data associated with a facility, with the planogram data indicating inventory locations within the facility for various types of items supported by product fixtures. The planogram data may be used, in some instances, to update virtual carts of users interacting with the items in the facility.

Abstract: This disclosure describes techniques for updating planogram data associated with a facility. The planogram may indicate inventory locations within the facility for various types of items supported by product fixtures. In particular an image of a product fixture is analyzed to identify image segments corresponding to product groups, where each product group consists of instances of the same product and each image segment corresponds to a group of image points. Image data is further analyzed to determine coordinates of the points of each image segment. A product space corresponding to the product group is then defined based on the coordinates of the points of the product group. In some cases, for example, a product space may be defined in terms of the coordinates of the corners of a rectangular bounding box or volume.

Abstract: This disclosure describes techniques for updating planogram data associated with a facility. The planogram may indicate, for different shelves and other inventory locations within the facility, which items are on which shelves. For example, the planogram data may indicate that a particular item is located on a particular shelf. Therefore, when a system identifies that a user has taken an item from that shelf, the system may update a virtual cart of that user to indicate addition of the particular item. In some instances, however, a new item may be stocked on the example shelf instead of a previous item. The techniques described herein may use sensor data generated in the facility to identify this change and update the planogram data to indicate an association between the shelf and the new item.

Abstract: As agents move about a materials handling facility, tracklets representative of the position of each agent are maintained along with a confidence score indicating a confidence that the position of the agent is known. If the confidence score falls below a threshold level, image data of the agent associated with the low confidence score is obtained and processed to generate one or more embedding vectors representative of the agent at a current position. Those embedding vectors are then compared with embedding vectors of other candidate agents to determine a set of embedding vectors having a highest similarity. The candidate agent represented by the set of embedding vectors having the highest similarity score is determined to be the agent and the position of that candidate agent is updated to the current position, thereby re-identifying the agent.

Abstract: This disclosure describes techniques for updating planogram data associated with a facility. The planogram may indicate, for different shelves and other inventory locations within the facility, which items are on which shelves. For example, the planogram data may indicate that a particular item is located on a particular shelf. Therefore, when a system identifies that a user has taken an item from that shelf, the system may update a virtual cart of that user to indicate addition of the particular item. In some instances, however, a new item may be stocked on the example shelf instead of a previous item. The techniques described herein may use sensor data generated in the facility to identify this change and update the planogram data to indicate an association between the shelf and the new item.

Abstract: A computing system for recognizing salient events depicted in a video utilizes learning algorithms to detect audio and visual features of the video. The computing system identifies one or more salient events depicted in the video based on the audio and visual features.

Abstract: Technologies to detect persuasive multimedia content by using affective and semantic concepts extracted from the audio-visual content as well as the sentiment of associated comments are disclosed. The multimedia content is analyzed and compared with a persuasiveness model.

Abstract: Technologies for analyzing temporal components of multimodal data to detect short-term multimodal events, determine relationships between short-term multimodal events, and recognize long-term multimodal events, using a deep learning architecture, are disclosed.

Abstract: A multi-modal interaction modeling system can model a number of different aspects of a human interaction across one or more temporal interaction sequences. Some versions of the system can generate assessments of the nature or quality of the interaction or portions thereof, which can be used to, among other things, provide assistance to one or more of the participants in the interaction.

Abstract: Images are retrieved and ranked according to relevance to attributes of a multi-attribute query through training image attribute detectors for different attributes annotated in a training dataset. Pair-wise correlations are learned between pairs of the annotated attributes from the training dataset of images. Image datasets may are searched via the trained attribute detectors for images comprising attributes in a multi-attribute query. The retrieved images are ranked as a function of comprising attributes that are not within the query subset plurality of attributes but are paired to one of the query subset plurality of attributes by the pair-wise correlations, wherein the ranking is an order of likelihood that the different ones of the attributes will appear in an image with the paired one of the query subset plurality of attributes.

Abstract: Technologies to detect persuasive multimedia content by using affective and semantic concepts extracted from the audio-visual content as well as the sentiment of associated comments are disclosed. The multimedia content is analyzed and compared with a persuasiveness model.

Abstract: Images are retrieved and ranked according to relevance to attributes of a multi-attribute query through training image attribute detectors for different attributes annotated in a training dataset. Pair-wise correlations are learned between pairs of the annotated attributes from the training dataset of images. Image datasets may are searched via the trained attribute detectors for images comprising attributes in a multi-attribute query. The retrieved images are ranked as a function of comprising attributes that are not within the query subset plurality of attributes but are paired to one of the query subset plurality of attributes by the pair-wise correlations, wherein the ranking is an order of likelihood that the different ones of the attributes will appear in an image with the paired one of the query subset plurality of attributes.

Abstract: Technologies for analyzing temporal components of multimodal data to detect short-term multimodal events, determine relationships between short-term multimodal events, and recognize long-term multimodal events, using a deep learning architecture, are disclosed.

Abstract: Images are retrieved and ranked according to relevance to attributes of a multi-attribute query through training image attribute detectors for different attributes annotated in a training dataset. Pair-wise correlations are learned between pairs of the annotated attributes from the training dataset of images. Image datasets may are searched via the trained attribute detectors for images comprising attributes in a multi-attribute query. The retrieved images are ranked as a function of comprising attributes that are not within the query subset plurality of attributes but are paired to one of the query subset plurality of attributes by the pair-wise correlations, wherein the ranking is an order of likelihood that the different ones of the attributes will appear in an image with the paired one of the query subset plurality of attributes.

Abstract: A computing system for recognizing salient events depicted in a video utilizes learning algorithms to detect audio and visual features of the video. The computing system identifies one or more salient events depicted in the video based on the audio and visual features.

Abstract: View-specific object detectors are learned as a function of scene geometry and object motion patterns. Motion directions are determined for object images extracted from a training dataset and collected from different camera scene viewpoints. The object images are categorized into clusters as a function of similarities of their determined motion directions, the object images in each cluster are acquired from the same camera scene viewpoint. Zenith angles are estimated for object image poses in the clusters relative to a position of a horizon in the cluster camera scene viewpoint, and azimuth angles of the poses as a function of a relation of the determined motion directions of the clustered images to the cluster camera scene viewpoint. Detectors are thus built for recognizing objects in input video, one for each of the clusters, and associated with the estimated zenith angles and azimuth angles of the poses of the respective clusters.

Abstract: View-specific object detectors are learned as a function of scene geometry and object motion patterns. Motion directions are determined for object images extracted from a training dataset and collected from different camera scene viewpoints. The object images are categorized into clusters as a function of similarities of their determined motion directions, the object images in each cluster are acquired from the same camera scene viewpoint. Zenith angles are estimated for object image poses in the clusters relative to a position of a horizon in the cluster camera scene viewpoint, and azimuth angles of the poses as a function of a relation of the determined motion directions of the clustered images to the cluster camera scene viewpoint. Detectors are thus built for recognizing objects in input video, one for each of the clusters, and associated with the estimated zenith angles and azimuth angles of the poses of the respective clusters.

Abstract: View-specific object detectors are learned as a function of scene geometry and object motion patterns. Motion directions are determined for object images extracted from a training dataset and collected from different camera scene viewpoints. The object images are categorized into clusters as a function of similarities of their determined motion directions, the object images in each cluster are acquired from the same camera scene viewpoint. Zenith angles are estimated for object image poses in the clusters relative to a position of a horizon in the cluster camera scene viewpoint, and azimuth angles of the poses as a function of a relation of the determined motion directions of the clustered images to the cluster camera scene viewpoint. Detectors are thus built for recognizing objects in input video, one for each of the clusters, and associated with the estimated zenith angles and azimuth angles of the poses of the respective clusters.

Abstract: Images are retrieved and ranked according to relevance to attributes of a multi-attribute query through training image attribute detectors for different attributes annotated in a training dataset. Pair-wise correlations are learned between pairs of the annotated attributes from the training dataset of images. Image datasets may are searched via the trained attribute detectors for images comprising attributes in a multi-attribute query. The retrieved images are ranked as a function of comprising attributes that are not within the query subset plurality of attributes but are paired to one of the query subset plurality of attributes by the pair-wise correlations, wherein the ranking is an order of likelihood that the different ones of the attributes will appear in an image with the paired one of the query subset plurality of attributes.