Abstract: This disclosure includes technologies for action recognition in general. The disclosed system may automatically detect various types of actions in a video, including reportable actions that cause shrinkage in a practical application for loss prevention in the retail industry. Further, appropriate responses may be invoked if a reportable action is recognized. In some embodiments, a three-branch architecture may be used in a machine learning model for action and/or activity recognition. The three-branch architecture may include a main branch for action recognition, an auxiliary branch for learning/identifying an actor (e.g., human parsing) related to an action, and an auxiliary branch for learning/identifying a scene related to an action. In this three-branch architecture, the knowledge of the actor and the scene may be integrated in two different levels for action and/or activity recognition.

Abstract: Methods and systems are provided for generating a multi-label classification system. The multi-label classification system can use a multi-label classification neural network system to identify one or more labels for an image. The multi-label classification system can explicitly take into account the relationship between classes in identifying labels. A relevance sub-network of the multi-label classification neural network system can capture relevance information between the classes. Such a relevance sub-network can decouple independence between classes to focus learning on relevance between the classes.

Abstract: This disclosure includes technologies for object tracking in general. The disclosed system can detect the event type based on one or more tracked objects. Further, appropriate responses may be invoked based on the event type.

Abstract: This disclosure includes technologies for automated verification via digital image transformation or analysis in several progressive stages, which may include a stage of retrieval-model-based mismatch detection, a stage of cross-class mismatch detection, or a stage of inner-class mismatch detection. Further, the disclosed system is designed to progressively execute the verification process from a generality-attentive manner to a specificity-attentive manner. Finally, the disclosed system is designed to launch appropriate responses based on the verification outcome.

Abstract: This disclosure includes technologies for video recognition in general. The disclosed system can automatically detect various types of actions in a video, including reportable actions that cause shrinkage in a practical application for loss prevention in the retail industry. The temporal evolution of spatio-temporal features in the video are used for action recognition. Such features may be learned via a 4D convolutional operation, which is adapted to model low-level features based on a residual 4D block. Further, appropriate responses may be invoked if a reportable action is recognized.

Abstract: This disclosure includes technologies for action recognition in general. The disclosed system may automatically detect various types of actions in a video, including reportable actions that cause shrinkage in a practical application for loss prevention in the retail industry. Further, appropriate responses may be invoked if a reportable action is recognized. In some embodiments, a three-branch architecture may be used in a machine learning model for action and/or activity recognition. The three-branch architecture may include a main branch for action recognition, an auxiliary branch for learning/identifying an actor (e.g., human parsing) related to an action, and an auxiliary branch for learning/identifying a scene related to an action. In this three-branch architecture, the knowledge of the actor and the scene may be integrated in two different levels for action and/or activity recognition.

Abstract: This disclosure includes computer vision technologies for image categorization, such as used for product recognition. In one embodiment, the disclosed system uses a channel interaction network to learn stronger fine-grained features and to distinguish the subtle differences between two similar images. Additionally, the disclosed channel interaction network may be integrated into an existing feature extractor network to boost its performance for image categorization.

Abstract: Methods and systems for training machine vision models (MVMs) with “noisy” training datasets are described. A noisy set of images is received, where labels for some of the images are “noisy” and/or incorrect. A progressively-sequenced learning curriculum is designed for the noisy dataset, where the images that are easiest to learn machine-vision knowledge from are sequenced near the beginning of the curriculum and images that are harder to learn machine-vision knowledge from are sequenced later in the curriculum. An MVM is trained via providing the sequenced curriculum to a supervised learning method, so that the MVM learns from the easiest examples first and the harder training examples later, i.e., the MVM progressively accumulates knowledge from simplest to most complex. To sequence the curriculum, the training images are embedded in a feature space and the “complexity” of each image is determined via density distributions and clusters in the feature space.

Abstract: Aspects of this disclosure include technologies for appearance-flow-based image generation. In applications for pose-guided person image generation or virtual try-on, the disclosed system can model the appearance flow between source and target clothing regions. Further, a cascaded appearance flow estimation network is used to progressively refine the appearance flow. The resulting appearance flow can properly encode the geometric changes between the source and the target for image generation.

Abstract: This disclosure includes computer vision technologies, specifically for embeddings and metric learning. In various practical applications, such as product recognition, image retrieval, face recognition, etc., the disclosed technologies use a cross-batch memory mechanism to memorize prior embeddings, so that a pair-based learning model can mine more pairs across multiple mini-batches or even over the whole dataset. The disclosed technologies not only boost the performance for various applications, but considerably improve the computation itself with its memory-efficient approach.

Abstract: This disclosure includes technologies for ranking or generating compatible objects. In retail-oriented applications, the disclosed technologies can rank products based on their respective compatibilities with contextual products, both in shape and appearance, and facilitate users to select products compatible with contextual products or surrounding conditions. In design-oriented applications, the disclosed technologies can generate diverse objects compatible with contextual objects or surrounding conditions.

Abstract: Methods and systems for training machine vision models (MVMs) with “noisy” training datasets are described. A noisy set of images is received, where labels for some of the images are “noisy” and/or incorrect. A progressively-sequenced learning curriculum is designed for the noisy dataset, where the images that are easiest to learn machine-vision knowledge from are sequenced near the beginning of the curriculum and images that are harder to learn machine-vision knowledge from are sequenced later in the curriculum. An MVM is trained via providing the sequenced curriculum to a supervised learning method, so that the MVM learns from the easiest examples first and the harder training examples later, i.e., the MVM progressively accumulates knowledge from simplest to most complex. To sequence the curriculum, the training images are embedded in a feature space and the “complexity” of each image is determined via density distributions and clusters in the feature space.

Abstract: Aspects of this disclosure include technologies for detecting irregular scans, specifically when a retail system fails to collect the genuine information of a product. The disclosed retail system retrieves images covering a specific region, e.g., the designated scanning area. Further the disclosed retail system uses neural networks to detect the product from such images and track a moving path of the product over the specific region. Irregular scans may be detected when the tracked product in the images does not match what is collected by the scanner of the retail system.

Abstract: Aspects of this disclosure include technologies for character-based text detection and recognition. The disclosed single-stage model is configured for joint text detection and word recognition in natural images. In the disclosed solution, a character recognition branch is integrated into a word detection model. This results in an end-to-end trainable model that can implement text detection and word recognition jointly. Further, the disclosed technical solution includes an iterative character detection method, which is configured to generate character-level bounding boxes on real-world images by using synthetic data first.

Abstract: Aspects of this disclosure include technologies for appearance-flow-based image generation. In applications for pose-guided person image generation or virtual try-on, the disclosed system can model the appearance flow between source and target clothing regions. Further, a cascaded appearance flow estimation network is used to progressively refine the appearance flow. The resulting appearance flow can properly encode the geometric changes between the source and the target for image generation.

Abstract: Aspects of this disclosure include technologies for detecting mislabeled products. In one embodiment, the disclosed system will capture an image of a product when the MRL of the product is scanned or being scanned. After recognizing the product in the image, the size of the area containing the product may be calculated. Subsequently, the disclosed system can determine whether the MRL mismatches the product in the image if this size of the area containing the product does not match the standard size associated with the MRL.

Abstract: Aspects of this disclosure include technologies for object registration based on a dual-stream pyramid registration network, which is configured to compute multi-scale deformation fields from dual feature pyramids. The disclosed technologies further enable the multi-scale deformation fields to be refined in a coarse-to-fine manner, resulting in the capability for handling significant deformations between two objects, such as large displacements in spatial domain or slice space. Further, the disclosed technologies enable various functions based on the registered objects, such as automatic labeling, image comparison and differentiation, and medical image registration and navigation.

Abstract: Weakly supervised instance segmentation refers to the task of training a system to detect object locations and segment instances of the detected objects, where the training data includes only images and image-level labels. This disclosure includes an enhanced pipeline and enhanced training methods that progressively mine pixel-wise labels, when trained via image-level labels. Four cascaded modules are employed, including: a multi-label classification module, an object detection module, an instance refinement module, and instance segmentation module. The modules share a common backbone. The cascaded pipeline is trained alternatively with a curriculum learning strategy which generalizes image level supervision to pixel level supervision, and a post validation training stage, which runs in the inverse order. In the curriculum learning stage, a proposal refinement sub-module is employed to locate object parts and finding key pixels during classification.