Abstract: This disclosure includes technologies for action recognition in general. The disclosed system may automatically detect various types of actions in a video, including reportable actions that cause shrinkage in a practical application for loss prevention in the retail industry. Further, appropriate responses may be invoked if a reportable action is recognized. In some embodiments, a three-branch architecture may be used in a machine learning model for action and/or activity recognition. The three-branch architecture may include a main branch for action recognition, an auxiliary branch for learning/identifying an actor (e.g., human parsing) related to an action, and an auxiliary branch for learning/identifying a scene related to an action. In this three-branch architecture, the knowledge of the actor and the scene may be integrated in two different levels for action and/or activity recognition.

Abstract: Methods and systems are provided for generating a multi-label classification system. The multi-label classification system can use a multi-label classification neural network system to identify one or more labels for an image. The multi-label classification system can explicitly take into account the relationship between classes in identifying labels. A relevance sub-network of the multi-label classification neural network system can capture relevance information between the classes. Such a relevance sub-network can decouple independence between classes to focus learning on relevance between the classes.

Abstract: This disclosure includes technologies for object tracking in general. The disclosed system can detect the event type based on one or more tracked objects. Further, appropriate responses may be invoked based on the event type.

Abstract: This disclosure includes technologies for automated verification via digital image transformation or analysis in several progressive stages, which may include a stage of retrieval-model-based mismatch detection, a stage of cross-class mismatch detection, or a stage of inner-class mismatch detection. Further, the disclosed system is designed to progressively execute the verification process from a generality-attentive manner to a specificity-attentive manner. Finally, the disclosed system is designed to launch appropriate responses based on the verification outcome.

Abstract: This disclosure includes computer vision technologies for image categorization, such as used for product recognition. In one embodiment, the disclosed system uses a channel interaction network to learn stronger fine-grained features and to distinguish the subtle differences between two similar images. Additionally, the disclosed channel interaction network may be integrated into an existing feature extractor network to boost its performance for image categorization.

Abstract: This disclosure includes technologies for video recognition in general. The disclosed system can automatically detect various types of actions in a video, including reportable actions that cause shrinkage in a practical application for loss prevention in the retail industry. The temporal evolution of spatio-temporal features in the video are used for action recognition. Such features may be learned via a 4D convolutional operation, which is adapted to model low-level features based on a residual 4D block. Further, appropriate responses may be invoked if a reportable action is recognized.

Abstract: This disclosure includes technologies for action recognition in general. The disclosed system may automatically detect various types of actions in a video, including reportable actions that cause shrinkage in a practical application for loss prevention in the retail industry. Further, appropriate responses may be invoked if a reportable action is recognized. In some embodiments, a three-branch architecture may be used in a machine learning model for action and/or activity recognition. The three-branch architecture may include a main branch for action recognition, an auxiliary branch for learning/identifying an actor (e.g., human parsing) related to an action, and an auxiliary branch for learning/identifying a scene related to an action. In this three-branch architecture, the knowledge of the actor and the scene may be integrated in two different levels for action and/or activity recognition.

Abstract: Aspects of this disclosure include technologies for monitoring retail transactions, including regular and irregular transactions associated with a check-out machine. The disclosed technical solution utilizes various GUI elements, their configurations, and their interactions with a user to present retail transactions and their information thereof.

Abstract: Aspects of this disclosure include technologies for appearance-flow-based image generation. In applications for pose-guided person image generation or virtual try-on, the disclosed system can model the appearance flow between source and target clothing regions. Further, a cascaded appearance flow estimation network is used to progressively refine the appearance flow. The resulting appearance flow can properly encode the geometric changes between the source and the target for image generation.

Abstract: This disclosure includes computer vision technologies, specifically for embeddings and metric learning. In various practical applications, such as product recognition, image retrieval, face recognition, etc., the disclosed technologies use a cross-batch memory mechanism to memorize prior embeddings, so that a pair-based learning model can mine more pairs across multiple mini-batches or even over the whole dataset. The disclosed technologies not only boost the performance for various applications, but considerably improve the computation itself with its memory-efficient approach.

Abstract: This disclosure includes technologies for ranking or generating compatible objects. In retail-oriented applications, the disclosed technologies can rank products based on their respective compatibilities with contextual products, both in shape and appearance, and facilitate users to select products compatible with contextual products or surrounding conditions. In design-oriented applications, the disclosed technologies can generate diverse objects compatible with contextual objects or surrounding conditions.

Abstract: Aspects of this disclosure include technologies for detecting irregular scans, specifically when a retail system fails to collect the genuine information of a product. The disclosed retail system retrieves images covering a specific region, e.g., the designated scanning area. Further the disclosed retail system uses neural networks to detect the product from such images and track a moving path of the product over the specific region. Irregular scans may be detected when the tracked product in the images does not match what is collected by the scanner of the retail system.

Abstract: Aspects of this disclosure include technologies for character-based text detection and recognition. The disclosed single-stage model is configured for joint text detection and word recognition in natural images. In the disclosed solution, a character recognition branch is integrated into a word detection model. This results in an end-to-end trainable model that can implement text detection and word recognition jointly. Further, the disclosed technical solution includes an iterative character detection method, which is configured to generate character-level bounding boxes on real-world images by using synthetic data first.

Abstract: Aspects of this disclosure include technologies for appearance-flow-based image generation. In applications for pose-guided person image generation or virtual try-on, the disclosed system can model the appearance flow between source and target clothing regions. Further, a cascaded appearance flow estimation network is used to progressively refine the appearance flow. The resulting appearance flow can properly encode the geometric changes between the source and the target for image generation.

Abstract: Aspects of this disclosure include technologies for object registration based on a dual-stream pyramid registration network, which is configured to compute multi-scale deformation fields from dual feature pyramids. The disclosed technologies further enable the multi-scale deformation fields to be refined in a coarse-to-fine manner, resulting in the capability for handling significant deformations between two objects, such as large displacements in spatial domain or slice space. Further, the disclosed technologies enable various functions based on the registered objects, such as automatic labeling, image comparison and differentiation, and medical image registration and navigation.

Abstract: Aspects of this disclosure include technologies for detecting mislabeled products. In one embodiment, the disclosed system will capture an image of a product when the MRL of the product is scanned or being scanned. After recognizing the product in the image, the size of the area containing the product may be calculated. Subsequently, the disclosed system can determine whether the MRL mismatches the product in the image if this size of the area containing the product does not match the standard size associated with the MRL.

Abstract: Weakly supervised instance segmentation refers to the task of training a system to detect object locations and segment instances of the detected objects, where the training data includes only images and image-level labels. This disclosure includes an enhanced pipeline and enhanced training methods that progressively mine pixel-wise labels, when trained via image-level labels. Four cascaded modules are employed, including: a multi-label classification module, an object detection module, an instance refinement module, and instance segmentation module. The modules share a common backbone. The cascaded pipeline is trained alternatively with a curriculum learning strategy which generalizes image level supervision to pixel level supervision, and a post validation training stage, which runs in the inverse order. In the curriculum learning stage, a proposal refinement sub-module is employed to locate object parts and finding key pixels during classification.

Abstract: Aspects of the technology described herein provide a system for improved synthesis of a target domain image from a source domain image. A generator that performs the synthesis is formed based on texture propagation from the first domain to the second domain by making use of a bidirectional generative adversarial network. A framework is provided for training that includes texture propagation with a shape prior constraint.

Abstract: Aspects of the technology described herein provide a system for improved synthesis of a target domain image from a source domain image. A generator that performs the synthesis is formed based on the texture propagation from the first domain to the second domain with a bidirectional generative adversarial network, which is trained for the texture propagation with a shape prior constraint.

Abstract: An input method editor (IME) provides a distributed platform architecture that enables associating multiple applications with the IME to provide extended functionalities. The presentations of the applications, such as skins, may be different from each other and that of the IME. The applications may be represented in a manifest file that is human-readable and editable. The IME collects multiple parameters relating to a user input into a host application including a query input by the user and a scenario of the host application, and selects one or more applications to provide candidates based on a score or ranking of the applications under the collected multiple parameters. Machine-learning may be used to improve the score or ranking. The candidates may include text candidates, rich candidates, and informative candidates.