Abstract: A saliency detection explicitly models complementary information between appearance, or color, and depth information in images. A mutual-information minimization is used as a regularizer to reduce the redundancy between appearance features from RGB and geometric features from depth in the latent space. Then the latent features of each of the appearance and geometric modalities are fused to achieve multi-modal feature fusion for saliency detection.

Abstract: Videos captured in low light conditions can be processed in order to identify an activity being performed in the video. The processing may use both the video and audio streams for identifying the activity in the low light video. The video portion is processed to generate a darkness-aware feature which may be used to modulate the features generated from the audio and video features. The audio features may be used to generate a video attention feature and the video features may be used to generate an audio attention feature. The audio and video attention features may also be used in modulating the audio video features. The modulated audio and video features may be used to predict an activity occurring in the video.

Abstract: Repetitive activities can be captured in audio video content. The AV content can be processed in order to predict the number of repetitive activities present in the AV content. The accuracy of the predicted number may be improved, especially for AV content with challenging conditions, by basing the predictions on both the audio and video portions of the AV content.

Abstract: A transformer based vision-linguistic (VL) model and training technique uses a number of different image patches covering the same portion of an image, along with a text description of the image to train the model. The model and pre-training techniques may be used in domain specific training of the model. The model can be used for fine-grained image-text tasks in the fashion domain.

Abstract: This disclosure relates to improved techniques for performing human parsing functions using neural network architectures. The neural network architecture can model human objects in images using a hierarchal graph of interconnected nodes that correspond to anatomical features at various levels. Multi-level inference information can be generated for each of the nodes using separate inference processes. The multi-level inference information for each node can be combined or fused to generate final predictions for each of the nodes. Parsing results may be generated based on the final predictions.

Abstract: This disclosure relates to improved techniques for performing computer vision functions, including common object detection and instance segmentation. The techniques described herein utilize neural network architectures to perform these functions in various types of images, such as natural images, UAV images, satellite images, and other images. The neural network architecture can include a dense location regression network that performs object localization and segmentation functions, at least in part, by generating offset information for multiple sub-regions of candidate object proposals, and utilizing this dense offset information to derive final predictions for locations of target objects. The neural network architecture also can include a discriminative region-of-interest (RoI) pooling network that performs classification of the localized objects, at least in part, by sampling various sub-regions of candidate proposals and performing adaptive weighting to obtain discriminative features.

Abstract: This disclosure relates to improved techniques for performing computer vision functions including common object counting and instance segmentation. The techniques described herein utilize a neural network architecture to perform these functions. The neural network architecture can be trained using image-level supervision techniques that utilize a loss function to jointly train an image classification branch and a density branch of the neural network architecture. The neural network architecture constructs per-category density maps that can be used to generate analysis information comprising global object counts and locations of objects in images.

Abstract: This disclosure relates to improved techniques for synthesizing three-dimensional (3D) scenes. The techniques can utilize a neural network architecture to analyze images for detecting objects, classifying scenes and objects, and determining degree of freedom information for objects in the images. These tasks can be performed by, at least in part, using inter-object and object-scene dependency information that captures the spatial correlations and dependencies among objects in the images, as well as the correlations and relationships of objects to scenes associated with the images. 3D scenes corresponding to the images can then be synthesized using the inferences provided by the neural network architecture.