Abstract: Techniques and systems are provided for performing online adaptation of machine learning model(s). For example, a process may include obtaining features extracted from a image by a machine learning model during inference and determining, by the machine learning model based on the features during inference, a plurality of keypoint estimates in the image and/or a bounding region estimate associated with an object in the image. The process may further include generating pseudo-label(s) based on the plurality of keypoint estimates and/or the bounding region estimate. The process may include determining at least one self-supervised loss based on the plurality of keypoint estimates and/or the bounding region estimate. The process may further include adapting, based on the at least one self-supervised loss, parameter(s) of the machine learning model. The process may include generating, using the machine learning model with the adapted parameter(s), a segmentation mask for the image (or another image).

Abstract: A method receives first and second data generated from a first and second domains including first and second set of objects, receiving first class labels for each of the first set of objects, and receiving second class labels for each of the second set of objects. The method generates a training dataset by augmenting the first data and corresponding first class labels, and locally updating neural network parameters of a model based on the training dataset. The method generates a validation dataset by augmenting the second data and corresponding second class labels, and globally updating the neural network parameters of the model based on the validation dataset. The method also generates multiple target labels for target data generated from a target domain including a third set of objects after globally updating the neural network parameters of the model based on the validation dataset.

Abstract: A processor-implemented method comprises processing a single level cost volume across multiple processing stages by varying a receptive field across each of the processing stages. The method also includes performing a learning-based correspondence estimation task based on the processing. The varying may include processing a different resolution of the cost volume at each processing stage while maintaining a same neighborhood sampling radius. The resolution may increase from a first processing stage to a later processing stage. The varying may also include varying a neighborhood sampling radius at each of the processing stages while maintaining a same resolution. The task may be optical flow estimation or stereo estimation.

Abstract: Techniques and systems are provided for training a machine learning (ML) model. A technique can include generating a first set of features for objects in images, predicting image feature labels for the first set of features, comparing the predicted image feature labels to ground truth image feature labels to evaluate a first loss function, perform a perspective transform on the first set of features to generate a birds eye view (BEV) projected image features, combining the BEV projected image features and a first set of flattened features to generate combined image features, generating a segmented BEV map of the environment based on the combined image features, comparing the segmented BEV map to a ground truth segmented BEV map to evaluate a second loss function, and training the ML model for generation of segmented BEV maps based on the evaluated first loss function and the evaluated second loss function.

Abstract: Certain aspects of the present disclosure provide a method, including: processing input data with a feature extraction stage of a machine learning model to generate a feature map; applying an attention map to the feature map to generate an augmented feature map; processing the augmented feature map with a refinement stage of the machine learning model to generate a refined feature map; processing the refined feature map with a first regression stage of the machine learning model to generate multi-dimensional task output data; and processing the refined feature data with an attention stage of the machine learning model to generate an updated attention map.

Abstract: Certain aspects of the present disclosure provide techniques for desparsified convolution. An activation tensor is received, and a convolution output is generated for the activation tensor, comprising: selecting a subset of weight elements, corresponding to a set of non-zero elements in the activation tensor, from a weight tensor, and multiplying the set of non-zero elements and the set of weight elements.

Abstract: Systems and techniques are provided for processing one or more images. For instance, according to some aspects of the disclosure, a method may include obtaining an unlabeled image and generating at least one transformed image based on the unlabeled image. The method may include processing the unlabeled image using a pre-trained semantic segmentation model to generate a first segmentation output. The method may further include processing the at least one transformed image using the pre-trained semantic segmentation model to generate at least a second segmentation output. The method may include fine-tuning, based on the first segmentation output and at least the second segmentation output, one or more parameters of the pre-trained semantic segmentation model.

Abstract: Systems and techniques are provided for processing data (e.g., image data). For instance, according to some aspects of the disclosure, a method may include receiving, at a transformer of a machine learning system, learnable queries, keys, and values obtained from a feature map of a segmentation model of the machine learning system. The method may further include learning, via the transformer, a mapping between an unsupervised output and a supervised output of the segmentation model based on the feature map.

Abstract: Certain aspects of the present disclosure provide techniques for machine learning-based deblurring. An input image is received, and a deblurred image is generated based on the input image using a neural network, comprising: generating a feature tensor by processing the input image using a first portion of the neural network, generating a motion mask by processing the feature tensor using a motion portion of the neural network, and generating the deblurred image by processing the feature tensor and the motion mask using a deblur portion of the neural network.

Abstract: Certain aspects of the present disclosure provide techniques for improved domain adaptation in machine learning. A feature tensor is generated by processing input data using a feature extractor. A first set of logits is generated by processing the feature tensor using a domain-agnostic classifier, and a second set of logits is generated by processing the feature tensor using a domain-specific classifier. A loss is computed based at least in part on the first set of logits and the second set of logits, where the loss includes a divergence loss component. The feature extractor, the domain-agnostic classifier, and the domain-specific classifier are refined using the loss.

Abstract: Certain aspects of the present disclosure provide techniques and apparatus for biometric authentication using an anti-spoofing protection model refined using online data. The method generally includes receiving a biometric data input for a user. Features for the received biometric data input are extracted through a first machine learning model. It is determined, using the extracted features for the received biometric data input and a second machine learning model, whether the received biometric data input for the user is authentic or inauthentic. It is determined whether to add the extracted features for the received biometric data input, labeled with an indication of whether the received biometric data input is authentic or inauthentic, to a finetuning data set. The second machine learning model is adjusted based on the finetuning data set.

Abstract: Certain aspects of the present disclosure provide techniques for efficient bottleneck processing via dimensionality transformation. The techniques include receiving a tensor, and processing the tensor in a bottleneck block in a neural network model, comprising applying a space-to-depth tensor transformation, applying a depthwise convolution, and applying a depth-to-space tensor transformation.

Abstract: Certain aspects of the present disclosure provide techniques for generating fine depth maps for images of a scene based on semantic segmentation and segment-based refinement neural networks. An example method generally includes generating, through a segmentation neural network, a segmentation map based on an image of a scene. The segmentation map generally comprises a map segmenting the scene into a plurality of regions, and each region of the plurality of regions is generally associated with one of a plurality of categories. A first depth map of the scene is generated through a first depth neural network based on a depth measurement of the scene. A second depth map of the scene is generated through a depth refinement neural network based on the segmentation map and the first depth map. One or more actions are taken based on the second depth map of the scene.

Abstract: Systems and techniques are provided for unsupervised scene-decompositional normalizing flows. An example process can include obtaining a scene-decompositional model having a normalizing flow neural network architecture. The process can include determining, based on processing data depicting multiple targets in a scene using the scene-decompositional model, a distribution of scene data as a mixture of flows from one or more background components and one or more foreground components. The process can further include identifying, based on processing the distribution of scene data using the scene-decompositional model, a target associated with the one or more foreground components and included in the data depicting the multiple targets in the scene.

Abstract: Systems and techniques are described herein for performing optical flow estimation for one or more frames. For example, a process can include determining an optical flow prediction associated with a plurality of frames. The process can include determining a position of at least one feature associated with a first frame and determining, based on the position of the at least one feature in the first frame and the optical flow prediction, a position estimate of a search area for searching for the at least one feature in a second frame.

Abstract: Certain aspects of the present disclosure provide techniques and apparatus for efficient processing of visual content using machine learning models. An example method generally includes generating, from an input, an embedding tensor for the input. The embedding tensor for the input is projected into a reduced-dimensional space projection of the embedding tensor based on a projection matrix. An attention value for the input is derived based on the reduced-dimensional space projection of the embedding tensor and a non-linear attention function. A match, in the reduced-dimensional space, is identified between a portion of the input and a corresponding portion of a target against which the input is evaluated based on the attention value for the input. One or more actions are taken based on identifying the match.

Abstract: Aspects of the present disclosure relate to a novel framework for integrating both semantic and instance contexts for panoptic segmentation. In one example aspect, a method for processing image data includes: processing semantic feature data and instance feature data with a panoptic encoding generator to generate a panoptic encoding; processing the panoptic encoding to generate a panoptic segmentation features; and generating the panoptic segmentation mask based on the panoptic segmentation features.

Abstract: Certain aspects of the present disclosure provide techniques method for self-supervised training of a machine learning model to predict the location of a device in a spatial environment, such as a spatial environment including multiple discrete planes. An example method generally includes receiving an input data set of scene data. A generator model is trained to map scene data in the input data set to points in three-dimensional space. One or more critic models are trained to backpropagate a gradient to the generator model to push the points in the three-dimensional space to one of a plurality of planes in the three-dimensional space. At least the generator is deployed.

Abstract: A processor-implemented method of video processing using includes receiving, via an artificial neural network (ANN), a video including a first frame and a second frame. A saliency map is generated based on the first frame of the video. The second frame of the video is sampled based on the saliency map. A first portion of the second frame is sampled at a first resolution and a second portion of the second frame is sampled at a second resolution. The first resolution is different than the second resolution. A resampled second frame is generated based on the sampling of the second frame. The resampled second frame is processed to determine an inference associated with the video.

Abstract: Systems and techniques are described herein for performing optical flow estimation for one or more frames. For example, a process can include determining an optical flow prediction associated with a plurality of frames. The process can include determining a position of at least one feature associated with a first frame and determining, based on the position of the at least one feature in the first frame and the optical flow prediction, a position estimate of a search area for searching for the at least one feature in a second frame. The process can include determining, from within the search area, a position of the at least one feature in the second frame.