Abstract: Certain aspects of the present disclosure provide techniques and apparatus for regenerative learning to enhance dense predictions. In one example method, an input image is accessed. A dense prediction output is generated based on the input image using a dense prediction machine learning (ML) model, and a regenerated version of the input image is generated. A first loss is generated based on the input image and a corresponding ground truth dense prediction, and a second loss is generated based on the regenerated version of the input image. One or more parameters of the dense prediction ML model are updated based on the first and second losses.

Abstract: This disclosure provides systems, methods, and devices for image signal processing that support training object recognition models. In a first aspect, a method of image processing includes training a first modality imaging system; receiving time-synchronized first input data samples and second input data samples from the first modality imaging system and a second modality imaging system, respectively; processing the first input data samples in the first modality imaging system to generate first output; processing the second input data samples in the second modality imaging system to generate second output; and training the second modality imaging system based on the first output and the second output. Other aspects and features are also claimed and described.

Abstract: Methods, systems, and apparatuses for image segmentation are provided. For example, a computing device may obtain an image, and may apply a process to the image to generate input image feature data and input image segmentation data. Further, the computing device may obtain reference image feature data and reference image classification data for a plurality of reference images. The computing device may generate reference image segmentation data based on the reference image feature data, the reference image classification data, and the input image feature data. The computing device may further blend the input image segmentation data and the reference image segmentation data to generate blended image segmentation data. The computing device may store the blended image segmentation data within a data repository. In some examples, the computing device provides the blended image segmentation data for display.

Abstract: Techniques and systems are provided for training a machine learning (ML) model. A technique can include generating a first set of features for objects in images, predicting image feature labels for the first set of features, comparing the predicted image feature labels to ground truth image feature labels to evaluate a first loss function, perform a perspective transform on the first set of features to generate a birds eye view (BEV) projected image features, combining the BEV projected image features and a first set of flattened features to generate combined image features, generating a segmented BEV map of the environment based on the combined image features, comparing the segmented BEV map to a ground truth segmented BEV map to evaluate a second loss function, and training the ML model for generation of segmented BEV maps based on the evaluated first loss function and the evaluated second loss function.

Abstract: Systems and techniques are provided for processing data (e.g., image data). For instance, according to some aspects of the disclosure, a method may include receiving, at a transformer of a machine learning system, learnable queries, keys, and values obtained from a feature map of a segmentation model of the machine learning system. The method may further include learning, via the transformer, a mapping between an unsupervised output and a supervised output of the segmentation model based on the feature map.

Abstract: Systems and techniques are provided for processing one or more images. For instance, according to some aspects of the disclosure, a method may include obtaining an unlabeled image and generating at least one transformed image based on the unlabeled image. The method may include processing the unlabeled image using a pre-trained semantic segmentation model to generate a first segmentation output. The method may further include processing the at least one transformed image using the pre-trained semantic segmentation model to generate at least a second segmentation output. The method may include fine-tuning, based on the first segmentation output and at least the second segmentation output, one or more parameters of the pre-trained semantic segmentation model.

Abstract: Aspects of the present disclosure relate to a novel framework for integrating both semantic and instance contexts for panoptic segmentation. In one example aspect, a method for processing image data includes: processing semantic feature data and instance feature data with a panoptic encoding generator to generate a panoptic encoding; processing the panoptic encoding to generate a panoptic segmentation features; and generating the panoptic segmentation mask based on the panoptic segmentation features.

Abstract: Certain aspects of the present disclosure provide techniques for training neural networks using hierarchical supervision. An example method generally includes training a neural network with a plurality of stages using a training data set and an initial number of classification clusters into which data in the training data set can be classified. A cluster-validation set performance metric is generated for each stage based on a reduced number of classification clusters relative to the initial number of classification clusters and a validation data set. A number of classification clusters to implement at each stage is selected based on the cluster-validation set performance metric and an angle selected relative to the cluster-validation set performance metric for a last stage of the neural network. The neural network is retrained based on the training data set and the selected number of classification clusters for each stage, and the trained neural network is deployed.

Abstract: Certain aspects of the present disclosure provide techniques for cross-task distillation. A depth map is generated by processing an input image using a first machine learning model, and a segmentation map is generated by processing the depth map using a second machine learning model. A segmentation loss is computed based on the segmentation map and a ground-truth segmentation map, and the first machine learning model is refined based on the segmentation loss.

Abstract: A method applies a distance-based loss function to a boundary recognition model. The method classifies boundaries of an input with the boundary recognition model. The method also performs semantic segmentation based on the classifying of the boundaries, and outputting a segmentation map showing different classes of objects from the input, based on the semantic segmentation. The method may train an inverse transforming artificial neural network to predict a perspective transformation of an image so that the trained artificial neural network represents the distance-based loss function. The method may freeze weights of the inverse transforming artificial neural network, after training, to obtain the distance-based loss function. Training of the inverse transforming artificial neural network may include generating shifted, translated, and scaled versions of the image such that a ground truth comprises values corresponding to the amounts of shifting, translating, and scaling.