Abstract: A processor-implemented method for selective parameter efficient fine-tuning (PEFT) includes receiving a large language model (LLM). The LLM has multiple layers with each layer having a set of parameters. A subset of the parameters are identified to fine-tune for a downstream task based on a score function. An adapter is applied to the identified subset of the parameters to fine-tune. Only the identified subset of the parameters is fine-tuned.

Abstract: Systems and techniques are provided for processing data (e.g., image data). For instance, according to some aspects of the disclosure, a method may include receiving, at a transformer of a machine learning system, learnable queries, keys, and values obtained from a feature map of a segmentation model of the machine learning system. The method may further include learning, via the transformer, a mapping between an unsupervised output and a supervised output of the segmentation model based on the feature map.

Abstract: A processor-implemented method performed for text-based video editing includes receiving a video input and a text prompt. The video input includes a sequence of video frames. Features of the video input are extracted to generate a latent representation of the video input. Noise is injected to the latent representation of the video input to generate a noise injected latent. The noise is conditioned on the video input. An artificial neural network (ANN) model processes the noise injected latent based on the text prompt to adapt the video input according to the text prompt.

Abstract: Certain aspects of the present disclosure provide techniques for generating an output image based on a text prompt. A method may include receiving the text prompt; providing a user interface comprising one or more input elements associated with one or more words of the text prompt; receiving input corresponding to at least one of the one or more input elements, the input indicating a semantic importance for each of at least one of the one or more words associated with the at least one of the one or more input elements; and generating the output image based on the text prompt and the input.

Abstract: Techniques and apparatus for generating content with multiple specified attributes using a generative artificial intelligence model are described. An example method generally includes receiving a request to generate an output of a machine learning model, the request specifying a plurality of attributes of the output of the machine learning model. A set of intermediate outputs is generated via a plurality of adapters of the machine learning model. Each respective adapter of the plurality of adapters may be associated with a respective attribute of the specified plurality of attributes and include a respective mask in a low-rank dimension associated with the respective adapter. The set of intermediate outputs is merged into a combined output of the plurality of adapters of the machine learning model, and the output of the machine learning model is generated based on the combined output of the plurality of adapters.

Abstract: Techniques and apparatus for generating content with multiple specified attributes using a generative artificial intelligence model are described. An example method generally includes receiving a request to generate an output of a machine learning model, the request specifying a plurality of attributes of the output of the machine learning model. A set of intermediate outputs is generated via a plurality of adapters of the machine learning model. Each respective adapter of the plurality of adapters may be associated with a respective attribute of the specified plurality of attributes and include a respective mask in a low-rank dimension associated with the respective adapter. The set of intermediate outputs is merged into a combined output of the plurality of adapters of the machine learning model, and the output of the machine learning model is generated based on the combined output of the plurality of adapters.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method of image processing includes receiving a plurality of image frames by a computing device and using machine learning models to identify corrupted or occluded image frames. A first machine learning model may identify corrupted image frames, while a second machine learning model may identify partially occluded image frames. The method may further include generating updated versions of image frames captured by vehicle cameras, such as based on feature vectors from the first and second machine learning models. The feature vectors may be fused and provided to a third machine learning model to generate updated versions of occluded image frames. The method may further include determining vehicle control instructions based on the updated versions. Other aspects and features are also claimed and described.

Abstract: Certain aspects of the present disclosure provide techniques and apparatus for feature masking. A feature tensor is accessed in a neural network, and a feature mask is generated by processing the feature tensor using a masking subnetwork, where the masking subnetwork was trained based at least in part on a polarization constraint and an activation constraint to generate feature masks. A masked feature tensor is generated based on the feature tensor and the feature mask, and an output inference is generated using the neural network based at least in part on the masked feature tensor.

Abstract: Techniques and systems are provided for performing online adaptation of machine learning model(s). For example, a process may include obtaining features extracted from a image by a machine learning model during inference and determining, by the machine learning model based on the features during inference, a plurality of keypoint estimates in the image and/or a bounding region estimate associated with an object in the image. The process may further include generating pseudo-label(s) based on the plurality of keypoint estimates and/or the bounding region estimate. The process may include determining at least one self-supervised loss based on the plurality of keypoint estimates and/or the bounding region estimate. The process may further include adapting, based on the at least one self-supervised loss, parameter(s) of the machine learning model. The process may include generating, using the machine learning model with the adapted parameter(s), a segmentation mask for the image (or another image).

Abstract: Certain aspects of the present disclosure provide techniques and apparatus for improved machine learning. In an example method, a first latent tensor generated during a first iteration of processing data using a denoising backbone of a diffusion machine learning model is accessed. A guidance scale is generated based on processing the first latent tensor using a guidance machine learning model. A second latent tensor is generated during a second iteration of processing data using the denoising backbone based on the first latent tensor and the first guidance scale, and an output from the diffusion machine learning model is generated based at least in part on the second latent tensor.

Abstract: Certain aspects of the present disclosure provide techniques and apparatus for machine learning. In an example method, a first adapted machine learning model comprising a first base model and an adapter trained for the first base model is accessed. A second base model is accessed. One or more linear projections are generated for the second base model based on the first base model, where the one or more linear projections align tensors generated by the second base model with tensors generated by the first base model. A projected base model is generated based on the second base model and the one or more linear projections. A second adapted machine learning model comprising the projected base model and the adapter is generated.

Abstract: Certain aspects of the present disclosure provide techniques and apparatus for machine learning. In an example method, a first adapted machine learning model comprising a first base model and an adapter trained for the first base model is accessed. One or more adapter components are generated based on projecting the adapter to a range space and a null space of the first base model. A second base model is accessed, and a projected adapter is generated based on projecting the one or more adapter components to a range space and a null space of the second base model. A second adapted machine learning model comprising the second base model and the projected adapter is generated, and a machine learning model output is generated using the second adapted machine learning model.

Abstract: A system stores first and second images generated by first and second cameras; applies a segmentation model to the first image to generate a first segmentation mask identifying object instances; applies the segmentation model to the second image to generate a second segmentation mask identifying the object instances; projects the first segmentation mask to a viewpoint of the second camera to generate a first projected segmentation mask; converts the first projected segmentation mask and the second segmentation mask to first and second semantic masks, respectively; and computes a first similarity value based on the first and second semantic masks. This may be repeated exchanging the first and second images to compute a second similarity value. The system determines a loss value based on the first similarity value and the second similarity value and trains the segmentation model based on the loss value.

Abstract: A method receives first and second data generated from a first and second domains including first and second set of objects, receiving first class labels for each of the first set of objects, and receiving second class labels for each of the second set of objects. The method generates a training dataset by augmenting the first data and corresponding first class labels, and locally updating neural network parameters of a model based on the training dataset. The method generates a validation dataset by augmenting the second data and corresponding second class labels, and globally updating the neural network parameters of the model based on the validation dataset. The method also generates multiple target labels for target data generated from a target domain including a third set of objects after globally updating the neural network parameters of the model based on the validation dataset.

Abstract: A method includes receiving one or more images, wherein at least one of the one or more images depicts a water region and analyzing, by one or more processors, the one or more images using a first machine learning model to determine a depth of the water region. The method also includes analyzing, by the one or more processors, the one or more images using a second machine learning model to determine a surface normal of the water region and performing, by the one or more processors, using a third machine learning model, multi-class segmentation of the one or more images. Additionally, the method includes performing one or more fusion operations on outputs of at least two of the first machine learning model, the second machine learning model and the third machine learning model to generate a classification of the water region.

Abstract: Aspects of the present disclosure relate to a novel framework for integrating both semantic and instance contexts for panoptic segmentation. In one example aspect, a method for processing image data includes: processing semantic feature data and instance feature data with a panoptic encoding generator to generate a panoptic encoding; processing the panoptic encoding to generate a panoptic segmentation features; and generating the panoptic segmentation mask based on the panoptic segmentation features.

Abstract: Systems and techniques are described herein for training and using a machine-learning model (e.g., a neural network). For example, a computing device can: process, using a first trained neural network, data specific to a user to obtain intermediate activation data representing the data, the first trained neural network comprising a plurality of neural network layers; process, using a second trained neural network, the intermediate activation data to generate an output representing the data, the second trained neural network comprising a subset of neural network layers from the plurality of neural network layers of the first trained neural network; determine a loss based on the output; and update parameters of the second trained neural network based on the loss.

Abstract: Certain aspects of the present disclosure provide techniques for domain adaptation. An input tensor comprising channel state information (CSI) for a wireless signal is determined, where each channel in the input tensor corresponds to a respective degree of freedom (DoF) in the wireless signal. A domain-adapted tensor is generated by processing the input tensor using a domain-adaptation network comprising, for each respective DoF in the wireless signal, a respective convolution path. The domain-adapted tensor is provided to a neural network trained for position estimation.

Abstract: Systems and techniques are described herein for detecting objects. For instance, a method for detecting objects is provided. The method may include generating first image features based on a first image; generating second image features based on a second image; aligning the first image features and the second image features to generate aligned features; estimating motion parameters based on the aligned features; adjusting the aligned features based on the motion parameters to generate adjusted aligned image features; and detecting an object based on the adjusted aligned image features.

Abstract: A processor-implemented method performed for text-based video editing includes receiving a video input and a text prompt. The video input includes a sequence of video frames. Features of the video input are extracted to generate a latent representation of the video input. Noise is injected to the latent representation of the video input to generate a noise injected latent. The noise is conditioned on the video input. An artificial neural network (ANN) model processes the noise injected latent based on the text prompt to adapt the video input according to the text prompt.