Abstract: Systems and techniques are described for image processing. An imaging system can include an image sensor that captures image data. An image signal processor (ISP) of the imaging system can demosaic the image data. The imaging system can input the image data into one or more trained machine learning models, in some cases along with metadata associated with the image data. The one or more trained machine learning models can output settings for a set of parameters of the ISP based on the image data and/or the metadata. The imaging system can generate an output image by processing the image data using the ISP, with the parameters of the ISP set according to the settings. Each pixel of the pixels of the image data can be processed using a respective setting for adjusting a corresponding parameter. The parameters of the ISP can include gain, offset, gamma, and Gaussian filtering.

Abstract: Imaging systems and techniques are described. In some examples, an imaging system generates a segmentation map of an image by processing image data associated with the image using a segmentation mapper. Different object types in the image are categorized into different regions in the segmentation map. The imaging system generates an augmented segmentation map by processing at least the segmentation map using a segmentation map error correction engine. The imaging system generates processed image data by processing the image using the augmented segmentation map.

Abstract: Systems and techniques are described herein for displaying information. For instance, a device for displaying information is provided. The device may include at least one memory; and at least one processor coupled to the at least one memory and configured to: detect an object in an image of a scene obtained from a first camera; determine that a user is gazing at a representation of the object displayed at a display based on an image of the user obtained from a second camera; and based on determining that the user is gazing at the representation of the object displayed at the display, display, via the display, information associated with the object.

Abstract: Systems and techniques are described herein for generating image content. For instance, an apparatus for generating image content is provided. The method may include a user interface configured to: display an image of a field of view; and receive a user input indicative of a desired change to the image, wherein the desired change to the image comprises a change to the field of view; and at least one processor configured to: provide at least part of the image and an indication of the desired change as inputs to a generative machine-learning model; obtain an altered image from the generative machine-learning model, wherein the altered image comprises at least part of the image of the field of view and generated pixels outside of the field of view.

Abstract: Systems and techniques are described herein for Processing data. For instance, a method for processing data is provided. The method may include: obtaining image data having a first resolution; downsampling the image data to generate downsampled image data, wherein the downsampled image data has a second resolution that is lower than the first resolution; processing the downsampled image data to generate processed downsampled image data; and generating, using a machine-learning model, upsampled image data based on the processed downsampled image data and the image data.

Abstract: This disclosure provides systems, methods, and devices for image signal processing that support spatially-adaptive binning and remosaicing of image data. In a first aspect, a method of image processing includes receiving image data comprising at least a portion of an image frame. A first portion of the image data is coded according to a first binning pattern, and a second portion of the image data is coded according to a second binning pattern different from the first binning pattern. Other aspects and features are also claimed and described.

Abstract: Techniques are provided for using one or more machine learning systems to process input data including image data. The input data including the image data can be obtained, and at least one machine learning system can be applied to at least a portion of the image data to determine at least one color component value for one or more pixels of at least the portion of the image data. Based on application of the at least one machine learning system to at least the portion of the image data, output image data for a frame of output image data can be generated. The output image data includes at least one color component value for one or more pixels of the frame of output image data. Application of the at least one machine learning system causes the output image data to have a reduced dimensionality relative to the input data.

Abstract: Techniques are provided for using one or more machine learning systems to process input data including image data. The input data including the image data can be obtained, and at least one machine learning system can be applied to at least a portion of the image data to determine at least one color component value for one or more pixels of at least the portion of the image data. Based on application of the at least one machine learning system to at least the portion of the image data, output image data for a frame of output image data can be generated. The output image data includes at least one color component value for one or more pixels of the frame of output image data. Application of the at least one machine learning system causes the output image data to have a reduced dimensionality relative to the input data.

Abstract: Sensor data processing systems and techniques are described. In some examples, a sensor data processing system receives sensor data (e.g., image data). The system receives a sensor data processing parameter that is associated with a sensor data processing function, such as denoising, and that is consistent across the sensor data. The system adjusts a trained machine learning model based on the sensor data processing parameter to generate an adjusted machine learning model. The system processes the sensor data using the adjusted machine learning model to apply the sensor data processing function to the sensor data according to the sensor data processing parameter and to generate processed sensor data (e.g., denoised image data).

Abstract: An imaging system can obtain image data, for instance from an image sensor. The imaging system can supply the image data as input data to a machine learning system, which can generate one or more maps based on the image data. Each map can identify strengths at which a certain image processing function is to be applied to each pixel of the image data. Different maps can be generated for different image processing functions, such as noise reduction, sharpening, or color saturation. The imaging system can generate a modified image based on the image data and the one or more maps, for instance by applying each of one or more image processing functions in accordance with each of the one or more maps. The imaging system can supply the image data and the one or more maps to a second machine learning system to generate the modified image.

Abstract: Techniques are provided for using one or more machine learning systems to process input data including image data. The input data including the image data can be obtained, and at least one machine learning system can be applied to at least a portion of the image data to determine at least one color component value for one or more pixels of at least the portion of the image data. Based on application of the at least one machine learning system to at least the portion of the image data, output image data for a frame of output image data can be generated. The output image data includes at least one color component value for one or more pixels of the frame of output image data. Application of the at least one machine learning system causes the output image data to have a reduced dimensionality relative to the input data.

Abstract: Techniques are provided for using one or more machine learning systems to process input data including image data. The input data including the image data can be obtained, and at least one machine learning system can be applied to at least a portion of the image data to determine at least one color component value for one or more pixels of at least the portion of the image data. Based on application of the at least one machine learning system to at least the portion of the image data, output image data for a frame of output image data can be generated. The output image data includes at least one color component value for one or more pixels of the frame of output image data. Application of the at least one machine learning system causes the output image data to have a reduced dimensionality relative to the input data.

Abstract: Systems and techniques are described for image processing. An imaging system can include an image sensor that captures image data. An image signal processor (ISP) of the imaging system can demosaic the image data. The imaging system can input the image data into one or more trained machine learning models, in some cases along with metadata associated with the image data. The one or more trained machine learning models can output settings for a set of parameters of the ISP based on the image data and/or the metadata. The imaging system can generate an output image by processing the image data using the ISP, with the parameters of the ISP set according to the settings. Each pixel of the pixels of the image data can be processed using a respective setting for adjusting a corresponding parameter. The parameters of the ISP can include gain, offset, gamma, and Gaussian filtering.

Abstract: Systems and techniques are described for image processing. An imaging system can include an image sensor that captures image data. An image signal processor (ISP) of the imaging system can demosaic the image data. The imaging system can input the image data into one or more trained machine learning models, in some cases along with metadata associated with the image data. The one or more trained machine learning models can output settings for a set of parameters of the ISP based on the image data and/or the metadata. The imaging system can generate an output image by processing the image data using the ISP, with the parameters of the ISP set according to the settings. Each pixel of the pixels of the image data can be processed using a respective setting for adjusting a corresponding parameter. The parameters of the ISP can include gain, offset, gamma, and Gaussian filtering.

Abstract: Systems and techniques are described for image processing. An imaging system can include an image sensor that captures image data. An image signal processor (ISP) of the imaging system can demosaic the image data. The imaging system can input the image data into one or more trained machine learning models, in some cases along with metadata associated with the image data. The one or more trained machine learning models can output settings for a set of parameters of the ISP based on the image data and/or the metadata. The imaging system can generate an output image by processing the image data using the ISP, with the parameters of the ISP set according to the settings. Each pixel of the pixels of the image data can be processed using a respective setting for adjusting a corresponding parameter. The parameters of the ISP can include gain, offset, gamma, and Gaussian filtering.

Abstract: Techniques are provided for using one or more machine learning systems to process input data including image data. The input data including the image data can be obtained, and at least one machine learning system can be applied to at least a portion of the image data to determine at least one color component value for one or more pixels of at least the portion of the image data. Based on application of the at least one machine learning system to at least the portion of the image data, output image data for a frame of output image data can be generated. The output image data includes at least one color component value for one or more pixels of the frame of output image data. Application of the at least one machine learning system causes the output image data to have a reduced dimensionality relative to the input data.

Abstract: A method performed by a deep neural network (DNN) includes receiving, at a layer of the DNN during an inference stage, a layer input comprising content associated with a DNN input received at the DNN. The method also includes quantizing one or more parameters of a plurality of parameters associated with the layer based on the content of the layer input. The method further includes performing a task corresponding to the DNN input, the task performed with the one or more one quantized parameters.

Abstract: Techniques are provided for using one or more machine learning systems to process input data including image data. The input data including the image data can be obtained, and at least one machine learning system can be applied to at least a portion of the image data to determine at least one color component value for one or more pixels of at least the portion of the image data. Based on application of the at least one machine learning system to at least the portion of the image data, output image data for a frame of output image data can be generated. The output image data includes at least one color component value for one or more pixels of the frame of output image data. Application of the at least one machine learning system causes the output image data to have a reduced dimensionality relative to the input data.

Abstract: An imaging system can obtain image data, for instance from an image sensor. The imaging system can supply the image data as input data to a machine learning system, which can generate one or more maps based on the image data. Each map can identify strengths at which a certain image processing function is to be applied to each pixel of the image data. Different maps can be generated for different image processing functions, such as noise reduction, sharpening, or color saturation. The imaging system can generate a modified image based on the image data and the one or more maps, for instance by applying each of one or more image processing functions in accordance with each of the one or more maps. The imaging system can supply the image data and the one or more maps to a second machine learning system to generate the modified image.

Abstract: Techniques and systems are provided for providing a rendering engine model for raw media data. In some examples, a system obtains media data captured by a data capturing device and embeds, in a media item containing the media data, a rendering engine model including a description of a neural network configured to process the media data and generate a particular media data output, the description defining a neural network architecture for the neural network. The system then outputs the media item with the rendering engine model embedded in the media item, the rendering engine model indicating how to execute the neural network to process the media data in the media item and generate the particular media data output based on the description of the neural network.