Abstract: Systems and techniques are described herein for processing data (e.g., image data) using convolution as a transformer (CAT) operations. The method includes receiving, at a convolution engine of a machine learning system, a first set of features, the first set of features being associated with an image and having a three-dimensional shape, applying, via the convolution engine, a depth-wise separable convolutional filter to the first set of features to generate a first output, applying, via the convolution engine, a pointwise convolutional filter to the first output to generate a second output based on global information from a spatial dimension and a channel dimension associated with the image, modifying the second output to the three-dimensional shape to generate a second set of features and combining the first set of features and the second set of features to generate an output set of features.

Abstract: Systems, methods, and non-transitory media are provided for predictive camera initialization. An example method can include obtain, from a first image capture device, image data depicting a scene; classify the scene based on the image data; based on the classification of the scene, predict a camera use event; and based on the predicted camera use event, adjust a power mode of at least one of the first image capture device and a second image capture device.

Abstract: Methods, systems, and apparatuses for image segmentation are provided. For example, a computing device may obtain an image, and may apply a process to the image to generate input image feature data and input image segmentation data. Further, the computing device may obtain reference image feature data and reference image classification data for a plurality of reference images. The computing device may generate reference image segmentation data based on the reference image feature data, the reference image classification data, and the input image feature data. The computing device may further blend the input image segmentation data and the reference image segmentation data to generate blended image segmentation data. The computing device may store the blended image segmentation data within a data repository. In some examples, the computing device provides the blended image segmentation data for display.

Abstract: Systems, methods, and non-transitory media are provided for predictive camera initialization. An example method can include obtain, from a first image capture device, image data depicting a scene; classify the scene based on the image data; based on the classification of the scene, predict a camera use event; and based on the predicted camera use event, adjust a power mode of at least one of the first image capture device and a second image capture device.

Abstract: Systems and techniques are provided for processing one or more frames. For example, a process can include obtaining a first plurality of frames associated with a first settings domain from an image capture system, wherein the first plurality of frames is captured prior to obtaining a capture input. The process can include obtaining a reference frame associated with a second settings domain from the image capture system, wherein the reference frame is captured proximate to obtaining the capture input. The process can include obtaining a second plurality of frames associated with the second settings domain from the image capture system, wherein the second plurality of frames is captured after the reference frame. The process can include, based on the reference frame, transforming at least a portion of the first plurality of frames to generate a transformed plurality of frames associated with the second settings domain.

Abstract: Systems and techniques are described for imaging. An imaging system includes an image sensor with a plurality of photodetectors, grouped into a first group of photodetectors and a second group of photodetectors. The imaging system can reset its image sensor. The imaging system exposes its image sensor to light from a scene. The plurality of photodetectors convert the light into charge. The imaging system stores analog photodetector signals corresponding to the charge from each the photodetectors. The imaging system reads first digital pixel data from a first subset of the analog photodetector signals corresponding to the first group of photodetectors without reading second digital pixel data from a second subset of the analog photodetector signals corresponding to the second group of photodetectors. The imaging system generates an image of the scene using the first digital pixel data.

Abstract: Systems and techniques are described for processing image data to generate an image with a synthetic depth of field (DoF). An imaging system receives first image data of a scene captured by a first image sensor. The imaging system receives second image data of the scene captured by a second image sensor. The first image sensor is offset from the second image sensor by an offset distance. The imaging system generates, using at least the first image data and the second image data as inputs to one or more trained machine learning systems, an image having a synthetic depth of field corresponding to a simulated aperture size. The simulated aperture size is associated with the offset distance. The imaging system outputs the image.

Abstract: Systems and techniques are provided for processing one or more frames. For example, a process can include obtaining a first plurality of frames associated with a first settings domain from an image capture system, wherein the first plurality of frames is captured prior to obtaining a capture input. The process can include obtaining a reference frame associated with a second settings domain from the image capture system, wherein the reference frame is captured proximate to obtaining the capture input. The process can include obtaining a second plurality of frames associated with the second settings domain from the image capture system, wherein the second plurality of frames is captured after the reference frame. The process can include, based on the reference frame, transforming at least a portion of the first plurality of frames to generate a transformed plurality of frames associated with the second settings domain.

Abstract: Systems and techniques are described for processing image data to generate an image with a synthetic depth of field (DoF). An imaging system receives first image data of a scene captured by a first image sensor. The imaging system receives second image data of the scene captured by a second image sensor. The first image sensor is offset from the second image sensor by an offset distance. The imaging system generates, using at least the first image data and the second image data as inputs to one or more trained machine learning systems, an image having a synthetic depth of field corresponding to a simulated aperture size. The simulated aperture size is associated with the offset distance. The imaging system outputs the image.

Abstract: Systems and techniques are provided for processing one or more frames. For example, a process can include obtaining a first plurality of frames associated with a first settings domain from an image capture system, wherein the first plurality of frames is captured prior to obtaining a capture input. The process can include obtaining a reference frame associated with a second settings domain from the image capture system, wherein the reference frame is captured proximate to obtaining the capture input. The process can include obtaining a second plurality of frames associated with the second settings domain from the image capture system, wherein the second plurality of frames is captured after the reference frame. The process can include, based on the reference frame, transforming at least a portion of the first plurality of frames to generate a transformed plurality of frames associated with the second settings domain.

Abstract: Systems and techniques are described for imaging. An imaging system includes an image sensor with a plurality of photodetectors, grouped into a first group of photodetectors and a second group of photodetectors. The imaging system can reset its image sensor. The imaging system exposes its image sensor to light from a scene. The plurality of photodetectors convert the light into charge. The imaging system stores analog photodetector signals corresponding to the charge from each the photodetectors. The imaging system reads first digital pixel data from a first subset of the analog photodetector signals corresponding to the first group of photodetectors without reading second digital pixel data from a second subset of the analog photodetector signals corresponding to the second group of photodetectors. The imaging system generates an image of the scene using the first digital pixel data.

Abstract: Systems, methods, and non-transitory media are provided for predictive camera initialization. An example method can include obtain, from a first image capture device, image data depicting a scene; classify the scene based on the image data; based on the classification of the scene, predict a camera use event; and based on the predicted camera use event, adjust a power mode of at least one of the first image capture device and a second image capture device.

Abstract: Systems, methods, and computer-readable media are provided for low power, variable focus. An example method can include obtaining, based on a trigger, a first image of a scene captured by an image sensor, the first image being captured with a lens of the image sensor in a first configuration of a plurality of available lens configurations; determining, based on the first image and a first detection result, whether an object of interest is present in the first image; in response to determining that the object of interest is not present in the first image, adjusting the lens to a second configuration selected from the plurality of lens configurations; obtaining, by the image sensor, a second image of the scene while the lens is in the second configuration; and determining, based on the second image and a second detection result, that the object of interest is present in the second image.

Abstract: Systems, methods, and computer-readable media are provided for foreground image segmentation. In some examples, a method can include obtaining a first image of a target and a second image of the target, the first image having a first field-of-view (FOV) and the second image having a second FOV; determining, based on the first image, a first segmentation map that identifies a first estimated foreground region in the first image; determining, based on the second image, a second segmentation map that identifies a second estimated foreground region in the second image; generating a third segmentation map based on the first segmentation map and the second segmentation map; and generating, using the second segmentation map and the third segmentation map, a refined segmentation mask that identifies the target as a foreground region of the first and/or second image.

Abstract: A method performed by a deep neural network (DNN) includes receiving, at a layer of the DNN during an inference stage, a layer input comprising content associated with a DNN input received at the DNN. The method also includes quantizing one or more parameters of a plurality of parameters associated with the layer based on the content of the layer input. The method further includes performing a task corresponding to the DNN input, the task performed with the one or more one quantized parameters.

Abstract: Techniques and systems are provided for processing one or more images. For example, a process can obtain a plurality of images captured during a session. The plurality of images include a current image of a current face. The process can extract features of the current face from the current image. The process can compare the extracted features of the current face to extracted features of a previous face from a previous image of the plurality of images captured during the session. The process can determine, based on comparing the extracted features of the current face to the extracted features of the previous face, whether the current face from the current image matches the previous face from the previous image. The process can determine whether to lock access to a computing device based on whether the current face from the current image matches the previous face from the previous image.

Abstract: Systems, methods, and computer-readable media are provided for foreground image segmentation. In some examples, a method can include obtaining a first image of a target and a second image of the target, the first image having a first field-of-view (FOV) and the second image having a second FOV; determining, based on the first image, a first segmentation map that identifies a first estimated foreground region in the first image; determining, based on the second image, a second segmentation map that identifies a second estimated foreground region in the second image; generating a third segmentation map based on the first segmentation map and the second segmentation map; and generating, using the second segmentation map and the third segmentation map, a refined segmentation mask that identifies the target as a foreground region of the first and/or second image.

Abstract: An imaging system can obtain image data, for instance from an image sensor. The imaging system can supply the image data as input data to a machine learning system, which can generate one or more maps based on the image data. Each map can identify strengths at which a certain image processing function is to be applied to each pixel of the image data. Different maps can be generated for different image processing functions, such as noise reduction, sharpening, or color saturation. The imaging system can generate a modified image based on the image data and the one or more maps, for instance by applying each of one or more image processing functions in accordance with each of the one or more maps. The imaging system can supply the image data and the one or more maps to a second machine learning system to generate the modified image.

Abstract: Methods and apparatus for determining a depth of an object within a scene are provided. Image data of a scene can be captured using a lens configured to project an image of the scene onto an image sensor. The lens has a known focal length and is movable between at least a first lens position and a second lens position. A first image of the scene is captured with the lens at a first lens position, and a second image of the scene is captured with the lens at a second, different position. By measuring a first dimension of the object using the first image and a second dimension of the object using the second image, a depth of the object may be determined based upon a ratio of the first and second dimensions, the focal length of the lens, and a distance between the first and second lens positions.

Abstract: Devices and methods for providing seamless preview images for multi-camera devices having two or more asymmetric cameras. A multi-camera device may include two asymmetric cameras disposed to image a target scene. The multi-camera device further includes a processor coupled to a memory component and a display, the processor configured to retrieve an image generated by a first camera from the memory component, retrieve an image generated by a second camera from the memory component, receive input corresponding to a preview zoom level, retrieve spatial transform information and photometric transform information from memory, modify at least one image received from the first and second cameras by the spatial transform and the photometric transform, and provide on the display a preview image comprising at least a portion of the at least one modified image and a portion of either the first image or the second image based on the preview zoom level.