Abstract: A method includes obtaining, using at least one processing device of an electronic device, multiple image frames of a scene, where each image frame includes a luma channel and chroma channels. The method also includes blending, using the at least one processing device, the image frames to generate a blended image. Blending the image frames includes determining first weights to be applied when blending the luma channels of the image frames and separate second weights to be applied when blending the chroma channels of the image frames; generating a fused luma image using the luma channels of the image frames and the first weights and separately generating at least one fused chroma image using the chroma channels of the image frames and the second weights; and combining the fused luma image and the at least one fused chroma image.

Abstract: A method includes obtaining, using at least one processing device of an electronic device, a sequence of images of a scene. The method also includes performing, using the at least one processing device, instance segmentation of the sequence of images to generate a sequence of segmented images. Performing the instance segmentation includes (i) segmenting each of the images into semantic classes and (ii) separately identifying two or more objects in the scene associated with a common semantic class, where different semantic classes are associated with different types of image contents. The method further includes processing, using the at least one processing device, the sequence of segmented images to generate a final image of the scene. The sequence of segmented images is processed using one or more image processing operations determined using a natural language processing (NLP) engine.

Abstract: A method for training data generation includes obtaining a first set of image frames of a scene and a second set of image frames of the scene using multiple exposure settings. The method also includes generating an alignment map, a blending map, and an input image using the first set of image frames. The method further includes generating a ground truth image using the alignment map, the blending map, and the second set of image frames. In addition, the method includes using the ground truth image and the input image as an image pair in a training dataset when training a machine learning model to reduce image distortion and noise.

Abstract: An electronic device is provided. The electronic device includes a display, a camera module disposed under the display, and a processor electrically connected to the display and the camera module. The processor is configured to acquire a sample frame by using the camera module, identify whether a light source object is included in the sample frame, determine an imaging parameter for acquisition of first multiple frames when the light source object is identified to be included in the sample frame, acquire multiple frames, based on the imaging parameter, composite the multiple frames to generate a composite frame, identify an attribute of the light source object included in the composite frame, and perform frame correction of the composite frame, based on the identified attribute.

Abstract: A method includes obtaining, using at least one processing device of an electronic device, multiple sets of training image frames, where each set of training image frames has an associated ground truth image. The method also includes applying, using the at least one processing device, motion blur and warping to the multiple sets of training image frames in order to generate additional sets of training image frames. In addition, the method includes training, using the at least one processing device, a machine learning model to align image frames and remove motion blur from the image frames based on at least the additional sets of training image frames and the ground truth images.

Abstract: A method includes of fixed pattern noise (FPN) reduction includes capturing, at runtime by an image sensor of an electronic device, noisy calibration video frames, and for each noisy calibration video frame: denoising, by a processor of the electronic device, the noisy calibration video frame to obtain a denoising residue frame, and determining, by the processor, whether the denoising residue frame is suitable for aggregation based on motion of the noisy calibration video frame relative to a previously captured noisy calibration video frame. The method further includes calibrating, by the processor, an FPN map for the image sensor based on an aggregation of the denoising residue frames that are determined to be suitable for aggregation, after the calibrating, capturing, by the image sensor, noisy target video frames, and generating, by the processor, clean target video frames from the noisy target video frames based on the calibrated FPN map.

Abstract: A method includes obtaining at least one input image and processing the at least one input image using an encoder model. The method also includes generating an output based on the processing of the at least one input image using the encoder model. The method also includes providing the output to a diffusion model and generating, using the diffusion model and based on the output, at least one image in a latent space. The method also includes outputting a final image result based on the at least one image in the latent space.

Abstract: A method includes training, using at least one processing device of an electronic device, a student variational autoencoder (VAE) based on a teacher VAE. Training the student VAE based on the teacher VAE includes using a first optimizer and a second optimizer. The first optimizer is configured to align a latent space of the student VAE with a latent space of the teacher VAE. The second optimizer is configured to optimize reconstruction performance of the student VAE. During use of the first optimizer, parameters of the teacher VAE may be frozen, and parameters of an encoder of the student VAE may be adjusted. During use of the second optimizer, the parameters of the encoder of the student VAE may be frozen, and parameters of a decoder of the student VAE may be adjusted.

Abstract: A method includes obtaining, using at least one under display camera, one or more first image frames associated with a first diffraction pattern and one or more second image frames associated with a second diffraction pattern. The first diffraction pattern and the second diffraction pattern are related through a transformation. The method also includes generating a first deblurred image using the one or more first image frames and a second deblurred image using the one or more second image frames. The method further includes combining the first and second deblurred images while exploiting complementary types of image artifacts created by the first and second diffraction patterns to generate an image of a scene.

Abstract: A method includes obtaining, using at least one processing device of an electronic device, a video having a sequence of image frames. During each of multiple iterations, the method also includes identifying, using the at least one processing device, features of a specified image frame in the sequence and features of a denoised version of a preceding image frame. During each of the multiple iterations, the method further includes aligning, using the at least one processing device, the features of the specified image frame in the sequence and the features of the denoised version of the preceding image frame to generate aligned features. In addition, during each of the multiple iterations, the method includes generating, using the at least one processing device, a denoised version of the specified image frame based on the aligned features.

Abstract: A method of constructing a dictionary of kernels for blending in multi-frame processing (MFP) includes generating, for each ground truth (GT) image in a set of GT images represented in a full color space, a corresponding feature matrix by generating a set of synthetic raw images represented in a color filter mosaic space, performing demosaicing and registration operations on each of the synthetic raw images to generate a corresponding registered image, choosing a set of patches in each registered image, the location of each patch in one registered image coinciding with the location of a corresponding patch in each other registered image, generating, for each patch, a corresponding pixel value matrix containing pixel values of each color channel of each pixel included in the corresponding patch, and extracting features from the pixel value matrices and generating the feature matrix corresponding to the GT image based on the extracted features.

Abstract: A method includes obtaining, using a multi-mode image sensor, first image frames and second image frames. The first image frames include image data having a first bit depth, and the second image frames include image data having a second bit depth smaller than the first bit depth. The first image frames are captured using two or more exposure times, and the second image frames are captured using at least one exposure time shorter than the two or more exposure times. At least one of the second image frames is captured in between each consecutive pair of the first image frames. The method also includes providing at least the first image frames to a multi-frame processing (MFP) pipeline and processing at least the first image frames using the MFP pipeline to generate an output image.

Abstract: A method includes obtaining a reference frame from among multiple image frames of a scene. The method also includes generating a segmentation mask using the reference frame, where the segmentation mask contains information for separation of foreground and background in the scene. The method further includes applying the segmentation mask to each of the multiple image frames to generate foreground image frames and background image frames. The method also includes performing multi-frame registration on each of the foreground image frames to generate registered foreground image frames. The method further includes performing multi-frame registration on each of the background image frames to generate registered background image frames. In addition, the method includes combining the registered foreground image frames and the registered background image frames to generate a combined registered multi-frame image of the scene.

Abstract: An electronic device includes at least one imaging sensor configured to obtain an input set of images of a scene. The electronic device also includes at least one processing device configured to generate a differentiable 3D model of the scene based on an iterative process using the input set of images and project the differentiable 3D model into an image space to generate an estimated burst of selfie images. To obtain the input set of images, the at least one processing device may be configured to obtain an initial burst of images, generate a map indicating subjects in the scene based on the initial burst of images, provide a prompt for a user to move the electronic device to capture at least one additional burst of images, and obtain the input set of images based on the initial burst of images and the at least one additional burst of images.

Abstract: A method includes obtaining a ground truth image and generating multiple image frames using the ground truth image, a modeled optical blur, and a modeled global motion. The method also includes generating multiple mosaic image frames using the image frames and a color filter array and generating multiple raw input image frames using the mosaic image frames and a noise model associated with at least one imaging sensor. The method further includes providing the raw input image frames to a multi-frame processing pipeline in order to generate synthetic training data. In addition, the method includes training a machine learning-based image processing engine using the ground truth image and the synthetic training data.

Abstract: To employ low resolution, noisy tone mapping operations for high resolution images, at least one raw image frame is converted to a first image at a higher resolution and a second image at a lower resolution. Tone mapping is applied to the second image to derive a third image at the lower resolution. Histogram matching and regularization are performed to determine a lookup table approximating histogram matching of the second image to the third image. A global gain map is derived based on luma for the first image and the lookup table. A local gain map is derived by up-sampling and denoising residual differences between the third image and the lookup table applied to the second image. Based on the global gain map and the local gain map, a total gain map is determined for tone mapping the first image to produce a fourth image at the first resolution.

Abstract: A method includes obtaining multiple images via a multi-frame capture operation. The method also includes processing the multiple images using a multi-frame processing pipeline to generate a single frame image. The method also includes processing the multiple images to generate a backlit segmentation map that identifies regions in a scene of the single frame image that are illuminated by at least one background light source. The method also includes generating a dehazed single frame image based on modifying a local contrast of the single frame image using the backlit segmentation map.

Abstract: A method for operating an electronic device includes capturing, by an image sensor module, a stream from a pixel array. The method also includes processing, by the image sensor module, the stream to generate a preview stream and a full frame stream. The method further includes compressing, by the image sensor module, the preview stream using a first compression and the full frame stream using a second compression. In addition, the method includes outputting, by the image sensor module, the compressed preview stream and the compressed full frame stream.

Abstract: A method includes obtaining an under-display camera (UDC) image captured using a camera located under a display. The method also includes processing, using at least one processing device of an electronic device, the UDC image based on a machine learning model to restore the UDC image. The method further includes displaying or storing the restored image corresponding to the UDC image. The machine learning model is trained using (i) a ground truth image and (ii) a synthetic image generated using the ground truth image and a point spread function that is based on an optical transmission model of the display.

Abstract: A method includes identifying, using at least one processing device of an electronic device, a spatially-variant point spread function associated with an under-display camera. The spatially-variant point spread function is based on an optical transmission model and a layout of a display associated with the under-display camera. The method also includes generating, using the at least one processing device, a ground truth image. The method further includes performing, using the at least one processing device, a convolution of the ground truth image based on the spatially-variant point spread function in order to generate a synthetic sensor image. The synthetic sensor image represents a simulated image captured by the under-display camera. In addition, the method includes providing, using the at least one processing device, the synthetic sensor image and the ground truth image as an image pair to train a machine learning model to perform under-display camera point spread function inversion.