Abstract: In one implementation, a method of performing perspective correction is performed by a device including an image sensor, a display, one or more processors, and a non-transitory memory. The method includes: capturing, using the image sensor, an image of a physical environment; obtaining a first depth map including a plurality of depths respectively associated with a plurality of pixels of the image of the physical environment; generating a second depth map by aligning one or more portions of the first depth map based on a control signal associated with the image of the physical environment; transforming, using the one or more processors, the image of the physical environment based on the second depth map; and displaying, via the display, the transformed image.

Abstract: An adaptive approach to image bracket determination and a more memory-efficient approach to image fusion, which are designed to generate low noise and high dynamic range (HDR) images in a wide variety of capturing conditions, are described. An incoming preview image stream may be obtained from an image capture device. When a capture request is received, an analysis may be performed on an image from the preview image stream that has a predetermined temporal relationship to the image capture request. Based on the analysis, a set of images (and their respective capture parameters, e.g., exposure time) may be determined for the image capture device to capture. As the determined set of images are captured, they may be registered and fused in a memory-efficient manner that, e.g., places an upper limit on the overall memory footprint of the registration and fusion operations—regardless of how many images are captured in total.

Abstract: Devices, methods, and non-transitory program storage devices (NPSDs) for an improved, so-called “hybrid” image registration process are disclosed herein, comprising: obtaining a first set of captured images, wherein the first set of captured images comprises a reference image and one or more bracketed images; and for each of the one or more bracketed images: performing a first (e.g., global) registration operation and a second (e.g., dense, or other localized) registration operation on the bracketed image with respect to the reference image, wherein each of the first and second registration operations produces an output; generating a blend map for the bracketed image, wherein each value in the blend map indicates whether to use the first or second registration operation output for a corresponding one or more pixels when registering the bracketed image with the reference image; and registering the bracketed image with the reference image, according to the generated blend map.

Abstract: In one embodiment, a method includes obtaining an image comprising a plurality of pixels, determining, for a particular pixel of the plurality of pixels, a gradient value, classifying, based on the gradient value, the particular pixel into a flat class or one of a plurality of edge classes, and denoising the particular pixel based on the classification.

Abstract: Devices, methods, and computer-readable media are disclosed, describing an adaptive, subject-aware approach for image bracket selection and fusion, e.g., to generate high quality images in a wide variety of capturing conditions, including low light conditions. An incoming image stream may be obtained from an image capture device, comprising images captured using differing default exposure values, e.g., according to a predetermined pattern. When a capture request is received, it may be detected whether one or more human or animal subjects are present in the incoming image stream. If a subject is detected, an exposure time of one or more images selected from the incoming image stream may be reduced relative to its default exposure time. Prior to the fusion operation, one of the selected images may be designated a reference image for the fusion operation based, at least in part, on a sharpness score and/or a blink score of the image.

Abstract: Devices, methods, and computer-readable media describing an adaptive approach for image selection, fusion, and noise reduction, e.g., to generate low noise and high dynamic range (HDR) images with improved motion freezing in a variety of capturing conditions. An incoming image stream may be obtained from an image capture device, wherein the image stream comprises a variety of differently-exposed captures, e.g., EV0 images, EV? images, EV+ images. When a capture request is received, a set of rules may be used to evaluate one or more capture conditions associated with the images from the incoming image stream and determine which two or more images to select for a fusion operation. The fusion operation may be designed to adaptively fuse the selected images, e.g., in a fashion that is determined to be optimal from a noise variance minimization standpoint. A fusion-adaptive noise reduction process may further be performed on the resultant fused image.

Abstract: An adaptive approach to image bracket determination and a more memory-efficient approach to image fusion, which are designed to generate low noise and high dynamic range (HDR) images in a wide variety of capturing conditions, are described. An incoming preview image stream may be obtained from an image capture device. When a capture request is received, an analysis may be performed on an image from the preview image stream that has a predetermined temporal relationship to the image capture request. Based on the analysis, a set of images (and their respective capture parameters, e.g., exposure time) may be determined for the image capture device to capture. As the determined set of images are captured, they may be registered and fused in a memory-efficient manner that, e.g., places an upper limit on the overall memory footprint of the registration and fusion operations—regardless of how many images are captured in total.

Abstract: Devices, methods, and computer-readable media describing an adaptive approach for image selection, fusion, and noise reduction, e.g., to generate low noise and high dynamic range (HDR) images with improved motion freezing in a variety of capturing conditions. An incoming image stream may be obtained from an image capture device, wherein the image stream comprises a variety of differently-exposed captures, e.g., EV0 images, EV? images, EV+ images. When a capture request is received, a set of rules may be used to evaluate one or more capture conditions associated with the images from the incoming image stream and determine which two or more images to select for a fusion operation. The fusion operation may be designed to adaptively fuse the selected images, e.g., in a fashion that is determined to be optimal from a noise variance minimization standpoint. A fusion-adaptive noise reduction process may further be performed on the resultant fused image.

Abstract: Techniques for de-noising a digital image using a multi-band noise filter and a unique combination of texture and chroma metrics are described. A novel texture metric may be used during multi-band filter operations on an image's luma channel to determine if a given pixel is associated with a textured/smooth region of the image. A novel chroma metric may be used during the same multi-band filter operation to determine if the same pixel is associated with a blue/not-blue region of the image. Pixels identified as being associated with a smooth blue region may be aggressively de-noised and conservatively sharpened. Pixels identified as being associated with a textured blue region may be conservatively de-noised and aggressively sharpened. By coupling texture and chroma constraints it has been shown possible to mitigate noise in an image's smooth blue regions without affecting the edges/texture in other blue objects.

Abstract: Systems, methods, and computer readable media to fuse digital images are described. In general, techniques are disclosed that use multi-band noise reduction techniques to represent input and reference images as pyramids. Once decomposed in this manner, images may be fused using novel low-level (noise dependent) similarity measures. In some implementations similarity measures may be based on intra-level comparisons between reference and input images. In other implementations, similarity measures may be based on inter-level comparisons. In still other implementations, mid-level semantic features such as black-level may be used to inform the similarity measure. In yet other implementations, high-level semantic features such as color or a specified type of region (e.g., moving, stationary, or having a face or other specified shape) may be used to inform the similarity measure.

Abstract: Techniques for de-noising a digital image using a multi-band noise filter and a unique combination of texture and chroma metrics are described. A novel texture metric may be used during multi-band filter operations on an image's luma channel to determine if a given pixel is associated with a textured/smooth region of the image. A novel chroma metric may be used during the the same multi-band filter operation to determine if the same pixel is associated with a blue/not-blue region of the image. Pixels identified as being associated with a smooth blue region may be aggressively de-noised and conservatively sharpened. Pixels identified as being associated with a textured blue region may be conservatively de-noised and aggressively sharpened. By coupling texture and chroma constraints it has been shown possible to mitigate noise in an image's smooth blue regions without affecting the edges/texture in other blue objects.

Abstract: Systems, methods, and computer readable media to fuse digital images are described. In general, techniques are disclosed that use multi-band noise reduction techniques to represent input and reference images as pyramids. Once decomposed in this manner, images may be fused using novel low-level (noise dependent) similarity measures. In some implementations similarity measures may be based on intra-level comparisons between reference and input images. In other implementations, similarity measures may be based on inter-level comparisons. In still other implementations, mid-level semantic features such as black-level may be used to inform the similarity measure. In yet other implementations, high-level semantic features such as color or a specified type of region (e.g., moving, stationary, or having a face or other specified shape) may be used to inform the similarity measure.

Abstract: Method and system for detecting objects of interest in a camera monitored area are disclosed. Statistical analysis of block feature data, particularly Sobel edge and spatial high frequency responses is used to model the background of the scene and to segregate foreground objects from the background. This technique provides a robust motion detection scheme prone to catching genuine motions and immune against false alarms.

Abstract: Method and system for detecting objects of interest in a camera monitored area are disclosed. Statistical analysis of block feature data, particularly Sobel edge and spatial high frequency responses is used to model the background of the scene and to segregate foreground objects from the background. This technique provides a robust motion detection scheme prone to catching genuine motions and immune against false alarms.

Abstract: Various embodiments of the present invention relate to systems, devices and method of video encoding that select a quantization parameter set based on a global edge strength value and an available bitrate for a corresponding compressed frame. Quantization parameters are selected using a mathematical correlation between the global edge strength value of the I-frame and an available bitrate/maxim target frame size for the corresponding compressed frame.

Abstract: Various embodiments of the present invention relate to systems, devices and method of video encoding that select a quantization parameter set based on a global edge strength value and an available bitrate for a corresponding compressed frame. Quantization parameters are selected using a mathematical correlation between the global edge strength value of the I-frame and an available bitrate/maxim target frame size for the corresponding compressed frame.