Abstract: Embodiments relate to two stage multi-scale processing of an image. A first stage processing circuitry generates an unscaled single color version of the image that undergoes noise reduction before generating a high frequency component of the unscaled single color version. A scaler generates a first downscaled version of the image comprising a plurality of color components. A second stage processing circuitry generates a plurality of sequentially downscaled images based on the first downscaled version. The second stage processing circuitry processes the first downscaled version and the downscaled images to generate a processed version of the first downscaled version. The unscaled single color high frequency component and the processed version of the first downscaled version of the image are merged to generate a processed version of the image.

Abstract: Embodiments relate to a pixel defect detection circuit for detecting and correcting defective pixels in captured image frames. The pixel defect detection circuit includes a defect pixel location table that maps pixel locations in an image frame to respective confidence values, each confidence value indicating a likelihood that a corresponding pixel is defective. The pixel defect detection circuit further includes a dynamic defect processing circuit configured to determine whether a first pixel of an image frame is defective, and a flatness detection circuit configured to determine whether the first pixel is in a flat region of the image frame. The confidence value corresponding to the location of the first pixel is updated based upon whether the first pixel is determined be defective if the first pixel is determined to be in a flat region, and not updated if the first pixel is determined to not be in a flat region.

Abstract: An image signal processor may include a pixel defect correction component that tracks defect history for frames captured by an image sensor and applies the history when identifying and correcting defective pixels in a frame. The component maintains a defect pixel location table that includes a defect confidence value for pixels of the image sensor. The component identifies defective pixels in a frame, for example by comparing each pixel's value to the values of its neighbor pixels. If a pixel is detected as defective, its defect confidence value may be incremented. Otherwise, the value may be decremented. If a pixel's defect confidence value is over a defect confidence threshold, the pixel is considered defective and thus may be corrected. If a pixel's defect confidence value is under the threshold, the pixel is considered not defective and thus may not be corrected even if the pixel was detected as defective.

Abstract: Embodiments relate a keypoint detection circuit for identifying keypoints in captured image frames. The keypoint detection circuit generates an image pyramid based upon a received image frame, and determine multiple sets of keypoints for each octave of the pyramid using different levels of blur. In some embodiments, the keypoint detection circuit includes multiple branches, each branch made up of one or more circuits for determining a different set of keypoints from the image, or for determining a subsampled image for a subsequent octave of the pyramid. By determining multiple sets of keypoints for each of a plurality of pyramid octaves, a larger, more varied set of keypoints can be obtained and used for object detection and matching between images.

Abstract: Embodiments relate to a pixel defect detection circuit for detecting and correcting defective pixels in captured image frames. The pixel defect detection circuit includes a defect pixel location table that maps pixel locations in an image frame to respective confidence values, each confidence value indicating a likelihood that a corresponding pixel is defective. The pixel defect detection circuit further includes a dynamic defect processing circuit configured to determine whether a first pixel of an image frame is defective, and a flatness detection circuit configured to determine whether the first pixel is in a flat region of the image frame. The confidence value corresponding to the location of the first pixel is updated based upon whether the first pixel is determined be defective if the first pixel is determined to be in a flat region, and not updated if the first pixel is determined to not be in a flat region.

Abstract: Embodiments relate to two stage multi-scale processing of an image. A first stage processing circuitry generates an unscaled single color version of the image that undergoes noise reduction before generating a high frequency component of the unscaled single color version. A scaler generates a first downscaled version of the image comprising a plurality of color components. A second stage processing circuitry generates a plurality of sequentially downscaled images based on the first downscaled version. The second stage processing circuitry processes the first downscaled version and the downscaled images to generate a processed version of the first downscaled version. The unscaled single color high frequency component and the processed version of the first downscaled version of the image are merged to generate a processed version of the image.

Abstract: Embodiments relate to a first demosaicing circuit and a second demosaicing circuit that can perform demosaicing of image data. The first demosaicing circuit processes received image data to generate a first demosaiced image for obtaining statistic information on the received image data. The second demosaicing circuit performs demosaicing of the received image data to generate a second demosaiced image. A processing circuit pipeline performs at least one of resampling, noise processing, color processing and output rescaling performed on the second demosaiced image based on the statistics information obtained from the first demosaiced image.

Abstract: Embodiments relate to a pixel defect detection circuit for detecting and correcting defective pixels in captured image frames. The pixel defect detection circuit includes a defect pixel location table that maps pixel locations in an image frame to respective confidence values, each confidence value indicating a likelihood that a corresponding pixel is defective. The pixel defect detection circuit further includes a dynamic defect processing circuit configured to determine whether a first pixel of an image frame is defective, and a flatness detection circuit configured to determine whether the first pixel is in a flat region of the image frame. The confidence value corresponding to the location of the first pixel is updated based upon whether the first pixel is determined be defective if the first pixel is determined to be in a flat region, and not updated if the first pixel is determined to not be in a flat region.

Abstract: An image signal processor may include a pixel defect correction component that tracks defect history for frames captured by an image sensor and applies the history when identifying and correcting defective pixels in a frame. The component maintains a defect pixel location table that includes a defect confidence value for pixels of the image sensor. The component identifies defective pixels in a frame, for example by comparing each pixel's value to the values of its neighbor pixels. If a pixel is detected as defective, its defect confidence value may be incremented. Otherwise, the value may be decremented. If a pixel's defect confidence value is over a defect confidence threshold, the pixel is considered defective and thus may be corrected. If a pixel's defect confidence value is under the threshold, the pixel is considered not defective and thus may not be corrected even if the pixel was detected as defective.

Abstract: Systems and methods for automatic lens flare compensation may include a non-uniformity detector configured to operate on pixel data for an image in an image sensor color pattern. The non-uniformity detector may detect a non-uniformity in the pixel data in a color channel of the image sensor color pattern. The non-uniformity detector may generate output including location and magnitude values of the non-uniformity. A lens flare detector may determine, based at least on the location and magnitude values, whether the output of the non-uniformity detector corresponds to a lens flare in the image. In some embodiments, the lens flare detector may generate, in response to determining that the output corresponds to the lens flare, a representative map of the lens flare. A lens flare corrector may determine one or more pixel data correction values corresponding to the lens flare and apply the pixel data correction values to the pixel data.

Abstract: A pedestal level for an image sensor can be dynamically adjusted based on one or more parameters. The parameters include one or more operating conditions associated with the image sensor, pre-determined image sensor characterization data, the number of unused digital codes, and/or the number of clipped pixel signals. The operating conditions can include the temperature of the image sensor, the gain of at least one amplifier included in processing circuitry operably connected to at least one pixel, and/or the length of the integration period for at least one pixel in the image sensor. Based on the one or more of the parameters, the pedestal level is adjusted to reduce a number of unused digital codes in a distribution of dark current. Additionally or alternatively, the variance of the pixel signals can be reduced to permit the use of a lower pedestal level.

Abstract: Embodiments relate to an architecture of a vision pipe included in an image signal processor. The architecture includes a front-end portion that includes a pair of image signal pipelines that generate an updated luminance image data. A back-end portion of the vision pipe architecture receives the updated luminance images from the front-end portion and performs, in parallel, scaling and various computer vision operations on the updated luminance image data. The back-end portion may repeatedly perform this parallel operation of computer vision operations on successively scaled luminance images to generate a pyramid image.

Abstract: Embodiments relate to color correction circuit operations performed by an image signal processor. The color correction circuit computes optimal color correction matrix on a per-pixel basis and adjusts it based on relative noise standard deviations of the color channels to steer the matrix.

Abstract: An image processing pipeline may process image data at multiple rates. A stream of raw pixel data collected from an image sensor for an image frame may be processed through one or more pipeline stages of an image signal processor. The stream of raw pixel data may then be converted into a full-color domain and scaled to a data size that is less than an initial data size for the image frame. The converted pixel data may be processed through one or more other pipelines stages and output for storage, further processing, or display. In some embodiments, a back-end interface may be implemented as part of the image signal processor via which image data collected from sources other than the image sensor may be received and processed through various pipeline stages at the image signal processor.

Abstract: Embodiments relate to an architecture of a vision pipe included in an image signal processor. The architecture includes a front-end portion that includes a pair of image signal pipelines that generate an updated luminance image data. A back-end portion of the vision pipe architecture receives the updated luminance images from the front-end portion and performs, in parallel, scaling and various computer vision operations on the updated luminance image data. The back-end portion may repeatedly perform this parallel operation of computer vision operations on successively scaled luminance images to generate a pyramid image.

Abstract: Embodiments relate to color correction circuit operations performed by an image signal processor. The color correction circuit computes optimal color correction matrix on a per-pixel basis and adjusts it based on relative noise standard deviations of the color channels to steer the matrix.

Abstract: Methods and systems for detecting keypoints in image data may include an image sensor interface receiving pixel data from an image sensor. A front-end pixel data processing circuit may receive pixel data and convert the pixel data to a different color space format. A back-end pixel data processing circuit may perform one or more operations on the pixel data. An output circuit may receive pixel data and output the pixel data to a system memory. A keypoint detection circuit may receive pixel data from the image sensor interface in the image sensor pixel data format or receive pixel data after processing by the front-end or the back-end pixel data processing circuits. The keypoint detection circuit may perform a keypoint detection operation on the pixel data to detect one or more keypoints in the image frame and output to the system memory a description of the one or more keypoints.

Abstract: Embodiments of the present disclosure generally relate to image signal processing logic, and in particular, to separating an undecimated image signal data to create two components with lower resolution and full-resolution, generating an interpolation guidance information based on the two components created by separation, forming a difference image data representing the difference between the chroma and luma values of each pixel and its neighboring pixels, and merging the processed image data from the processing pipelines with the unprocessed image data using the interpolation guidance information generated. The generation of the interpolation guidance information is based on determining distances between pixel values from a group comprising pixels from interpolation nodes, pixels diagonally located adjacent to the interpolation nodes, pixels horizontally adjacent to the interpolation nodes, and pixels vertically adjacent to the interpolation nodes.

Abstract: Methods and systems for detecting keypoints in image data may include an image sensor interface receiving pixel data from an image sensor. A front-end pixel data processing circuit may receive pixel data and convert the pixel data to a different color space format. A back-end pixel data processing circuit may perform one or more operations on the pixel data. An output circuit may receive pixel data and output the pixel data to a system memory. A keypoint detection circuit may receive pixel data from the image sensor interface in the image sensor pixel data format or receive pixel data after processing by the front-end or the back-end pixel data processing circuits. The keypoint detection circuit may perform a keypoint detection operation on the pixel data to detect one or more keypoints in the image frame and output to the system memory a description of the one or more keypoints.

Abstract: An input rescale module that performs cross-color correlated downscaling of sensor data in the horizontal and vertical dimensions. The module may perform a first-pass demosaic of sensor data, apply horizontal and vertical scalers to resample and downsize the data in the horizontal and vertical dimensions, and then remosaic the data to provide horizontally and vertically downscaled sensor data as output for additional image processing. The module may, for example, act as a front end scaler for an image signal processor (ISP). The demosaic performed by the module may be a relatively simple demosaic, for example a demosaic function that works on 3×3 blocks of pixels. The front end of module may receive and process sensor data at two pixels per clock (ppc); the horizontal filter component reduces the sensor data down to one ppc for downstream components of the input rescale module and for the ISP pipeline.