Abstract: Embodiments of the present disclosure relate to reducing line banding artifacts in raw image data. If the same line of pixel sensors (e.g., in a row) in an image sensor includes a subset of pixel sensors that receive bright light and another subset of pixel sensors that receive low light, line banding artifacts may appear in the capture raw image data. To reduce or eliminate the line banding artifacts, the raw image data is normalized by adding offset values to its pixel values. The offset values are determined from the pixel values obtained from masked pixel sensors on one or both sides of the image sensor.

Abstract: Embodiments relate to image signal processors (ISP) that include one or more auto-focus circuits. Each of the auto-focus circuits may be connected to an image sensor and may be separate from a statistics circuit and other image processing pipelines of the ISP. An image sensor may include one or more focus pixels that are used to generate data for auto-focusing. The auto-focus circuit may extract the focus pixel values and generate a signal to control the lens position of the image sensor. Each image sensor may include a separate auto-focus circuit. When other image processing pipelines of the ISP are processing the image data from one image sensor, the auto-focus circuit for another image sensor may continue to generate focus signals that control the lens position of the other image sensor. The other image sensor may be in standby but may continue to remain in focus.

Abstract: Embodiments relate to two stage multi-scale processing of an image. A first stage processing circuitry generates an unscaled single color version of the image that undergoes noise reduction before generating a high frequency component of the unscaled single color version. A scaler generates a first downscaled version of the image comprising a plurality of color components. A second stage processing circuitry generates a plurality of sequentially downscaled images based on the first downscaled version. The second stage processing circuitry processes the first downscaled version and the downscaled images to generate a processed version of the first downscaled version. The unscaled single color high frequency component and the processed version of the first downscaled version of the image are merged to generate a processed version of the image.

Abstract: Embodiments relate to a bilateral filter circuit for directional filtering of an image. The directional bilateral filter circuit determines an edge direction and a weight for the edge direction by processing differences between pixel values of pixels in a first block of pixels in the image. The bilateral filter circuit determines non-directional taps for pixels in a second block by processing pixel locations, and determines directional taps by processing differences between pixel values, gradient information for the second block and the edge direction. The bilateral filter circuit determines final filter taps for pixels in the second block by blending corresponding non-directional taps and directional taps using the weight. The bilateral filter circuit obtains a pixel value of a filtered image by multiplying the final filter taps to corresponding pixel values of the pixels in the second block and adding the multiplied values.

Abstract: Embodiments relate to directional bilateral filtering of a raw image. For each pixel in the image, a block of pixels surrounding that pixel is used for filtering. When the block of pixels in a Bayer pattern have directionality, directional filter coefficients are used instead of default filter coefficients. To obtain a directional tap, a directional filter coefficient is attenuated by an attenuation factor that differs based at least on the location of the pixels in the pixel block. The directional taps are blended with non-directional taps derived from the default filter coefficients using a weight representing confidence on the directionality. The filtered pixel values are then obtained by multiplying pixel values with corresponding taps.

Abstract: Embodiments relate to a pixel defect detection circuit for detecting and correcting defective pixels in captured image frames. The pixel defect detection circuit includes a defect pixel location table that maps pixel locations in an image frame to respective confidence values, each confidence value indicating a likelihood that a corresponding pixel is defective. The pixel defect detection circuit further includes a dynamic defect processing circuit configured to determine whether a first pixel of an image frame is defective, and a flatness detection circuit configured to determine whether the first pixel is in a flat region of the image frame. The confidence value corresponding to the location of the first pixel is updated based upon whether the first pixel is determined be defective if the first pixel is determined to be in a flat region, and not updated if the first pixel is determined to not be in a flat region.

Abstract: Methods and systems for detecting keypoints in image data may include an image sensor interface receiving pixel data from an image sensor. A front-end pixel data processing circuit may receive pixel data and convert the pixel data to a different color space format. A back-end pixel data processing circuit may perform one or more operations on the pixel data. An output circuit may receive pixel data and output the pixel data to a system memory. A keypoint detection circuit may receive pixel data from the image sensor interface in the image sensor pixel data format or receive pixel data after processing by the front-end or the back-end pixel data processing circuits. The keypoint detection circuit may perform a keypoint detection operation on the pixel data to detect one or more keypoints in the image frame and output to the system memory a description of the one or more keypoints.

Abstract: Embodiments relate to a bilateral filter circuit for directional filtering of an image. The directional bilateral filter circuit determines an edge direction and a weight for the edge direction by processing differences between pixel values of pixels in a first block of pixels in the image. The bilateral filter circuit determines non-directional taps for pixels in a second block by processing pixel locations, and determines directional taps by processing differences between pixel values, gradient information for the second block and the edge direction. The bilateral filter circuit determines final filter taps for pixels in the second block by blending corresponding non-directional taps and directional taps using the weight. The bilateral filter circuit obtains a pixel value of a filtered image by multiplying the final filter taps to corresponding pixel values of the pixels in the second block and adding the multiplied values.

Abstract: Embodiments relate to circuitry for pixel conversion of images for display. A circuit converts input pixel values of an image using a color conversion function. A lookup table memory circuit stores a mapping of color converted values and input pixel values where the mapping represents the color conversion function. The circuit produces a color converted value from the lookup table as a color converted version of a first input pixel value responsive to the first input pixel value being within a first range. The circuit may also produce a color converted version of a second input pixel value by interpolating a subset of the color converted values received from the lookup table responsive to the second input pixel being within a second input range.

Abstract: An image signal processor may include a pixel defect correction component that tracks defect history for frames captured by an image sensor and applies the history when identifying and correcting defective pixels in a frame. The component maintains a defect pixel location table that includes a defect confidence value for pixels of the image sensor. The component identifies defective pixels in a frame, for example by comparing each pixel's value to the values of its neighbor pixels. If a pixel is detected as defective, its defect confidence value may be incremented. Otherwise, the value may be decremented. If a pixel's defect confidence value is over a defect confidence threshold, the pixel is considered defective and thus may be corrected. If a pixel's defect confidence value is under the threshold, the pixel is considered not defective and thus may not be corrected even if the pixel was detected as defective.

Abstract: Embodiments relate to circuitry for performing fusion of two images captured with two different exposure times to generate a fused image having a higher dynamic range. Information about first keypoints is extracted from the first image by processing pixel values of pixels in the first image. A model describing correspondence between the first image and the second image is then built by processing at least the information about first keypoints. A processed version of the first image is warped using mapping information in the model to generate a warped version of the first image spatially more aligned to the second image than to the first image. The warped version of the first image is fused with a processed version of the second image to generate the fused image.

Abstract: Embodiments relate to a bilateral filter circuit for directional filtering of an image. The directional bilateral filter circuit determines an edge direction and a weight for the edge direction by processing differences between pixel values of pixels in a first block of pixels in the image. The bilateral filter circuit determines non-directional taps for pixels in a second block by processing pixel locations, and determines directional taps by processing differences between pixel values, gradient information for the second block and the edge direction. The bilateral filter circuit determines final filter taps for pixels in the second block by blending corresponding non-directional taps and directional taps using the weight. The bilateral filter circuit obtains a pixel value of a filtered image by multiplying the final filter taps to corresponding pixel values of the pixels in the second block and adding the multiplied values.

Abstract: Embodiments of the present disclosure relate to reducing line banding artifacts in raw image data. If the same line of pixel sensors (e.g., in a row) in an image sensor includes a subset of pixel sensors that receive bright light and another subset of pixel sensors that receive low light, line banding artifacts may appear in the capture raw image data. To reduce or eliminate the line banding artifacts, the raw image data is normalized by adding offset values to its pixel values. The offset values are determined from the pixel values obtained from masked pixel sensors on one or both sides of the image sensor.

Abstract: Embodiments relate to image signal processors (ISP) that include one or more auto-focus circuits. Each of the auto-focus circuits may be connected to an image sensor and may be separate from a statistics circuit and other image processing pipelines of the ISP. An image sensor may include one or more focus pixels that are used to generate data for auto-focusing. The auto-focus circuit may extract the focus pixel values and generate a signal to control the lens position of the image sensor. Each image sensor may include a separate auto-focus circuit. When other image processing pipelines of the ISP are processing the image data from one image sensor, the auto-focus circuit for another image sensor may continue to generate focus signals that control the lens position of the other image sensor. The other image sensor may be in standby but may continue to remain in focus.

Abstract: Embodiments relate to circuitry for pixel conversion of images for display. A circuit converts input pixel values of an image using a color conversion function. A lookup table memory circuit stores a mapping of color converted values and input pixel values where the mapping represents the color conversion function. The circuit produces a color converted value from the lookup table as a color converted version of a first input pixel value responsive to the first input pixel value being within a first range. The circuit may also produce a color converted version of a second input pixel value by interpolating a subset of the color converted values received from the lookup table responsive to the second input pixel being within a second input range.

Abstract: Embodiments relate to circuitry for performing fusion of two images captured with two different exposure times to generate a fused image having a higher dynamic range. Information about first keypoints is extracted from the first image by processing pixel values of pixels in the first image. A model describing correspondence between the first image and the second image is then built by processing at least the information about first keypoints. A processed version of the first image is warped using mapping information in the model to generate a warped version of the first image spatially more aligned to the second image than to the first image. The warped version of the first image is fused with a processed version of the second image to generate the fused image.

Abstract: Embodiments relate to a bilateral filter circuit for directional filtering of an image. The directional bilateral filter circuit determines an edge direction and a weight for the edge direction by processing differences between pixel values of pixels in a first block of pixels in the image. The bilateral filter circuit determines non-directional taps for pixels in a second block by processing pixel locations, and determines directional taps by processing differences between pixel values, gradient information for the second block and the edge direction. The bilateral filter circuit determines final filter taps for pixels in the second block by blending corresponding non-directional taps and directional taps using the weight. The bilateral filter circuit obtains a pixel value of a filtered image by multiplying the final filter taps to corresponding pixel values of the pixels in the second block and adding the multiplied values.

Abstract: Embodiments relate to directional bilateral filtering of a raw image. For each pixel in the image, a block of pixels surrounding that pixel is used for filtering. When the block of pixels in a Bayer pattern have directionality, directional filter coefficients are used instead of default filter coefficients. To obtain a directional tap, a directional filter coefficient is attenuated by an attenuation factor that differs based at least on the location of the pixels in the pixel block. The directional taps are blended with non-directional taps derived from the default filter coefficients using a weight representing confidence on the directionality. The filtered pixel values are then obtained by multiplying pixel values with corresponding taps.

Abstract: Embodiments relate a keypoint detection circuit for identifying keypoints in captured image frames. The keypoint detection circuit generates an image pyramid based upon a received image frame, and determine multiple sets of keypoints for each octave of the pyramid using different levels of blur. In some embodiments, the keypoint detection circuit includes multiple branches, each branch made up of one or more circuits for determining a different set of keypoints from the image, or for determining a subsampled image for a subsequent octave of the pyramid. By determining multiple sets of keypoints for each of a plurality of pyramid octaves, a larger, more varied set of keypoints can be obtained and used for object detection and matching between images.

Abstract: Embodiments relate to two stage multi-scale processing of an image. A first stage processing circuitry generates an unscaled single color version of the image that undergoes noise reduction before generating a high frequency component of the unscaled single color version. A scaler generates a first downscaled version of the image comprising a plurality of color components. A second stage processing circuitry generates a plurality of sequentially downscaled images based on the first downscaled version. The second stage processing circuitry processes the first downscaled version and the downscaled images to generate a processed version of the first downscaled version. The unscaled single color high frequency component and the processed version of the first downscaled version of the image are merged to generate a processed version of the image.