Abstract: Embodiments relate to identifying locations in an image where a green ghost is likely to occur by processing a raw version of the image. A block of pixels in the raw image is extracted and then each pixel value in the block is compared with a threshold value of a corresponding color channel. The compared results are then processed to generate bright light information for the block, indicating whether the green ghost is likely to be present at a location of the image diagonally opposed to the location of the block in the image relative to an optical center of an image sensor. By processing the raw image without subsequent processing prone to introducing artifacts and creating false positive, a likely location of the green orb or green ghost may be more accurately identified.

Abstract: An image processing circuit for multi-illumination white balance with thumbnail processing. The image processing circuit determines a set of initial weights for a source pixel in a thumbnail image by determining component values for multiple color channels of the source pixel. The image processing circuit determines a set of weights for the source pixel in a weight map for the thumbnail image. Each weight in the set of weights is determined based on corresponding initial weights from the set of initial weights. Each weight in the set of weights represents an intensity level of a respective chrominance class of multiple chrominance classes for the source pixel. The image processing circuit applies the set of weights to values of the color channels of the source pixel to generate color component values of the color channels of a target pixel in a target thumbnail image.

Abstract: An image processing circuit for multi-illumination white balance with thumbnail processing. The image processing circuit determines a set of initial weights for a source pixel in a thumbnail image by determining component values for multiple color channels of the source pixel. The image processing circuit determines a set of weights for the source pixel in a weight map for the thumbnail image. Each weight in the set of weights is determined based on corresponding initial weights from the set of initial weights. Each weight in the set of weights represents an intensity level of a respective chrominance class of multiple chrominance classes for the source pixel. The image processing circuit applies the set of weights to values of the color channels of the source pixel to generate color component values of the color channels of a target pixel in a target thumbnail image.

Abstract: An image processing circuit for performing local tone mapping (LTM) after image warping. The image processing circuit includes a warping circuit that warps an input image to generate a warped image, and a LTM circuit coupled to the warping circuit. The LTM circuit determines an input color component value for a color component of a pixel in a version of the warped image, determines an output color component value for the color component of the pixel, based on mapping of coordinates of pixels in the warped image to coordinates of pixels in the input image, determines a gain value for the pixel as a ratio of the output color component value relative to the input color component value, and adjusts color component values for color components of the pixel using the gain value to generate adjusted color component values for the color components of the pixel in an output image.

Abstract: Embodiments relate to identifying locations in an image where a green ghost is likely to occur by processing a raw version of the image. A block of pixels in the raw image is extracted and then each pixel value in the block is compared with a threshold value of a corresponding color channel. The compared results are then processed to generate bright light information for the block, indicating whether the green ghost is likely to be present at a location of the image diagonally opposed to the location of the block in the image relative to an optical center of an image sensor. By processing the raw image without subsequent processing prone to introducing artifacts and creating false positive, a likely location of the green orb or green ghost may be more accurately identified.

Abstract: Embodiments relate to extracting features from images, such as by identifying keypoints and generating keypoint descriptors of the keypoints. An apparatus includes a pyramid image generator circuit, a keypoint descriptor generator circuit, and a pyramid image buffer. The pyramid image generator circuit generates an image pyramid from an input image. The keypoint descriptor generator circuit processes the pyramid images for keypoint descriptor generation. The pyramid image buffer stores different portions of the pyramid images generated by the pyramid image generator circuit at different times and provides the stored portions of the pyramid images to the keypoint descriptor generator circuit for keypoint descriptor generation. When first portions of the pyramid images are no longer needed for the keypoint descriptor generation, the first portions are removed from the pyramid image buffer to provide space for second portions of the pyramid images that are needed for the keypoint descriptor generation.

Abstract: Embodiments relate to extracting features from images, such as by identifying keypoints and generating keypoint descriptors of the keypoints. An apparatus includes a pyramid image generator circuit, a keypoint descriptor generator circuit, and a pyramid image buffer. The pyramid image generator circuit generates an image pyramid from an input image. The keypoint descriptor generator circuit processes the pyramid images for keypoint descriptor generation. The pyramid image buffer stores different portions of the pyramid images generated by the pyramid image generator circuit at different times and provides the stored portions of the pyramid images to the keypoint descriptor generator circuit for keypoint descriptor generation. When first portions of the pyramid images are no longer needed for the keypoint descriptor generation, the first portions are removed from the pyramid image buffer to provide space for second portions of the pyramid images that are needed for the keypoint descriptor generation.

Abstract: Embodiments relate to generating keypoint descriptors of the keypoints using a sub-scale refinement and a sample pattern radius adjustment. An apparatus includes a sub-pixel refiner circuit and a keypoint descriptor generator circuit. The sub-pixel refiner circuit determines a keypoint scale value for a scale dimension of a keypoint in an image pyramid by performing an interpolation of response map (RM) pixel values of a pixel block of RM images defined around the keypoint. The keypoint descriptor generator circuit determines sample scales of the image pyramid based on the keypoint scale value and determines a radius value for each sample scale based on the keypoint scale value. The keypoint descriptor generator circuit samples patches of pixel values at the sample scales using the radius value for each sample scale to generate a keypoint descriptor of the keypoint.

Abstract: Embodiments relate to generating an image pyramid for feature extraction. A pyramid image generator circuit includes a first image buffer that stores image data at a first octave, a first blur filter circuit, a first spatial filter circuit, and a first decimator circuit. The first blur filter circuit generates a first pyramid image for a first scale of the first octave by applying a first amount of smoothing to the first image data stored in the first image buffer. The first spatial filter circuit and the first decimator generate second image data of a second octave that is higher than the first octave by applying a smoothing and a decimation to the first image data stored in the first image buffer. The first spatial filter circuit begins processing the first image data before the first blur filter circuit begins to process the first image data.

Abstract: Embodiments relate to generating keypoint descriptors of the keypoints using a sub-scale refinement and a sample pattern radius adjustment. An apparatus includes a sub-pixel refiner circuit and a keypoint descriptor generator circuit. The sub-pixel refiner circuit determines a keypoint scale value for a scale dimension of a keypoint in an image pyramid by performing an interpolation of response map (RM) pixel values of a pixel block of RM images defined around the keypoint. The keypoint descriptor generator circuit determines sample scales of the image pyramid based on the keypoint scale value and determines a radius value for each sample scale based on the keypoint scale value. The keypoint descriptor generator circuit samples patches of pixel values at the sample scales using the radius value for each sample scale to generate a keypoint descriptor of the keypoint.

Abstract: Embodiments relate to generating an image pyramid for feature extraction. A pyramid image generator circuit includes a first image buffer that stores image data at a first octave, a first blur filter circuit, a first spatial filter circuit, and a first decimator circuit. The first blur filter circuit generates a first pyramid image for a first scale of the first octave by applying a first amount of smoothing to the first image data stored in the first image buffer. The first spatial filter circuit and the first decimator generate second image data of a second octave that is higher than the first octave by applying a smoothing and a decimation to the first image data stored in the first image buffer. The first spatial filter circuit begins processing the first image data before the first blur filter circuit begins to process the first image data.

Abstract: Embodiments relate to extracting features from images, such as by identifying keypoints and generating keypoint descriptors of the keypoints. An apparatus includes a pyramid image generator circuit, a keypoint descriptor generator circuit, and a pyramid image buffer. The pyramid image generator circuit generates an image pyramid from an input image. The keypoint descriptor generator circuit processes the pyramid images for keypoint descriptor generation. The pyramid image buffer stores different portions of the pyramid images generated by the pyramid image generator circuit at different times and provides the stored portions of the pyramid images to the keypoint descriptor generator circuit for keypoint descriptor generation. When first portions of the pyramid images are no longer needed for the keypoint descriptor generation, the first portions are removed from the pyramid image buffer to provide space for second portions of the pyramid images that are needed for the keypoint descriptor generation.

Abstract: Systems and methods for improving operational efficiency of a video encoding system used to encode image data are provided. In embodiments, a video encoding system includes image processing circuitry configured to receive source image data and derive full-resolution image data and low-resolution image data from the source image data. The video encoding system also includes a low resolution pipeline configured to receive the low-resolution image data and determine one or more low resolution inter prediction modes based on the low-resolution image data. Furthermore, the video encoding system includes a main pipeline configured to encode the full-resolution image data based on the one or more low resolution inter prediction modes.

Abstract: Systems and methods for improving operational efficiency of a video encoding system used to encode image data are provided. In embodiments, a video encoding system includes image processing circuitry configured to receive source image data and derive full-resolution image data and low-resolution image data from the source image data. The video encoding system also includes a low resolution pipeline configured to receive the low-resolution image data and determine one or more low resolution inter prediction modes based on the low-resolution image data. Furthermore, the video encoding system includes a main pipeline configured to encode the full-resolution image data based on the one or more low resolution inter prediction modes.

Abstract: Determining a focus setting includes determining a plurality of regions of interest in a view of a scene, and, for each of the plurality of regions of interest, obtaining a set of image data for each of multiple focal positions, and then applying focus filters to the set of image data for each of the plurality of focal positions for each of the regions of interest to obtain a set of focus scores, i.e., a focus score for each focus filter applied to the set of image data for each of the focal positions. Further, determining a confidence value associated with each of the sets of focus scores, selecting a subset of the sets of focus scores based on the confidence values associated with each of the sets of focus scores, and determining a focus setting for the scene based on the selected subset of the focus scores.

Abstract: Embodiments relate to a normalized cross correlation (NCC) circuit that can perform a normalized cross correlation between input patch data and kernel data. An interface circuit of an image signal processor receives input patch data from a source. Input patch data is data that represents a portion of a frame of image data from the source. The NCC circuit includes a filtering circuit and a normalization circuit. The filtering circuit receives the input patch data from the interface circuit and performs a convolution on the received input patch data or processed patch data derived from the input patch data with kernel data to produce convolution output data. The normalization circuit computes a normalized score output based on the convolution output data and the kernel data. The normalized score output includes normalization scores for each location of the convolution output data.

Abstract: Embodiments relate to generation of hue maps for highlight recovery of an input image. An image having a plurality of color channels is obtained at a first resolution lower than a resolution of the input image. A hue for each color channel for each pixel is determined, using a pixel value for that color channel and pixel values for the plurality of color channels in the first image. Weights are determined for each pixel for each color channel, based on hues for the pixel and pixel values for the pixel in the first image. A plurality of candidate hue maps are generated, based on the weights and pixel values in the first image in a patch surrounding the pixel for the plurality of color channels.

Abstract: Embodiments of the present disclosure relate to a configurable convolution engine that receives configuration information to perform convolution or its variant operations on streaming input data of various formats. To process streaming input data, input data of multiple channels are received and stored in an input buffer circuit in an interleaved manner. Data values of the interleaved input data are retrieved and forwarded to multiplier circuits where multiplication with a corresponding filter element of a kernel is performed. Varying number of kernels with different sizes and sparsity can also be used for the convolution operations.

Abstract: Embodiments relate to a first demosaicing circuit and a second demosaicing circuit that can perform demosaicing of image data. The first demosaicing circuit processes received image data to generate a first demosaiced image for obtaining statistic information on the received image data. The second demosaicing circuit performs demosaicing of the received image data to generate a second demosaiced image. A processing circuit pipeline performs at least one of resampling, noise processing, color processing and output rescaling performed on the second demosaiced image based on the statistics information obtained from the first demosaiced image.

Abstract: Embodiments relate to a configurable convolution engine that receives configuration information to perform convolution and other deep machine learning operations on streaming input data of various formats. The convolution engine may include two convolution circuits that each generate a stream of values by applying convolution kernels to input data. The stream of values may each define one or more channels of image data. A channel merge circuit combines the streams of values from each convolution circuit in accordance with a selected mode of operation. In one mode, the first and second streams from the convolution circuits are merged into an output stream having the combined channels of the first and second streams in an interleaved manner. In another mode, the first stream from the first convolution circuit is fed into the input of the second convolution circuit.