Abstract: Various techniques are provided herein for processing raw image data in front-end processing logic of an image signal processing system. In one embodiment, the front-end processing logic includes a statistics processing unit configured to process raw image data acquired by an image sensor to obtain one or more sets of statistics. The statistics processing unit may first correct defective pixels in the raw image data and then correct lens shading errors in the raw image data prior to extracting the statistics information. In certain embodiments, black level compensation may be applied between the defective pixel correction and lens shading correction steps, and inverse black level compensation may be applied between the lens shading correction step and the extraction of the statistics information. The acquired statistics information may be utilized by an image signal processing pipeline for converting the raw image data into a color (e.g., RGB) and/or luma (e.g., YCbCr) image.

Abstract: A method for encoding an image is disclosed. The method generally includes the steps, of (A) generating a quantization matrix as a function of at least four parameters, (B) optimizing the parameters to maximize a quality metric for encoding the image and (C) encoding the image with the quantization matrix as optimized.

Abstract: Various techniques relating to image sharpening are provided. In one embodiment, a luminance image is obtained based upon image data acquired by an image sensor. A multi-scale unsharp mask, which may include at least two Gaussian filters of difference radii, is applied to the luminance image to determine a plurality of unsharp values. Each of the unsharp values may be compared to a corresponding threshold and, for the unsharp values that exceed their respective thresholds, the unsharp value is multiplied by a corresponding gain and added to a base image, which may be selected as the luminance image or the output of one of the Gaussian filters. Each gained unsharp value may be summed with the base image to produce a final sharpened output. In some embodiments, an attenuated gain may be applied to unsharp values that do not exceed their respective thresholds.

Abstract: A method to control weighted prediction for video compression. The method comprises (A) generating statistics based upon analysis of a plurality of video frames, (B) detecting a fade condition based upon the statistics, (C) generating one or more estimated weighted prediction parameters based upon the statistics when the fade condition is detected and (D) encoding the plurality of video frames. The encoding takes into account the estimated weighted prediction parameters when generated.

Abstract: A video transcoder is disclosed. The video transcoder generally comprises a processor and a video digital signal processor. The processor may be formed on a first die. The video digital signal processor may be formed on a second die and coupled to the processor. The video digital signal processor may have (i) a first module configured to perform a first operation in decoding an input video stream in a first format and (ii) a second module configured to perform a second operation in coding an output video stream in a second format, wherein the first operation and the second operation are performed in parallel.

Abstract: A method for transcoding from an MPEG-2 format to an H.264 format is disclosed. The method generally comprises the steps of (A) decoding an input video stream in the MPEG-2 format to generate a plurality of macroblocks; (B) determining a plurality of indicators from a pair of the macroblocks, the pair of the macroblocks being vertically adjoining; and (C) coding the pair of the macroblocks into an output video stream in the H.264 format using one of (i) a field mode coding and (ii) a frame mode coding as determined from the indicators.

Abstract: Various techniques are provided for the detection and correction of defective pixels in an image sensor. In accordance with one embodiment, a static defect table storing the locations of known static defects is provided, and the location of a current pixel is compared to the static defect table. If the location of the current pixel is found in the static defect table, the current pixel is identified as a static defect and is corrected using the value of the previous pixel of the same color. If the current pixel is not identified as a static defect, a dynamic defect detection process includes comparing pixel-to-pixel gradients between the current pixel a set of neighboring pixels against a dynamic defect threshold. If a dynamic defect is detected, a replacement value for correcting the dynamic defect may be determined by interpolating the value of two neighboring pixels on opposite sides of the current pixel in a direction exhibiting the smallest gradient.

Abstract: A method for deinterlacing a picture is disclosed. The method generally includes the steps of (A) generating a plurality of primary scores by searching along a plurality of primary angles for an edge in the picture proximate a location interlaced with a field of the picture, (B) generating a plurality of neighbor scores by searching for the edge along a plurality of neighbor angles proximate a particular angle of the primary angles corresponding to a particular score of the primary scores having a best value and (C) identifying a best score from a group of scores consisting of the particular score and the neighbor scores to generate an interpolated sample at the location.

Abstract: A video transcoder is disclosed. The video transcoder generally comprises a processor and a video digital signal processor. The processor may be formed on a first die. The video digital signal processor may be formed on a second die and coupled to the processor. The video digital signal processor may have (i) a first module configured to perform a first operation in decoding an input video stream in a first format and (ii) a second module configured to perform a second operation in coding an output video stream in a second format, wherein the first operation and the second operation are performed in parallel.

Abstract: A method for transcoding from an H.264 format to an MPEG-2 format is disclosed. The method generally comprises the steps of (A) decoding an input video stream in the H.264 format to generate a picture having a plurality of macroblock pairs that used an H.264 macroblock adaptive field/frame coding; (B) determining a mode indicator for each of the macroblock pairs; and (C) coding the macroblock pairs into an output video stream in the MPEG-2 format using one of (i) an MPEG-2 field mode coding and (ii) an MPEG-2 frame mode coding as determined from the mode indicators.

Abstract: An apparatus comprising a first circuit, a second circuit and a third circuit. The first circuit may be configured to determine one or more signal characteristics in a portion of an input video signal. The second circuit may be configured to select a multiplier value from a plurality of multiplier values in response to the signal characteristics. The third circuit may be configured to generate an encoded bitstream in response to (i) the input video signal and (ii) the selected multiplier value.

Abstract: A method for transcoding from an MPEG-2 format to an H.264 format is disclosed. The method generally comprises the steps of (A) decoding an input video stream in the MPEG-2 format to generate a plurality of macroblocks; (B) determining a plurality of indicators from a pair of the macroblocks, the pair of the macroblocks being vertically adjoining; and (C) coding the pair of the macroblocks into an output video stream in the H.264 format using one of (i) a field mode coding and (ii) a frame mode coding as determined from the indicators.

Abstract: Certain aspects of this disclosure relate to an image signal processing system that includes a flash controller that is configured to activate a flash device prior to the start of a target image frame by using a sensor timing signal. In one embodiment, the flash controller receives a delayed sensor timing signal and determines a flash activation start time by using the delayed sensor timing signal to identify a time corresponding to the end of the previous frame, increasing that time by a vertical blanking time, and then subtracting a first offset to compensate for delay between the sensor timing signal and the delayed sensor timing signal. Then, the flash controller subtracts a second offset to determine the flash activation time, thus ensuring that the flash is activated prior to receiving the first pixel of the target frame.

Abstract: Certain embodiments of the present disclosure provide a flexible memory input/output controller that is configured to the storing and reading of multiple types of pixels and pixel memory formats. For instance, the memory I/O controller may support the storing and reading of raw image pixels at various bits of precision, such as 8-bit, 10-bit, 12-bit, 14-bit, and 16-bit. Pixel formats that are unaligned with memory bytes (e.g., not being a multiple of 8-bits) may be stored in a packed manner. The memory I/O controller may also support various formats of RGB pixel sets and YCC pixel sets.

Abstract: The present disclosure provides techniques relates to the implementation of a raw pixel processing unit using a set of line buffers. In one embodiment, the set of line buffers may include a first subset and second subset. Various logical units of the raw pixel processing unit may be implemented using the first and second subsets of line buffers in a shared manner. For instance, in one embodiment, defective pixel correction and detection logic may be implemented using the first subset of line buffers. The second subset of line buffers may be used to implement lens shading correction logic, gain, offset, and clamping logic, and demosaicing logic. Further, noise reduction may also be implemented using at least a portion of each of the first and second subsets of line buffers.

Abstract: Systems and methods are disclosed for applying spatial filtering to raw image data. In one embodiment, a spatial filter may identify an n×n block of pixels from the current image frame, the n×n block including a plurality of neighboring pixels being centered about a current input pixel and being of the same color component as the current input pixel. The spatial filter may include a plurality of filter taps, with one filter tap corresponding to each of the pixels within the n×n block. A set of filtering coefficients for each filter tap, which may be based on a Gaussian function, may be determined. A pixel difference value between the current input pixel and each of the plurality of neighboring pixels in the n×n block are determined, and the pixel differences may be used to determine an attenuation factor for each filter tap.

Abstract: The present disclosure provides techniques for performing audio-video synchronization using an image signal processing system. In one embodiment, a time code register provides a current time stamp when sampled. The value of the time code register may be incremented at regular intervals based on a clock of the image signal processing system. At the start of a current frame acquired by an image sensor, the time code register is sampled, and a timestamp is stored into a timestamp register associated with the image sensor. The timestamp is then read from the time stamp register and written to a set of metadata associated with the current frame. The timestamp stored in the frame metadata may then be used to synchronize the current frame with a corresponding set of audio data.

Abstract: Disclosed embodiments provide for a an image signal processing system that includes back-end pixel processing unit that receives pixel data after being processed by at least one of a front-end pixel processing unit and a pixel processing pipeline. In certain embodiments, the back-end processing unit receives luma/chroma image data and may be configured to apply face detection operations, local tone mapping, bright, contrast, color adjustments, as well as scaling. Further, the back-end processing unit may also include a back-end statistics unit that may collect frequency statistics. The frequency statistics may be provided to an encoder and may be used to determine quantization parameters that are to be applied to an image frame.

Abstract: Certain embodiments disclosed herein relate to an image signal processing system includes overflow control logic that detects an overflow condition when a destination unit when a sensor input queue and/or front-end processing unit receives back pressure from a downstream destination unit. In one embodiment, pixels of a current frame are dropped when an overflow condition occurs. The number of dropped pixels may be tracked using a counter. Upon recovery of the overflow condition, the remaining pixels of the frame are received and each dropped pixel may be replaced using a replacement pixel value.

Abstract: An apparatus including a first circuit and a second circuit. The first circuit may be configured to generate (i) a first series of sequential frames, (ii) a plurality of local motion vectors for each of said frames, (iii) one or more global motion vectors for each of said frames, (iv) a second series of stabilized sequential frames, (v) a plurality of rough motion vectors and (vi) a digital bitstream in response to (i) a video input signal. The second circuit may be configured to store (i) the first series of sequential frames, (ii) the plurality of local motion vectors, (iii) the one or more global motion vectors, (iv) the second series of stabilized sequential frames and (v) the plurality of rough motion vectors.