Abstract: Memory latency tolerance methods and apparatus for maintaining an overall level of performance in block processing pipelines that prefetch reference data into a search window. In a general memory latency tolerance method, search window processing in the pipeline may be monitored. If status of search window processing changes in a way that affects pipeline throughput, then pipeline processing may be modified. The modification may be performed according to no stall methods, stall recovery methods, and/or stall prevention methods. In no stall methods, a block may be processed using the data present in the search window without waiting for the missing reference data. In stall recovery methods, the pipeline is allowed to stall, and processing is modified for subsequent blocks to speed up the pipeline and catch up in throughput. In stall prevention methods, processing is adjusted in advance of the pipeline encountering a stall condition.

Abstract: Block processing pipeline methods and apparatus in which reference data are stored to a memory according to tile formats to reduce memory accesses when fetching the data from the memory. When the pipeline stores reference data from a current frame being processed to memory as a reference frame, the reference samples are stored in macroblock sequential order. Each macroblock sample set is stored as a tile. Reference data may be stored in tile formats for luma and chroma. Chroma reference data may be stored in tile formats for chroma 4:2:0, 4:2:2, and/or 4:4:4 formats. A stage of the pipeline may write luma and chroma reference data for macroblocks to memory according to one or more of the macroblock tile formats in a modified knight's order. The stage may delay writing the reference data from the macroblocks until the macroblocks have been fully processed by the pipeline.

Abstract: A block processing pipeline in which macroblocks are input to and processed according to row groups so that adjacent macroblocks on a row are not concurrently at adjacent stages of the pipeline. The input method may allow chroma processing to be postponed until after luma processing. One or more upstream stages of the pipeline may process luma elements of each macroblock to generate luma results such as a best mode for processing the luma elements. Luma results may be provided to one or more downstream stages of the pipeline that process chroma elements of each macroblock. The luma results may be used to determine processing of the chroma elements. For example, if the best mode for luma is an intra-frame mode, then a chroma processing stage may determine a best intra-frame mode for chroma and reconstruct the chroma elements according to the best chroma intra-frame mode.

Abstract: Methods and apparatus for caching neighbor data in a block processing pipeline that processes blocks in knight's order with quadrow constraints. Stages of the pipeline may maintain two local buffers that contain data from neighbor blocks of a current block. A first buffer contains data from the last C blocks processed at the stage. A second buffer contains data from neighbor blocks on the last row of a previous quadrow. Data for blocks on the bottom row of a quadrow are stored to an external memory at the end of the pipeline. When a block on the top row of a quadrow is input to the pipeline, neighbor data from the bottom row of the previous quadrow is read from the external memory and passed down the pipeline, each stage storing the data in its second buffer and using the neighbor data in the second buffer when processing the block.

Abstract: A block processing pipeline in which blocks are input to and processed according to row groups so that adjacent blocks on a row are not concurrently at adjacent stages of the pipeline. A stage of the pipeline may process a current block according to neighbor pixels from one or more neighbor blocks. Since adjacent blocks are not concurrently at adjacent stages, the left neighbor of the current block is at least two stages downstream from the stage. Thus, processed pixels from the left neighbor can be passed back to the stage for use in processing the current block without the need to wait for the left neighbor to complete processing at a next stage of the pipeline. In addition, the neighbor blocks may include blocks from the row above the current block. Information from these neighbor blocks may be passed to the stage from an upstream stage of the pipeline.

Abstract: Block processing pipeline methods and apparatus in which pixel data from a reference frame is prefetched into a search window memory. The search window may include two or more overlapping regions of pixels from the reference frame corresponding to blocks from the rows in the input frame that are currently being processed in the pipeline. Thus, the pipeline may process blocks from multiple rows of an input frame using one set of pixel data from a reference frame that is stored in a shared search window memory. The search window may be advanced by one column of blocks by initiating a prefetch for a next column of reference data from a memory. The pipeline may also include a reference data cache that may be used to cache a portion of a reference frame and from which at least a portion of a prefetch for the search window may be satisfied.

Abstract: A method of signaling additional chroma QP offset values that are specific to quantization groups is provided, in which each quantization group explicitly specifies its own set of chroma QP offset values. Alternatively, a table of possible sets of chroma QP offset values is specified in the header area of the picture, and each quantization group uses an index to select an entry from the table for determining its own set of chroma QP offset values. The quantization group specific chroma QP offset values are then used to determine the chroma QP values for blocks within the quantization group in addition to chroma QP offset values already specified for higher levels of the video coding hierarchy.

Abstract: A method of signaling additional chroma QP offset values that are specific to quantization groups is provided, in which each quantization group explicitly specifies its own set of chroma QP offset values. Alternatively, a table of possible sets of chroma QP offset values is specified in the header area of the picture, and each quantization group uses an index to select an entry from the table for determining its own set of chroma QP offset values. The quantization group specific chroma QP offset values are then used to determine the chroma QP values for blocks within the quantization group in addition to chroma QP offset values already specified for higher levels of the video coding hierarchy.

Abstract: Some embodiments provide a method of operating a device to capture an image of a high dynamic range (HDR) scene. Upon the device entering an HDR mode, the method captures and stores multiple images at a first image exposure level. Upon receiving a command to capture the HDR scene, the method captures a first image at a second image exposure level. The method selects a second image from the captured plurality of images. The method composites the first and second images to produce a composite image that captures the HDR scene. In some embodiments, the method captures multiple images at multiple different exposure levels.

Abstract: The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may compute noise statistics associated with image data by receiving a frame of the image data having a plurality of pixels. The image processing pipeline may then identify a plurality of portions of the frame of the image data such that each portion of the plurality of portions has a flat surface. The image processing pipeline may then calculate a plurality of gradients for each portion of the plurality of portions, determine one or more dominant gradient orientations for each portion of the plurality of portions, and generate a histogram that represents a plurality of dominant gradient orientations that corresponds to the plurality of portions. After generating the histogram, the image processing pipeline may store the histogram, which may represent the noise statistics, in a memory.

Abstract: An apparatus including a first circuit and a second circuit. The first circuit may be configured to generate (i) a first series of sequential frames, (ii) a plurality of local motion vectors for each of the frames, (iii) one or more global motion vectors for each of the frames, (iv) a second series of stabilized sequential frames, (v) a plurality of rough motion vectors and (vi) a digital bitstream in response to (i) a video input signal. The second circuit may be configured to generate a single motion vector in response to a plurality of motion vectors. The second circuit may be further configured to eliminate outlier vectors from the plurality of motion vectors.

Abstract: Various techniques are disclosed for collecting and processing auto-focus statistics data in an image signal processor (ISP). In one embodiment, a statistics collection engine in an ISP front-end processing unit may be configured to collect coarse (based on decimated raw data) and fine auto-focus statistics. Coarse auto-focus statistics may be collected on decimated Bayer RGB data and/or on linear camera luma values. Fine auto-focus statistics may be collected on raw Bayer RGB using a combination of a horizontal filter and edge detector, or may be collected on BayerY data (by applying a 3×1 transform to the raw Bayer RGB data). Edge sums may be accumulated using the filter outputs to determine auto-focus statistics.

Abstract: Systems and methods for correcting geometric distortion are provided. In one example, an electronic device may include an imaging device, which may obtain image data of a first resolution, and geometric distortion and scaling logic. The imaging device may include a sensor and a lens that causes some geometric distortion in the image data. The geometric distortion correction and scaling logic may scale and correct for geometric distortion in the image data by determining first pixel coordinates in uncorrected or partially corrected image data that, when resampled, would produce corrected output image data at second pixel coordinates. The geometric distortion correction and scaling logic may resample pixels around the image data at the first pixel coordinates to obtain the corrected output image data at the second pixel coordinates. The corrected output image data may be of a second resolution.

Abstract: Some embodiments provide a method of operating a device to capture an image of a high dynamic range (HDR) scene. Upon the device entering an HDR mode, the method captures and stores multiple images at a first image exposure level. Upon receiving a command to capture the HDR scene, the method captures a first image at a second image exposure level. The method selects a second image from the captured plurality of images. The method composites the first and second images to produce a composite image that captures the HDR scene. In some embodiments, the method captures multiple images at multiple different exposure levels.

Abstract: Systems and methods for processing raw image data are provided. One example of such a system may include memory to store image data in raw format from a digital imaging device and an image signal processor to process the image data. The image signal processor may include data conversion logic and a raw image processing pipeline. The data conversion logic may convert the image data into a signed format to preserve negative noise from the digital imaging device. The raw image processing pipeline may at least partly process the image data in the signed format. The raw image processing pipeline may also include, among other things, black level compensation logic, fixed pattern noise reduction logic, temporal filtering logic, defective pixel correction logic, spatial noise filtering logic, lens shading correction logic, and highlight recovery logic.

Abstract: Certain aspects of this disclosure relate to an image signal processing system that includes a flash controller that is configured to activate a flash device prior to the start of a target image frame by using a sensor timing signal. In one embodiment, the flash controller receives a delayed sensor timing signal and determines a flash activation start time by using the delayed sensor timing signal to identify a time corresponding to the end of the previous frame, increasing that time by a vertical blanking time, and then subtracting a first offset to compensate for delay between the sensor timing signal and the delayed sensor timing signal. Then, the flash controller subtracts a second offset to determine the flash activation time, thus ensuring that the flash is activated prior to receiving the first pixel of the target frame.

Abstract: A video transcoder is disclosed. The video transcoder generally comprises a processor and a video digital signal processor. The processor may be formed on a first die. The video digital signal processor may be formed on a second die and coupled to the processor. The video digital signal processor may have (i) a first module configured to perform a first operation in decoding an input video stream in a first format and (ii) a second module configured to perform a second operation in coding an output video stream in a second format, wherein the first operation and the second operation are performed in parallel.

Abstract: The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may collect statistics associated with fixed pattern noise of image data by receiving a first frame of the image data comprising a plurality of pixels. The image processing pipeline may then determine a sum of a first plurality of pixel values that correspond to at least a first portion of the plurality of pixels such that each pixel in at least the first portion of the plurality of pixels is disposed along a first axis within the frame of the image data. After determining the sum of the first plurality of pixel values, the image processing pipeline may store the sum of the first plurality of pixel values in a memory such that the sum of the first plurality of pixel values represent the statistics.

Abstract: An apparatus including a first circuit and a second circuit. The first circuit may be configured to perform image signal processing using encoding related information. The second circuit may be configured to encode image data using image signal processing related information, wherein said first circuit is further configured to pass said image signal processing related information to said second circuit and said second circuit is further configured to pass said encoding related information to said first circuit.

Abstract: A method for transcoding from an H.264 format to an MPEG-2 format is disclosed. The method generally comprises the steps of (A) decoding an input video stream in the H.264 format to generate a picture having a plurality of macroblock pairs that used an H.264 macroblock adaptive field/frame coding; (B) determining a mode indicator for each of the macroblock pairs; and (C) coding the macroblock pairs into an output video stream in the MPEG-2 format using one of (i) an MPEG-2 field mode coding and (ii) an MPEG-2 frame mode coding as determined from the mode indicators.