Abstract: Systems and methods for processing image data in RGB format are provided. In one example, an electronic device includes memory to store image data in raw or RGB format, or both, and an RGB image processing pipeline to process the image data. Specifically, the RGB image processing pipeline may process the image data regardless of whether the image data is of raw or RGB format. The RGB image processing pipeline may include receiving logic to receive the image data in raw or RGB format and demosaicing logic to, when the receiving logic receives the image data in raw format, convert the image data into RGB format. The logic may include local tone mapping logic configured to apply spatially varying tone curves to the image data, a color correction matrix configured to correct color in the image data, and gamma logic configured to transform the image data into gamma space.

Abstract: In an embodiment, a system includes a display processing unit configured to process a video sequence for a target display. In some embodiments, the display processing unit is configured to composite the frames from frames of the video sequence and one or more other image sources. The display processing unit may be configured to write the processed/composited frames to memory, and may also be configured to generate statistics over the frame data, where the generated statistics are usable to encode the frame in a video encoder. The display processing unit may be configured to write the generated statistics to memory, and the video encoder may be configured to read the statistics and the frames. The video encoder may be configured to encode the frame responsive to the statistics.

Abstract: Embodiments of the present invention may provide a video coder. The video coder may include an encoder to perform coding operations on a video signal in a first format to generate coded video data, and a decoder to decode the coded video data. The video coder may also include an inverse format converter to convert the decoded video data to second format that is different than the first format and an estimator to generate a distortion metric using the decoded video data in the second format and the video signal in the second format. The encoder may adjust the coding operations based on the distortion metric.

Abstract: Adaptive video processing for a target display panel may be implemented in or by a decoding/display pipeline associated with the target display panel. The adaptive video processing methods may take into account video content, display characteristics, and environmental conditions including but not limited to ambient lighting and viewer location when processing and rendering video content for a target display panel in an ambient setting or environment. The display-side adaptive video processing methods may use this information to adjust one or more video processing functions as applied to the video data to render video for the target display panel that is adapted to the display panel according to the ambient viewing conditions.

Abstract: A method and system for adaptively mixing video components with graphics/UI components, where the video components and graphics/UI components may be of different types, e.g., different dynamic ranges (such as HDR, SDR) and/or color gamut (such as WCG). The mixing may result in a frame optimized for a display device's color space, ambient conditions, viewing distance and angle, etc., while accounting for characteristics of the received data. The methods include receiving video and graphics/UI elements, converting the video to HDR and/or WCG, performing statistical analysis of received data and any additional applicable rendering information, and assembling a video frame with the received components based on the statistical analysis. The assembled video frame may be matched to a color space and displayed. The video data and graphics/UI data may have or be adjusted to have the same white point and/or primaries.

Abstract: Video processing techniques and pipelines that support capture, distribution, and display of high dynamic range (HDR) image data to both HDR-enabled display devices and display devices that do not support HDR imaging. A sensor pipeline may generate standard dynamic range (SDR) data from HDR data captured by a sensor using tone mapping, for example local tone mapping. Information used to generate the SDR data may be provided to a display pipeline as metadata with the generated SDR data. If a target display does not support HDR imaging, the SDR data may be directly rendered by the display pipeline. If the target display does support HDR imaging, then an inverse mapping technique may be applied to the SDR data according to the metadata to render HDR data for display. Information used in performing color gamut mapping may also be provided in the metadata and used to recover clipped colors for display.

Abstract: Adaptive video processing for a target display panel may be implemented in or by a server/encoding pipeline. The adaptive video processing methods may obtain and take into account video content and display panel-specific information including display characteristics and environmental conditions (e.g., ambient lighting and viewer location) when processing and encoding video content to be streamed to the target display panel in an ambient setting or environment. The server-side adaptive video processing methods may use this information to adjust one or more video processing functions as applied to the video data to generate video content in the color gamut and dynamic range of the target display panel that is adapted to the display panel characteristics and ambient viewing conditions.

Abstract: Systems and methods for correcting green channel non-uniformity (GNU) are provided. In one example, GNU may be corrected using energies between the two green channels (Gb and Gr) during green interpolation processes for red and green pixels. Accordingly, the processes may be efficiently employed through implementation using demosaic logic hardware. In addition, the green values may be corrected based on low-pass-filtered values of the green pixels (Gb and Gr). Additionally, green post-processing may provide some defective pixel correction on interpolated greens by correcting artifacts generated through enhancement algorithms.

Abstract: Systems and methods for local tone mapping are provided. In one example, an electronic device includes an electronic display, an imaging device, and an image signal processor. The electronic display may display images of a first bit depth, and the imaging device may include an image sensor that obtains image data of a higher bit depth than the first bit depth. The image signal processor may process the image data, and may include local tone mapping logic that may apply a spatially varying local tone curve to a pixel of the image data to preserve local contrast when displayed on the display. The local tone mapping logic may smooth the local tone curve applied to the intensity difference between the pixel and another nearby pixel exceeds a threshold.

Abstract: Systems and methods for generating local image statistics are provided. In one example, an image signal processing system may include a statistics pipeline with image processing logic and local image statistics collection logic. The image processing logic may receive and process pixels of raw image data. The local image statistics collection logic may generate a local histogram associated with a luminance of the pixels of a first block of pixels of the raw image data or a thumbnail in which a pixel of the thumbnail represents a downscaled version of the luminance of the pixels of the first block of the pixel. The raw image data may include many other blocks of pixels of the same size as the first block of pixels.

Abstract: A method for coding video is disclosed. The method generally includes the steps of (A) receiving a video signal having a series of pictures, each of the pictures having a plurality of blocks, (B) analyzing the blocks to forecast if coding the blocks in a zero-residual coding mode would generate a plurality of artifacts, (C) disabling the zero-residual coding mode for the blocks forecasted to generate at least one of the artifacts and (D) enabling the zero-residual coding mode for the blocks forecasted to generate none of the artifacts.

Abstract: Systems, methods, and devices for sharpening image data are provided. One example of an image signal processing system includes a YCC processing pipeline that includes luma sharpening logic. The luma sharpening logic may sharpen the luma component while avoiding sharpening some noise. Specifically, a multi-scale unsharp mask filter may obtain unsharp signals by filtering an input luma component, and sharp component determination logic may determine sharp signals representing differences between the unsharp signals and the luma component. Sharp lookup tables may “core” the sharp signals, which may prevent some noise from being sharpened. Output logic may determine a sharpened output luma signal by combining the sharp signals with, for example, luma component or one of the unsharp signals.

Abstract: Systems and methods for processing YCC image data provided. In one example, an electronic device includes memory to store image data in RGB or YCC format and a YCC image processing pipeline to process the image data. The YCC image processing pipeline may include receiving logic configured to receive the image data in RGB or YCC format and color space conversion logic configured to, when the image data is received in RGB format, convert the image data into YCC format. The YCC image processing logic may also include luma sharpening and chroma suppression logic; brightness, contrast, and color adjustment logic; gamma logic; chroma decimation logic; scaling logic; and chroma noise reduction logic.

Abstract: A block processing pipeline that includes a software pipeline and a hardware pipeline that run in parallel. The software pipeline runs at least one block ahead of the hardware pipeline. The stages of the pipeline may each include a hardware pipeline component that performs one or more operations on a current block at the stage. At least one stage of the pipeline may also include a software pipeline component that determines a configuration for the hardware component at the stage of the pipeline for processing a next block while the hardware component is processing the current block. The software pipeline component may determine the configuration according to information related to the next block obtained from an upstream stage of the pipeline. The software pipeline component may also obtain and use information related to a block that was previously processed at the stage.

Abstract: A video encoder may include a context-adaptive binary arithmetic coding (CABAC) encode component that converts each syntax element of a representation of a block of pixels to binary code, serializes it, and codes it mathematically, after which the resulting bit stream is output. A lookup table in memory and a context cache may store probability values for supported contexts, which may be retrieved from the table or cache for use in coding syntax elements. Depending on the results of a syntax element coding, the probability value for its context may be modified (e.g., increased or decreased) in the cache and, subsequently, in the table. After coding multiple syntax elements, and based on observed access patterns for probability values, a mapping or indexing for the cache or the table may be modified to improve cache performance (e.g., to reduce cache misses or access data for related contexts using fewer accesses).

Abstract: Methods and apparatus for configuring multiple components of a subsystem are described. The configuration memory of each of a plurality of components coupled to an interconnect includes a global configuration portion. The configuration memory of one of the components may be designated as a master global configuration for all of the components. A module coupled to the interconnect may receive writes to the components from a configuration source. For each write, the module may decode the write to determine addressing information and check to see if the write is addressed to the master global configuration. If the write is addressed to the master global configuration, the module broadcasts the write to the global configuration portion of each of the components via the interconnect. If the write is not addressed to the master global configuration, the module forwards the write to the appropriate component via the interconnect.

Abstract: In the video encoders described herein, blocks of pixels from a video frame may be encoded (e.g., using CAVLC encoding) in a block processing pipeline using wavefront ordering (e.g., in knight's order). Each of the encoded blocks may be written to a particular one of multiple DMA buffers such that the encoded blocks written to each of the buffers represent consecutive blocks of the video frame in scan order. A transcode pipeline may operate in parallel with (or at least overlapping) the operation of the block processing pipeline. The transcode pipeline may read encoded blocks from the buffers in scan order and merge them into a single bit stream (in scan order). A transcoder core of the transcode pipeline may decode the encoded blocks and encode them using a different encoding process (e.g., CABAC). In some cases, the transcoder may be bypassed.

Abstract: Blocks of pixels from a video frame may be encoded in a block processing pipeline using wavefront ordering, e.g. according to knight's order. Each of the encoded blocks may be written to a particular one of multiple buffers such that the blocks written to each of the buffers represent consecutive blocks of the frame in scan order. Stitching information may be written to the buffers at the end of each row. A stitcher may read the rows from the buffers in order and generate a scan order output stream for the frame. The stitcher component may read the stitching information at the end of each row and apply the stitching information to one or more blocks at the beginning of a next row to stitch the next row to the previous row. Stitching may involve modifying pixel(s) of the blocks and/or modifying metadata for the blocks.

Abstract: The video encoders described herein may make an initial determination to designate a macroblock as a skip macroblock, but may subsequently reverse that decision based on additional information. For example, an initial skip mode decision may be based on aggregate distortion metrics for the luma component of the macroblock (e.g., SAD, SATD, or SSD), then reversed based on an individual pixel difference metric, an aggregate or individual pixel metric for a chroma component of the macroblock, or on the position of the macroblock within a macroblock row. The final skip mode decision may be based, at least in part, on the maximum difference between any pixel in the macroblock (or in a region of interest within the macroblock) and the corresponding pixel in a reference frame. The initial skip mode decision may be made during an early stage of a pipelined video encoding process and reversed in a later stage.

Abstract: A knight's order processing method for block processing pipelines in which the next block input to the pipeline is taken from the row below and one or more columns to the left in the frame. The knight's order method may provide spacing between adjacent blocks in the pipeline to facilitate feedback of data from a downstream stage to an upstream stage. The rows of blocks in the input frame may be divided into sets of rows that constrain the knight's order method to maintain locality of neighbor block data. Invalid blocks may be input to the pipeline at the left of the first set of rows and at the right of the last set of rows, and the sets of rows may be treated as if they are horizontally arranged rather than vertically arranged, to maintain continuity of the knight's order algorithm.