Abstract: A method is provided that includes receiving pictures of a video sequence in a video encoder, and encoding the pictures to generate a compressed video bit stream that is transmitted to a video decoder in real-time, wherein encoding the pictures includes selecting a picture to be encoded as a delayed duplicate intra-predicted picture (DDI), wherein the picture would otherwise be encoded as an inter-predicted picture (P-picture), encoding the picture as an intra-predicted picture (I-picture) to generate the DDI, wherein the I-picture is reconstructed and stored for use as a reference picture for a decoder refresh picture, transmitting the DDI to the video decoder in non-real time, selecting a subsequent picture to be encoded as the decoder refresh picture, and encoding the subsequent picture in the compressed bit stream as the decoder refresh picture, wherein the subsequent P-picture is encoded as a P-picture predicted using the reference picture.

Abstract: A method for analyzing images to generate a plurality of output features includes receiving input features of the image and performing Fourier transforms on each input feature. Kernels having coefficients of a plurality of trained features are received and on-the-fly Fourier transforms (OTF-FTs) are performed on the coefficients in the kernels. The output of each Fourier transform and each OTF-FT are multiplied together to generate a plurality of products and each of the products are added to produce one sum for each output feature. Two-dimensional inverse Fourier transforms are performed on each sum.

Abstract: A low power video hardware engine is disclosed. The video hardware engine includes a video hardware accelerator unit. A shared memory is coupled to the video hardware accelerator unit, and a scrambler is coupled to the shared memory. A vDMA (video direct memory access) engine is coupled to the scrambler, and an external memory is coupled to the vDMA engine. The scrambler receives an LCU (largest coding unit) from the vDMA engine. The LCU comprises N×N pixels, and the scrambler scrambles N×N pixels in the LCU to generate a plurality of blocks with M×M pixels. N and M are integers and M is less than N.

Abstract: The disclosure provides a sample adaptive offset (SAO) encoder. The SAO encoder includes a statistics collection (SC) block and a rate distortion optimization (RDO) block coupled to the SC block. The SC block receives a set of deblocked pixels and a set of original pixels. The SC block categorizes each deblocked pixel of the set of deblocked pixels in at least one of a plurality of band and edge categories. The SC block estimates an error in each category as difference between a deblocked pixel of the set of deblocked pixels and corresponding original pixel of the set of original pixels. The RDO block determines a set of candidate offsets associated with each category and selects a candidate offset with a minimum RD cost. The minimum RD cost is used by a SAO type block and a decision block to generate final offsets for the SAO encoder.

Abstract: A hardware thread scheduler (HTS) is provided for a multiprocessor system. The HTS is configured to schedule processing of multiple threads of execution by resolving data dependencies between producer modules and consumer modules for each thread. Pattern adaptors may be provided in the scheduler that allows mixing of multiple data patterns across blocks of data. Transaction aggregators may be provided that allow re-using the same image data by multiple threads of execution while the image date remains in a given data buffer. Bandwidth control may be provided using programmable delays on initiation of thread execution. Failure and hang detection may be provided using multiple watchdog timers.

Abstract: A hardware thread scheduler (HTS) is provided for a multiprocessor system. The HTS is configured to schedule processing of multiple threads of execution by resolving data dependencies between producer modules and consumer modules for each thread. Pattern adaptors may be provided in the scheduler that allows mixing of multiple data patterns across blocks of data. Transaction aggregators may be provided that allow re-using the same image data by multiple threads of execution while the image date remains in a given data buffer. Bandwidth control may be provided using programmable delays on initiation of thread execution. Failure and hang detection may be provided using multiple watchdog timers.

Abstract: A method is provided that includes receiving pictures of a video sequence in a video encoder, and encoding the pictures to generate a compressed video bit stream that is transmitted to a video decoder in real-time, wherein encoding the pictures includes selecting a picture to be encoded as a delayed duplicate intra-predicted picture (DDI), wherein the picture would otherwise be encoded as an inter-predicted picture (P-picture), encoding the picture as an intra-predicted picture (I-picture) to generate the DDI, wherein the I-picture is reconstructed and stored for use as a reference picture for a decoder refresh picture, transmitting the DDI to the video decoder in non-real time, selecting a subsequent picture to be encoded as the decoder refresh picture, and encoding the subsequent picture in the compressed bit stream as the decoder refresh picture, wherein the subsequent P-picture is encoded as a P-picture predicted using the reference picture.

Abstract: A signal processing chain implements wide dynamic range (WDR) multi-frame processing including receiving raw image signals from a WDR sensor including a plurality of frames including a first frame including first exposure time pixel data and a second frame including second exposure time pixel data. Statistics for camera control are generated including first statistics for the first pixel data and second statistics for the second pixel data. The first and second pixel data are merged using WDR merge algorithm in a WDR merge block which utilizes the first and second statistics to generate a raw higher bit width single frame image. The single frame image is post-processed in post-processing block using at least a defect pixel correction algorithm, and at least a portion of tone mapping is performed on the single frame image after the post-processing to provide an output toned mapped image.

Abstract: A method of de-blocking filtering a processed video is provided. The processed video includes a plurality of blocks and each block includes a plurality of sub-blocks. A current block of the plurality of blocks includes vertical edges and horizontal edges. The processed video further includes a set of control parameters and reconstructed pixels corresponding to the current block. A boundary strength index is estimated at the vertical edges and at the horizontal edges of the current block. The set of control parameters, the reconstructed pixels corresponding to the current block and partially filtered pixels corresponding to a set of adjacent sub-blocks are loaded. The vertical edges and the horizontal edges of the current block are filtered based on the boundary strength index and the set of control parameters such that a vertical edge of the current block is filtered before filtering at least one horizontal edge of the current block.

Abstract: An apparatus for sample adaptive offset (SAO) filtering in video encoding. A unified processing engine collects statistics on a block of pixels, determines a minimum RD cost (J) for each category of band offsets and edge offsets; determines a RD cost to find the optimal SAO type and determines a cost for each of the left SAO parameters and the up SAO parameters. The unified processing engine operates for three iterations: once for luminance once for each chrominance. A SAO merge decision unit determines an optimal mode and generates current LCU Parameters. The RD offset unit determination includes determining whether the sign of the minimum offset is proper for the category of edge offset. The RD offset is determined using a programmable look-up table indexed by the offset to estimate a rate. The unified processing engine operates on a three stage pipeline: loading blocks; processing; and updating blocks.

Abstract: A progressive JPEG (joint photographic experts group) decoder is disclosed. The progressive JPEG decoder includes a processing unit that receives a plurality of progressive scans and generates a plurality of modified progressive scans (MPSs). A baseline JPEG decoder, coupled to the processing unit, includes a Huffman decoder, an inverse quantization unit and an inverse transform unit. The Huffman decoder receives a current MPS and generates a set of transform coefficients corresponding to the current MPS. The processing unit adds a set of transform coefficients corresponding to a previous MPS to the set of transform coefficients corresponding to the current MPS, and the processing unit generates quantization indices. The inverse quantization unit estimates inverse quantization values based on the quantization indices. The inverse transform unit estimates inverse transform decoded values based on the estimated inverse quantization values and generates a set of pixels corresponding to the current MPS.

Abstract: A video hardware engine which support dynamic frame padding is disclosed. The video hardware engine includes an external memory. The external memory stores a reference frame. The reference frame includes a plurality of reference pixels. A motion estimation (ME) engine receives a current LCU (largest coding unit), and defines a search area around the current LCU for motion estimation. The ME engine receives a set of reference pixels corresponding to the current LCU. The set of reference pixels of the plurality of reference pixels are received from the external memory. The ME engine pads a set of duplicate pixels along an edge of the reference frame when a part area of the search area is outside the reference frame.

Abstract: A VP8 video decoder is implemented by partitioning the required functions across multiple sub systems, with an optimal mapping to existing functional blocks. Key optimizations include the reuse of hardware designed for prior generation V?6 and VP7 decoders. In order to reduce implementation complexity, cost and power consumption, a non exact, approximate deblocking loop filter is implemented.

Abstract: A progressive JPEG (joint photographic experts group) decoder is disclosed. The progressive JPEG decoder includes a processing unit that receives a plurality of progressive scans and generates a plurality of modified progressive scans (MPSs). A baseline JPEG decoder, coupled to the processing unit, includes a Huffman decoder, an inverse quantization unit and an inverse transform unit. The Huffman decoder receives a current MPS and generates a set of transform coefficients corresponding to the current MPS. The processing unit adds a set of transform coefficients corresponding to a previous MPS to the set of transform coefficients corresponding to the current MPS, and the processing unit generates quantization indices. The inverse quantization unit estimates inverse quantization values based on the quantization indices. The inverse transform unit estimates inverse transform decoded values based on the estimated inverse quantization values and generates a set of pixels corresponding to the current MPS.

Abstract: A method for filtering noise for imaging includes receiving an image frame having position and range data. A filter size divides the frame into filter windows for processing each of the filter windows. For the first pixel, a space to the center pixel and a range difference between this pixel and the center pixel is determined and used for choosing a selected weight from weights in a 2D weight LUT including weighting for space and range difference, a filtered range value is calculated by applying the selected 2D weight to the pixel, and the range, filtered range value and selected 2D weight are summed. The determining, choosing, calculating and summing are repeated for at least the second pixel. A total sum of contributions from the first and second pixel are divided by the sum of selected 2D weights to generate a final filtered range value for the center pixel.

Abstract: A signal processing chain implements wide dynamic range (WDR) multi-frame processing including receiving raw image signals from a WDR sensor including a plurality of frames including a first frame including first exposure time pixel data and a second frame including second exposure time pixel data. Statistics for camera control are generated including first statistics for the first pixel data and second statistics for the second pixel data. The first and second pixel data are merged using WDR merge algorithm in a WDR merge block which utilizes the first and second statistics to generate a raw higher bit width single frame image. The single frame image is post-processed in post-processing block using at least a defect pixel correction algorithm, and at least a portion of tone mapping is performed on the single frame image after the post-processing to provide an output toned mapped image.

Abstract: A VP8 video decoder is implemented by partitioning the required functions across multiple sub systems, with an optimal mapping to existing functional blocks. Key optimizations include the reuse of hardware designed for prior generation V^6 and VP7 decoders. In order to reduce implementation complexity, cost and power consumption, a non exact, approximate deblocking loop filter is implemented.

Abstract: A method of de-mosaicing pixel data from an image processor includes generating a pixel block that includes a plurality of image pixels. The method also includes determining a first image gradient between a first set of pixels of the pixel block and a second image gradient between a second set of pixels of the pixel block. The method also includes determining a first adaptive threshold value based on intensity of a third set of pixels of the pixel block. The pixels of the third set of pixels are adjacent to one another. The method also includes filtering the pixel block in a vertical, horizontal, or neutral direction based on the first and second image gradients and the first adaptive threshold value utilizing a plurality of FIR filters to generate a plurality of component images.

Abstract: An apparatus for scaling images is provided that includes at least two input ports, a scaling component coupled to the at least two input ports, the scaling component including a plurality of scalers, the scaling component configurable to map any scaler to any input port of the at least two input ports and configurable to map more than one scaler to any input port, and a memory coupled to the at least two input ports and to outputs of the plurality of scalers, the memory configured to store image data for each input port and scaled image data output by the plurality of scalers.

Abstract: A progressive JPEG (joint photographic experts group) decoder is disclosed. The progressive JPEG decoder includes a processing unit that receives a plurality of progressive scans and generates a plurality of modified progressive scans (MPSs). A baseline JPEG decoder, coupled to the processing unit, includes a Huffman decoder, an inverse quantization unit and an inverse transform unit. The Huffman decoder receives a current MPS and generates a set of transform coefficients corresponding to the current MPS. The processing unit adds a set of transform coefficients corresponding to a previous MPS to the set of transform coefficients corresponding to the current MPS, and the processing unit generates quantization indices. The inverse quantization unit estimates inverse quantization values based on the quantization indices. The inverse transform unit estimates inverse transform decoded values based on the estimated inverse quantization values and generates a set of pixels corresponding to the current MPS.