Abstract: A method for designing a multi-threaded processing operation that includes, e.g., multimedia encoding/decoding, uses an architecture having multiple processors and optional hardware accelerators. The method includes the steps of: identifying a desired chronological sequence of processing stages for processing input data including identifying interdependencies of said processing stages; allotting each said processing sage to a processor; staggering the processing to accommodate the interdependencies; selecting a processing operation based on said allotting to arrive at a subset of possible pipelines that offer low average processing time; and, choosing one design pipeline from said subset to result in overall timing reduction to complete said processing operation. The invention provides a multi-threaded processing pipeline that is applicable in a System-on-Chip (SoC) using a DSP and shared resources such as DMA controller and on-chip memory, for increasing the throughput.

Abstract: The proposed technique provides simultaneous read and writes from a display controller using low-cost SDRAMs. This is achieved, in one example embodiment, by receiving a sequence of video frames at a first variable frame rate. A first video frame is then written in a first single-ported memory. The first video frame is then read from the first single-ported memory upon completing the writing of the first video frame in the first single-ported memory. The reading of the first video frame is then repeated from the first single-ported memory to maintain a second frame rate. The second frame rate is higher than the first variable frame rate. A second video frame is then written in a second single-ported memory upon completing the writing of the first video frame in the first single-ported memory such that the writing of the first video frame and the second video frame is at the first variable frame rate.

Abstract: A method and device for tracking error propagation and refreshing a video stream is provided. The proposed subject matter comprises of an error propagation tracking method that works in the sub-sampled domain to reduce computational cycles and memory bandwidth. Further, the tracking based update of the error propagation metric is done differently for static and non-static regions to avoid unnecessary refresh of static areas. Through suitable thresholding of the metric at a macroblock (MB) level, a set of refresh MBs are selected for each frame. These refresh MBs are coded either as an intra MB or as an inter MB that is predicted from one or more reliable reference frames (—frames that are known to be available at the decoder with negligible errors—). Such inter coding of refresh MBs improves the compression efficiency when compared to pure intra coding of refresh MBs.

Abstract: A method/system of transcoding an MPEG 2/4 bit stream into an H.264 format, handles an input MPEG 2/4 bit stream in a decoder, and identifies certain data in the input bit stream for reuse in the H.264 format; and, reuses the identified data in a re-encoder with assistance from a mapping module in transcoding by converting the input bit stream into an output H.264 format. The identified data includes information at a macrolevel and information at a picture level. The information at the macrolevel might comprise additional stages incorporated in the re-encoder module including a Mapping Process stage, a Sub Pixel Refinement stage, a Mode Selection stage to choose from Intra, Inter or Skip modes, followed by the standard H.264 encoding loop and the Entropy Coding Block. The information at the picture level might include; a) average quantizer of frame, and, b) total bits per frame.

Abstract: A motion estimation algorithm finds the best match for a given block or macroblock so that the resulting error signal has very low energy level which is computed, for e.g., by the SAD method. The motion estimation algorithm also provides for an optional sub-pixel level estimation and an inter4v search, and allows for restricting the number of searches for a Frame-frame ME (motion estimation) using Top-Top and Bottom-Bottom field MEs. The algorithm provides for a selective early exit and enables selecting a suitable search area with N candidate points (4 to 8) for starting the search. The search is conducted progressively till a minimum error signal (low energy level signal) is reached. The candidate points for search may be in a diamond shaped configuration, and there may be a plurality of successive diamond configurations, the number of which is configurable. The invention has application in MPEG-4 and H.264 standards.

Abstract: A design method for implementing a high-memory algorithm for motion estimation and compensation uses a low internal memory processor and a DMA engine that interacts with the processor and the algorithm. The DMA takes care of large data transfers from an external memory to the processor internal memory and vice-versa, without using the CPU clock cycles. The design method is scalable and is suited to handle huge bandwidths without slowing down the processor. To prevent the processor from being idle during DMA, the processing is pipelined and staggered so that motion compensation is performed on an earlier block or data that is available, while DMA fetches the reference data for the current block. Several DMAs may be set up under an ISR if necessary. The invention has application in video decoders including those conforming to H.264, VC-1, and MPEG-4 ASP.

Abstract: A method for designing a multi-threaded processing operation that includes, e.g., multimedia encoding/decoding, uses an architecture having multiple processors and optional hardware accelerators. The method includes the steps of: identifying a desired chronological sequence of processing stages for processing input data including identifying interdependencies of said processing stages; allotting each said processing sage to a processor; staggering the processing to accommodate the interdependencies; selecting a processing operation based on said allotting to arrive at a subset of possible pipelines that offer low average processing time; and, choosing one design pipeline from said subset to result in overall timing reduction to complete said processing operation. The invention provides a multi-threaded processing pipeline that is applicable in a System-on-Chip (SoC) using a DSP and shared resources such as DMA controller and on-chip memory, for increasing the throughput.

Abstract: A method for achieving bit rate controlled encoding, e.g., constant bit rate, uses a bit rate control model based on bits allocated per macroblock and based on error in estimating bits consumed in an encoding process in a macroblock. The method computes bits consumed per macroblock as a function of the formed bit rate control model, and allocates bits for the macroblock. To this end, a quadratic (second order) equation is used for the model, which equation might default into a first order equation requiring fewer computations, in the event that no second order solutions exist. In one form, the bit rate control model computes the bit rate to satisfy video buffer verifier (VBV) compliance in accordance with MPEG requirements. The bit rate control model makes allowances for the type of macroblock, i.e., inter, intra or bidirectional, and provides for any frame-skips.

Abstract: The proposed technique provides simultaneous read and writes from a display controller using low-cost SDRAMs. This is achieved, in one example embodiment, by receiving a sequence of video frames at a first variable frame rate. A first video frame is then written in a first single-ported memory. The first video frame is then read from the first single-ported memory upon completing the writing of the first video frame in the first single-ported memory. The reading of the first video frame is then repeated from the first single-ported memory to maintain a second frame rate. The second frame rate is higher than the first variable frame rate. A second video frame is then written in a second single-ported memory upon completing the writing of the first video frame in the first single-ported memory such that the writing of the first video frame and the second video frame is at the first variable frame rate.

Abstract: Apparatus and method for classifying regions of an image, based on the relative “importance” of the various areas and to adaptively use the importance information to allocate processing resources and input image formation.

Abstract: A PC-type computer has a system bus (e.g., a PCI bus) configured with a main CPU board, a statistical multiplexing (stat-mux) board, and a plurality of video/audio encoder boards, each configured to receive and compress a corresponding video/audio stream. The stat-mux board performs statistical multiplexing on the different compressed bitstreams to transmit multiple bitstreams over individual shared communication channels. Although each of the boards is configured to the system bus, each encoder board has a digital signal processor (DSP) with a synchronized serial interface (SSI) output port that is directly connected to an SSI input port on a DSP on the stat-mux board (which, in one embodiment, has four such DSPs each with six such SSI input ports). As such, (up to 24) compressed video/audio bitstreams generated on the various encoder boards can be transmitted directly to the stat-mux board without having to go through the system bus.

Abstract: When two or more different video streams a e compressed for concurrent transmission of multiple compressed video bitstreams over a single shared communication channel, control over both (1) the transmission of data over the shared channel and (2) the compression processing that generates the bitstreams is exercised taking into account the differing levels of latency required for the corresponding video applications. For example, interactive video games typically require lower latency than other video applications such as video streaming, web browsing, and electronic mail. A multiplexer and traffic controller takes these differing latency requirements, along with bandwidth and image fidelity requirements, into account when controlling both traffic flow and compression processing.

Abstract: An apparatus and method for implementing object motion segmentation and object trajectory segmentation for an image sequence. Specifically, block-based motion vectors for a pair of adjacent frames are used to derive optical flow, e.g., affine, motion parameters. Such optical flow motion parameters are employed to determine key objects where their motion and trajectory within a sequence of frames are calculated and stored. Such object motion information is used to improve or offer image processing functions such as context-based indexing of the input image sequence by using motion-based information.

Abstract: Apparatus and method for classifying regions of an image, based on the relative “importance” of the various areas and to adaptively use the importance information to allocate processing resources and input image formation.

Abstract: A method and concomitant apparatus for adapting the behavior of an MPEG-like encoder to information discontinuities within a received information stream, such that encoding quality and random access to a resulting encoded stream is retained near information discontinuity point without adversely impacting buffer utilization parameters. Specifically, an anchor frame comprising an I-frame preceding an information discontinuity is encoded as a P-frame, while an anchor frame following the information discontinuity is encoded as an I-frame.

Abstract: For video compression processing, each frame in a video sequence is segmented into one or more different regions, where the macroblocks of each region are to be encoded using the same quantizer value, but the quantizer value can vary between regions in a frame. For example, for the videophone or video-conferencing paradigm of one or more “talking heads” in front of a relatively static background, each frame is segmented into a foreground region corresponding to the talking head, a background region corresponding to the static background, and an intervening transition region. An encoding complexity measure is generated for each macroblock of the previous frame using a (e.g., first-order) rate distortion model and the resulting macroblock-level encoding complexities are used to generate an average encoding complexity for each region. These region complexities are then used to select quantizer values for each region in the current frame, e.g.

Abstract: A variety of different types of video frame encoders can be configured with, e.g., a multimedia processing subsystem, as long as the video frame encoder conforms to the interface protocol of the subsystem. A video controller in the subsystem performs the higher-level functions of coordinating the encoding of the video stream, thereby allowing the video frame encoder to limit its processing to the lower, frame level. In particular, the video controller provides information needed by the video frame encoder to encode the current frame in the video sequence. In addition to the raw image data, this information includes the type of frame to be encoded (e.g., an I or P frame), the currently available bandwidth for encoding the current frame, the time since the previous encoded frame, the desired frame rate, and a quality measure that may be used to trade off spatial and temporal qualities.

Abstract: Apparatus and method for classifying regions of an image, based on the relative “importance” of the various areas and to adaptively use the importance information to allocate processing resources and input image formation.

Abstract: A method and apparatus for encoding, illustratively, a video information stream to produce an encoded information stream according to a group of frames (GOF) information structure where the GOF structure and, optionally, a bit budget are modified in response to, respectively, information discontinuities and the presence of redundant information in the video information stream (due to, e.g., 3:2 pull-down processing).

Abstract: An apparatus and method for reducing memory resource requirements in, e.g., an image processing system by utilizing a packed data pixel representation and, optionally, M-ary pyramid decomposition, for pixel block or pixel group searching and matching operations.