Abstract: Object segmentation and tracking are improved by including directional information to guide the placement of an active contour (i.e., the elastic curve or ‘snake’) in estimating the object boundary. In estimating an object boundary the active contour deforms from an initial shape to adjust to image features using an energy minimizing function. The function is guided by external constraint forces and image forces to achieve a minimal total energy of the active contour. Both gradient strength and gradient direction of the image are analyzed in minimizing contour energy for an active contour model.

Abstract: A VLIW instruction format is introduced having a set of control bits which identify subinstruction sharing conditions. At compilation the VLIW instruction is analyzed to identify subinstruction sharing opportunities. Such opportunities are encoded in the control bits of the instruction. Before the instruction is moved into the instruction cache, the instruction is compressed into the new format to delete select redundant occurrences of a subinstruction. Specifically, where a subinstruction is to be shared by corresponding functional processing units of respective clusters, the subinstruction need only appear in the instruction once. The redundant appearance is deleted. The control bits are decoded at instruction parsing time to route a shared subinstruction to the associated functional processing units.

Abstract: A method for efficiently padding a macroblock of a video object plane employs two new instructions. The instructions, PadToRight and PadToLeft, are applied in alternating sequence during a PadPass 1 operation and a PadPass 2 operation. The results of these two operations are then averaged to pad each transparent pixel in each row of a macroblock that includes at least one opaque pixel. A Shift_in register is used to temporarily store data to facilitate the operation implemented by these instructions. Once the transparent pixels in each row have been padded horizontally, pixels in rows having shape data equal to zero (indicating all pixels in the row are transparent) are padded in a pre-processing step, followed by an upward propagation step. The two instructions are preferably implemented using 2:1 multiplexers implemented with an arithmetic logic unit. The method is particularly useful in set-top boxes, games, and other video applications.

Abstract: An object is tracked among a plurality of image frames. In an initial frame an operator selects an object. The object is distinguished from the remaining background portion of the image to yield a background and a foreground. A model of the background is used and updated in subsequent frames. A model of the foreground is used and updated in the subsequent frames. Pixels in subsequent frames are classified as belonging to the background or the foreground. In subsequent frames, decisions are made, including: which pixels do not belong to the background; which pixels in the foreground are to be updated; which pixels in the background were observed incorrectly in the current frame; and which background pixels are being observed for the first time. In addition, mask filtering is performed to correct errors, eliminate small islands and maintain spatial and temporal coherency of a foreground mask.

Abstract: A VLIW instruction format is introduced having a set of control bits which identify subinstruction sharing conditions. At compilation the VLIW instruction is analyzed to identify subinstruction sharing opportunities. Such opportunities are encoded in the control bits of the instruction. Before the instruction is moved into the instruction cache, the instruction is compressed into the new format to delete select redundant occurrences of a subinstruction. Specifically, where a subinstruction is to be shared by corresponding functional processing units of respective clusters, the subinstruction need only appear in the instruction once. The redundant appearance is deleted. The control bits are decoded at instruction parsing time to route a shared subinstruction to the associated functional processing units.

Abstract: A boundary macroblock of a video object is padded without significant synchronization overhead between a host processor and an existing coprocessor. The host processor determines horizontal and vertical graphics primitives as a function of shape data stored in a host memory. The shape data determine whether a dot, a line, or a rectangle primitive should be used to pad transparent pixels in the macroblock. The host processor communicates the primitives to a coprocessor, which renders the primitives in an interleaved pipeline fashion to pad transparent pixels of the macroblock based on texture data stored in video memory. The flow of primitives is in one direction from the host processor to the graphics coprocessor, and the texture data is not transferred back and forth between the host processor and coprocessor. This technique is especially useful for enabling acceleration of MPEG-4 video decoding utilizing existing coprocessors capable of accelerating MPEG-1/2 video decoding.

Abstract: A boundary macroblock of a video object is padded without significant synchronization overhead between a host processor and an existing coprocessor. The host processor determines horizontal and vertical graphics primitives as a function of shape data stored in a host memory. The shape data determine whether a dot, a line, or a rectangle primitive should be used to pad transparent pixels in the macroblock. The host processor communicates the primitives to a coprocessor, which renders the primitives in an interleaved pipeline fashion to pad transparent pixels of the macroblock based on texture data stored in video memory. The flow of primitives is in one direction from the host processor to the graphics coprocessor, and the texture data is not transferred back and forth between the host processor and coprocessor. This technique is especially useful for enabling acceleration of MPEG-4 video decoding utilizing existing coprocessors capable of accelerating MPEG-1/2 video decoding.

Abstract: Data are prefetched into a cache from a prefetch region of memory, based on a program instruction reference and on compile-time information that indicates the bounds of the prefetch region, a size of a prefetch block, and a location of the prefetch block. If the program reference address lies with the prefetch region, an offset distance is used to determine the address of the prefetch block. Prefetching is performed either from a continuous one-dimensional prefetch region, or an embedded multi-dimensional prefetch region. The prefetch block address is respectively determined in one dimension or multiple dimensions. Program-directed prefetching is implemented by a media processor or by a separate processing component in communication with the media processor. The primary components include a program-directed prefetch controller, a cache, a function unit, and a memory. Preferably, region registers store the compile-time information, and the prefetched data are stored in a cache prefetch buffer.

Abstract: A deep-pipeline system substantially reduces the overhead of setup delays and pipeline delays by dynamically controlling access of a plurality of configuration register sets by both a host central processing unit (CPU) and the stages of the pipelines. A master configuration register set is loaded with configuration parameters by the host CPU in response to an index count provided by a setup-index counter. A plurality of other counters are employed to track timing events in the system. In one embodiment, a run-index counter provides a run-index count to the first stage of the pipeline that is propagated along the stages, enabling configuration register sets to transfer configuration parameters to the stages of the pipeline when required to enable processing of a task. In an alternative embodiment, a plurality of D flip-flops sequentially propagates a state for successive registers, so that the setup-index counter is not required.

Abstract: A processor including a transposable register file. The register file allows normal row-wise access to data and also allows a transposed column-wise access to data stored in a column among registers of the register file. In transposed access mode, a data operand is accessed in a given partition of each of n registers. One register stores a first partition. An adjacent register stores the second partition, and so forth for each of n partitions of the operand. A queue-based transposable register file also is implemented. The queue-based transposable register file includes a head pointer and a tail pointer and has a virtual register. Data written into the virtual register is written into one of the registers as selected by the head pointer. Data read from the virtual register is read from one of the registers as selected by the tail pointer.

Abstract: The template data transfer coprocessor (TDTP) offloads block data transfer operations from a mediaprocessor. A uni-block template, program-guided template, an indirect template and queue-based template are described. The TDTP includes a template interpreter that employs an event-driven control mechanism to set up a template and compute block information and block information for each template. The programming involved in defining block data transfers for video and image processing algorithms is substantially reduced by the use of these templates.

Abstract: The register file of a processor includes embedded operand queues. The configuration of the register file into registers and operand queues is defined dynamically by a computer program. The programmer determines the trade-off between the number and size of the operand queue(s) versus the number of registers used for the program. The programmer partitions a portion of the registers into one or more operand queues. A given queue occupies a consecutive set of registers, although multiple queues need not occupy consecutive registers. An additional address bit is included to distinguish operand queue addresses from register addresses. Queue state logic tracks status information for each queue, including a header pointer, tail pointer, start address, end address and number of vacancies value. The program sets the locations and depth of a given operand queue within the register file.

Abstract: An area of on-chip memory is allocated to store one or more tables of commonly-used opcodes. The normal opcode in the instruction is replaced with a shorter code identifying an index into the table. As a result, the instruction is compressed. For a VLIW architecture, in which an instruction includes multiple subinstructions (multiple opcodes), the instruction loading bandwidth is substantially reduced. Preferably, an opcode table is dynamically loaded. Different tasks are programmed with a respective table of opcodes to be stored in the opcode table. The respective table is loaded when task switching.

Abstract: Intraoperative ultrasound (US) is integrated with stereotactic systems, where a system interactively registers two-dimensional (2D) US and three-dimensional (3D) magnetic resonance (MR) images. The registration is based on tracking a US probe with a bC magnetic position sensor. A transformation algorithm is performed to transform coordinates of points between two different spaces, where MR and US image spaces are independently registered with the position sensor space and where coordinate points can be registered between the MR and US spaces. A calibration procedure can be performed, and a phantom can be used to determine and analyze registration errors. The registered MR images can reconstructed using either zero-order or first-order interpolation.

Abstract: Image data for an image frame are allocated among three groups. In one group are image data which are part of a derived motion boundary, along with image data which differ by at least a threshold amount from a corresponding point among normalized background data. In another group are image data which closely correspond to the normalized background image data. A third group includes the remaining pixels. An initial estimate for the object boundary is adjusted iteratively based on the groupings, followed by application of an active contour model.

Abstract: Multi-ported pipelined memory is located on a processor die serving as an addressable on-chip memory for efficiently processing streaming data. The memory sustains multiple wide memory accesses per cycle, clocks synchronously with the rest of the processor, and stores a significant portion of an image. Such memory bypasses the register file directly providing data to the processor's functional units. The memory includes multiple memory banks which permit multiple memory accesses per cycle. The memory banks are connected in pipelined fashion to pipeline registers placed at regular intervals on a global bus. The memory sustains multiple transactions per cycle, at a larger memory density than that of a multi-ported static memory, such as a register file.

Abstract: A video processing environment includes a user interface and processing shell from which various video processing ‘plug-in’ programs are accessed. The shell insulates the plug-ins from the intricacies of reading various file formats. The user interface allows an operator to load a video sequence, define and view one or more video objects on any one or more frames of the video sequence, edit existing video object segmentations, view video objects across a series of video frames, and encode video objects among a video sequence in a desired format. Various encoding parameters can be adjusted allowing the operator to view the video sequence encoded at the various parameter settings. The user interface includes a video window, a time-line window, a zoom window, a set of menus including a menu of plug-in programs, and a set of dialogue boxes, including encoding parameter dialogue boxes.

Abstract: A coarsely estimated object boundary is formed by a sequence of edge boundary points. A local affine transformation along the estimated object boundary is identified by analyzing edge energy of a current image frame. A sequence of edge boundary points which have edge energy change ratios exceeding a threshold value is identified as a local affine transformation. A refined estimate of the object boundary is determined for the local affine transformation using a local segmentation process based on a key contour point search strategy. End points of the contour are assumed to be known. An initial key contour point is derived from the coarsely estimate object boundary. Candidate key points then are derived and corresponding curves analyzed to select a curve as the improved estimated boundary for the local affine transformation. The curve having a minimum average edge energy change ratio is selected.

Abstract: An object mask and a set of control points defined for a given frame are subjected to morphological processing to remove false edge points and provide a more robust mask for use in tracking the object in a next frame. The morphological processing is performed on a frame by frame basis corresponding to object tracking so that errors added in by the object tracker do not accumulate, and instead are filtered out. Rapidly moving objects which are troublesome for edge-based object trackers are more readily tracked. Also, regions of low contrast or regions locked onto in the background when trying to identify the object are more readily distinguished and eliminated from the object mask.

Abstract: Partitioned sigma instructions are provided in which processor capacity is effectively distributed among multiple sigma operations which are executed concurrently. Special registers are included for aligning data on memory word boundaries to reduce packing overhead in providing long data words for multimedia instructions which implement shifting data sequences over multiple iterations. Extended partitioned arithmetic instructions are provided to improve precision and avoid accumulated carry over errors. Partitioned formatting instructions, including partitioned interleave, partitioned compress, and partitioned interleave and compress pack subwords in an effective order for other partitioned operations.