Abstract: An object mask and a set of control points defined for a given frame are subjected to morphological processing to remove false edge points and provide a more robust mask for use in tracking the object in a next frame. The morphological processing is performed on a frame by frame basis corresponding to object tracking so that errors added in by the object tracker do not accumulate, and instead are filtered out. Rapidly moving objects which are troublesome for edge-based object trackers are more readily tracked. Also, regions of low contrast or regions locked onto in the background when trying to identify the object are more readily distinguished and eliminated from the object mask.

Abstract: Partitioned sigma instructions are provided in which processor capacity is effectively distributed among multiple sigma operations which are executed concurrently. Special registers are included for aligning data on memory word boundaries to reduce packing overhead in providing long data words for multimedia instructions which implement shifting data sequences over multiple iterations. Extended partitioned arithmetic instructions are provided to improve precision and avoid accumulated carry over errors. Partitioned formatting instructions, including partitioned interleave, partitioned compress, and partitioned interleave and compress pack subwords in an effective order for other partitioned operations.

Abstract: A video processing environment includes a user interface and processing shell from which various video processing ‘plug-in’ programs are accessed. The shell insulates the plug-ins from the intricacies of reading various file formats. The user interface allows an operator to load a video sequence, define and view one or more video objects on any one or more frames of the video sequence, edit existing video object segmentations, view video objects across a series of video frames, and encode video objects among a video sequence in a desired format. Various encoding parameters can be adjusted allowing the operator to view the video sequence encoded at the various parameter settings. The user interface includes a video window, a time-line window, a zoom window, a set of menus including a menu of plug-in programs, and a set of dialogue boxes, including encoding parameter dialogue boxes.

Abstract: A method for filtering an angular signal which avoids errors in averaging and differencing operations. Components of a hue signal are separated into groups based upon the sign of a corresponding filter coefficient ai for the respective component and the range of the hue component value vi. A weighted mean is computed for each of the groups. The weighted mean of the resulting values having positive filter coefficients and the weighted mean of the resulting values having negative filter coefficients then are derived. The computed value is redefined if needed to be in modulus 2&pgr;. The weighted distance of the two values then is derived. The result is the filtered hue value for a given image data pixel. A filtered hue value is obtained in the same manner for each image pixel of a desired image object or of a given image frame.

Abstract: Data are prefetched into a cache from a prefetch region of memory, based on a program instruction reference and on compile-time information that indicates the bounds of the prefetch region, a size of a prefetch block, and a location of the prefetch block. If the program reference address lies with the prefetch region, an offset distance is used to determine the address of the prefetch block. Prefetching is performed either from a continuous one-dimensional prefetch region, or an embedded multi-dimensional prefetch region. The prefetch block address is respectively determined in one dimension or multiple dimensions. Program-directed prefetching is implemented by a media processor or by a separate processing component in communication with the media processor. The primary components include a program-directed prefetch controller, a cache, a function unit, and a memory. Preferably, region registers store the compile-time information, and the prefetched data are stored in a cache prefetch buffer.

Abstract: A programmable graphics pipeline and method for processing multiple partitioned multimedia data, such as graphics data, image data, video data, or audio data. A preferred embodiment of the programmable graphics pipeline includes an instruction cache, a register file, and a vector functional unit that perform partitioned instructions. In addition, an enhanced rasterization unit is used to generate inverse-mapped source coordinates in addition to destination output coordinates for graphics and other media processing. An enhanced texture address unit generates corresponding memory addresses of source texture data for graphics processing and source media data for media processing. Data retrieved from memory are stored in an enhanced texture cache for use by the vector functional unit. A vector output unit includes a blending unit for graphics data and an output buffer for wide media data.

Abstract: A method for efficiently padding a macroblock of a video object plane employs two new instructions. The instructions, PadToRight and PadToLeft, are applied in alternating sequence during a PadPass 1 operation and a PadPass 2 operation. The results of these two operations are then averaged to pad each transparent pixel in each row of a macroblock that includes at least one opaque pixel. A Shift_in register is used to temporarily store data to facilitate the operation implemented by these instructions. Once the transparent pixels in each row have been padded horizontally, pixels in rows having shape data equal to zero (indicating all pixels in the row are transparent) are padded in a pre-processing step, followed by an upward propagation step. The two instructions are preferably implemented using 2:1 multiplexers implemented with an arithmetic logic unit. The method is particularly useful in set-top boxes, games, and other video applications.

Abstract: A template is analyzed to determine a step size for searching within a search area. First, the template is padded with pixels to increase its size. Cross-correlation between the padded template and the original template leads to identification of an effective step size. Step sizes for each of horizontal and vertical axes are derived. Windows of the search area, selected based on the step size, then are tested in a fast search stage by correlating the template to the window. Any tested window which has a correlation coefficient exceeding a specific value is a local match. A full search of the vicinity of the local match then is performed for all potential windows within an area bounded by one step to either side of the local match along either axis. The location(s) corresponding to the highest correlation(s) exceeding the threshold value are matches.

Abstract: A boundary macroblock of a video object is padded without significant synchronization overhead between a host processor and an existing coprocessor. The host processor determines horizontal and vertical graphics primitives as a function of shape data stored in a host memory. The shape data determine whether a dot, a line, or a rectangle primitive should be used to pad transparent pixels in the macroblock. The host processor communicates the primitives to a coprocessor, which renders the primitives in an interleaved pipeline fashion to pad transparent pixels of the macroblock based on texture data stored in video memory. The flow of primitives is in one direction from the host processor to the graphics coprocessor, and the texture data is not transferred back and forth between the host processor and coprocessor. This technique is especially useful for enabling acceleration of MPEG-4 video decoding utilizing existing coprocessors capable of accelerating MPEG-1/2 video decoding.

Abstract: A hierarchy of deformation operations is implemented to deform a template and match the deformed template to an object in a video frame. At each level, the constraints on the template deformations are relaxed, while the spatial range of the object boundary search is narrowed. At a highest level, an initial template is translated, rotated and scaled to coarsely locate the object within a given image frame. At a middle level, an affine transformation is implemented, globally or locally, to deform the template. For a local affine transformation process, a sup-portion, such as an articulation or appendage portion of the template is deformed. The middle level refines the template to get the template boundary close to the actual object boundary within a given frame. At the lowest level, a local segmentation algorithm is applied to deform the now close boundary to finely match the object boundary.

Abstract: A system for image object tracking and segmentation, includes (i) a modified adaptive resonance theory-2 (M-ART2) model for detecting changes of scenes, (ii) a two-dimensional correlative autopredicitve search (2D CAPS) method for object tracking, (iii) a derivation of contour energy for use in modelling an object edge, and (iv) an active contour model with global relaxation for defining optimal image object boundaries. The modelled object boundary is derived from an edge potential energy calculation and a set of current edge points. For each edge point, several candidate data points are also selected. Rather than calculate an energy value for each potential contours, a solution is achieved by comparing energy differences along a travel path. The selected contour is built in steps. An optimal path is selected from the potential paths.

Abstract: A video processing environment includes a user interface and processing shell from which various video processing ‘plug-in’ programs are accessed. The shell insulates the plug-ins from the intricacies of reading various file formats. The user interface allows an operator to load a video sequence, define and view one or more video objects on any one or more frames of the video sequence, edit existing video object segmentations, view video objects across a series of video frames, and encode video objects among a video sequence in a desired format. Various encoding parameters can be adjusted allowing the operator to view the video sequence encoded at the various parameter settings. The user interface includes a video window, a time-line window, a zoom window, a set of menus including a menu of plug-in programs, and a set of dialogue boxes, including encoding parameter dialogue boxes.

Abstract: Adaptive noise filtering is applied to an image frame of HSI data to reduce and more uniformly distribute noise while preserving image feature edges. An adaptive spatial filter includes a plurality of averaging kernels. An appropriate kernel is selected for each pixel for each of the hue and saturation components. A set of thresholds are defined for selecting the kernel for the hue component. Another set of thresholds are defined for selecting the kernel for the saturation component. The kernel for the saturation component is selected by comparing the intensity component to the saturation component thresholds. The kernel for the hue component is selected by comparing the product of intensity component and the saturation component to the hue component thresholds. A color gradient operation is applied to the filtered HSI data to aid in detecting image object boundaries. Object segmentation and other image processing techniques may be performed on the filtered HSI data.

Abstract: An object is tracked among a plurality of image frames. In an initial frame an operator selects an object. The object is distinguished from the remaining background portion of the image to yield a background and a foreground. A model of the background is used and updated in subsequent frames. A model of the foreground is used and updated in the subsequent frames. Pixels in subsequent frames are classified as belonging to the background or the foreground. In subsequent frames, decisions are made, including: which pixels do not belong to the background; which pixels in the foreground are to be updated; which pixels in the background were observed incorrectly in the current frame; and which background pixels are being observed for the first time. In addition, mask filtering is performed to correct errors, eliminate small islands and maintain spatial and temporal coherency of a foreground mask.

Abstract: A video processing environment includes a user interface and processing shell from which various video processing ‘plug-in’ programs are accessed. The shell insulates the plug-ins from the intricacies of reading various file formats. The user interface allows an operator to load a video sequence, define and view one or more video objects on any one or more frames of the video sequence, edit existing video object segmentations, view video objects across a series of video frames, and encode video objects among a video sequence in a desired format. Various encoding parameters can be adjusted allowing the operator to view the video sequence encoded at the various parameter settings. The user interface includes a video window, a time-line window, a zoom window, a set of menus including a menu of plug-in programs, and a set of dialogue boxes, including encoding parameter dialogue boxes.

Abstract: The optical flow of an array of pixels in an image field is determined using adaptive spatial and temporal gradients. Artifacts are avoided for image objects which are moving smoothly relative to the image field background. Data from three image frames are used to determine optical flow. A parameter is defined and determined frame by frame which is used to determine whether to consider the data looking forward from frame k to k+1 or the data looking backward from frame k−1 to frame k in initializing spatial and or temporal gradients for frame k. The parameter identifies signifies the areas of occlusion, so that the gradients looking backward from frame k−1 to frame k can be used for the occluded pixel regions. The gradients looking forward are used in the other areas.

Abstract: Multi-ported pipelined memory is located on a processor die serving as an addressable on-chip memory for efficiently processing streaming data. The memory sustains multiple wide memory accesses per cycle, clocks synchronously with the rest of the processor, and stores a significant portion of an image. Such memory bypasses the register file directly providing data to the processor's functional units. The memory includes multiple memory banks which permit multiple memory accesses per cycle. The memory banks are connected in pipelined fashion to pipeline registers placed at regular intervals on a global bus. The memory sustains multiple transactions per cycle, at a larger memory density than that of a multi-ported static memory, such as a register file.

Abstract: Control points used in deriving an object boundary for a prior frame are overlaid onto a current frame. An initial estimate of an object boundary are derived from the control points and edge energy data. The operator adjusts the control points to better model the boundary for the current frame. For each updated control point, the object boundary is rederived.

Abstract: A template is analyzed to determine a step size for searching within a search area. First, the template is padded with pixels to increase its size. Cross-correlation between the padded template and the original template leads to identification of an effective step size. Step sizes for each of horizontal and vertical axes are derived. Windows of the search area, selected based on the step size, then are tested in a fast search stage by correlating the template to the window. Any tested window which has a correlation coefficient exceeding a specific value is a local match. A full search of the vicinity of the local match then is performed for all potential windows within an area bounded by one step to either side of the local match along either axis. The location(s) corresponding to the highest correlation(s) exceeding the threshold value are matches.

Abstract: A template is analyzed to determine a step size for searching within a search area. First, the template is padded with pixels to increase its size. Cross-correlation between the padded template and the original template leads to identification of an effective step size. Step sizes for each of horizontal and vertical axes are derived. Windows of the search area, selected based on the step size, then are tested in a fast search stage by correlating the template to the window. Any tested window which has a correlation coefficient exceeding a specific value is a local match. A full search of the vicinity of the local match then is performed for all potential windows within an area bounded by one step to either side of the local match along either axis. The location(s) corresponding to the highest correlation(s) exceeding the threshold value are matches.