Abstract: A chromatic aberration (CA) correction technique is presented that substantially removes CA from an image captured by a digital camera. In general, the effects of any in-camera sharpening are reversed by applying a blurring kernel. The image is then super-sampled to approximate its state prior to the application of in-camera sampling. One of the color channels is designated as a reference channel, and an objective function is established for each of the non-reference channels. The reference color channel is assumed to be CA-free, while the objective functions are used to compute the unknown CA parameters for each non-reference channel. These sets are used in a CA removal function to substantially remove the CA associated with each of the non-reference channels. The image is then sampled to return it to its original resolution, and a sharpening filter is applied if needed to undo the effects of the previously applied blurring kernel.

Abstract: The stereo movie editing technique described herein combines knowledge of both multi-view stereo algorithms and human depth perception. The technique creates a digital editor, specifically for stereographic cinema. The technique employs an interface that allows intuitive manipulation of the different parameters in a stereo movie setup, such as camera locations and screen position. Using the technique it is possible to reduce or enhance well-known stereo movie effects such as cardboarding and miniaturization. The technique also provides new editing techniques such as directing the user's attention and easier transitions between scenes.

Abstract: Foreground object matting uses flash/no-flash images pairs to obtain a flash-only image. A trimap is obtained from the flash-only image. A joint Bayesian algorithm uses the flash-only image, the trimap and one of the image of the scene taken without the flash or the image of the scene taken with the flash to generate a high quality matte that can be used to extract the foreground from the background.

Abstract: Two-dimensional (2D) video is converted into multi-view video. The 2D video is segmented to generate a temporally consistent segmented 2D video which is made up of a sequence of segmented frames. The multi-view video is generated by employing user-guided operations to generate depth assignments for the segments associated with user-assigned regions of the segmented frames, where a user-assigned region is formed from a group of contiguous segments selected by the user.

Abstract: A Bayesian two-color image demosaicer and method for processing a digital color image to demosaic the image in such a way as to reduce image artifacts. The method and system are an improvement on and an enhancement to previous demosaicing techniques. A preliminary demosaicing pass is performed on the image to assign each pixel a fully specified RGB triple color value. The final color value of pixel in the processed image is restricted to be a linear combination of two colors. Fully-specified RGB triple color values for each pixel in an image used to find two clusters represented favored two colors. The amount of contribution from these favored two colors on the final color value then is determined. The method and system also can process multiple images to improve the demosaicing results. When using multiple images, sampling can be performed at a finer resolution, known as super resolution.

Abstract: Stereoscopic image display is described. In an embodiment, a location of the eye pupils of a viewer is determined and tracked. An image is displayed within a first focus for viewing with the left eye of the viewer, and the image is displayed within a second focus for viewing with the right eye of the viewer. A positional change of the eye pupils is tracked and a sequential image that corresponds to the positional change of the eye pupils is generated for stereoscopic viewing. In another embodiment, an image is displayed for stereoscopic viewing and a head position of a viewer relative to a center of the displayed image is determined. A positional change of the viewer's head is tracked, and a sequential image that corresponds to the positional change of the viewer's head is generated for stereoscopic viewing.

Abstract: A system and process for compressing and decompressing multiple video streams depicting substantially the same dynamic scene from different viewpoints that from a grid of viewpoints. Each frame in each contemporaneous set of video frames of the multiple streams is represented by at least a two layers—a main layer and a boundary layer. Compression of the main layers involves first designating one or more of these layers in each set of contemporaneous frames as keyframes. For each set of contemporaneous frames in time sequence order, the main layer of each keyframe is compressed using an inter-frame compression technique. In addition, the main layer of each non-keyframe within the frame set under consideration is compressed using a spatial prediction compression technique. Finally, the boundary layers of each frame in the current frame set are each compressed using an intra-frame compression technique. Decompression is generally the reverse of the compression process.

Abstract: Tilt is reduced or eliminated in captured digital images. Edges in a first image are detected. Angles corresponding to the detected edges are determined. A dominant angle is selected from the determined angles. The first image is rotated according to the selected dominant angle to generate a second image. The second image is a de-tilted version of the first image.

Abstract: Systems and methods for sketching reality are described. In one aspect, a set of vector primitives is identified from a 2-D sketch. In one implementation, the 2-D sketch is hand-drawn by a user. A 2.5D geometry model is automatically generated from the vector primitives. The 2.5D geometry model is automatically rendered and presented to a user. In one implementation, the user provides 2-D sketch-based user inputs to modify one or more of lighting position, lighting direction, lighting intensity, texture, color, and geometry of the presentation.

Abstract: Matte-based video restoration technique embodiments are presented which model spatio-temporally varying film wear artifacts found in digitized copies of film media. In general, this is accomplished by employing residual color information in recovering of artifact mattes. To this end, the distributions of artifact colors and their fractional contribution to each pixel of each frame being considered are extracted based on color information from the spatial and temporal neighborhoods of the pixel. The extracted information can then be used to restore the video by removing the artifacts.

Abstract: The present symmetric stereo matching technique provides a method for iteratively estimating a minimum energy for occlusion and disparity using belief propagation. The minimum energy is based on an energy minimization framework in which a visibility constraint is embedded. By embedding the visibility constraint, the present symmetric stereo matching technique treats both images equally, instead of treating one as a reference image. The visibility constraint ensures that occlusion in one view and the disparity in another view are consistent.

Abstract: Deinterlacing of video involves converting interlaced video to progressive video by interpolating a missing pixel in the interlaced video from other pixels in the video. A plurality of interpolants are provided, each of which interpolates a pixel value from other pixels that are nearby in space and/or time. The data costs of using the various interpolants is calculated. A particular one of the interpolants is chosen based on the data costs associated with the various interpolants. The chosen interpolant is used to interpolate the value of the missing pixel. The interpolated pixel value may be refined based on exemplars. The exemplars may be taken from the video that is being deinterlaced.

Abstract: An “Image Denoiser” provides a probabilistic process for denoising color images by segmenting an input image into regions, estimating statistics within each region, and then estimating a clean (or denoised) image using a probabilistic model of image formation. In one embodiment, estimated blur between each region is used to reduce artificial sharpening of region boundaries resulting from denoising the input image. In further embodiments, the estimated blur is used for additional purposes, including sharpening edges between one or more regions, and selectively blurring or sharpening one or more specific regions of the image (i.e., “selective focus”) while maintaining the original blurring between the various regions.

Abstract: An automatic purple fringing removal system and method for automatically eliminating purple-fringed regions from high-resolution images. The technique is based on the observations that purple-fringing regions often are adjacent near-saturated regions, and that purple-fringed regions are regions in which the blue and red color intensities are substantially greater than the green color intensity. The automatic purple fringing removal system and method implements these two observations by automatically detecting a purple-fringed region in an image and then automatically correcting the region. Automatic detection is achieved by finding near-saturated regions and candidate regions, and then defining a purple-fringed region as a candidate region adjacent a near-saturated region.

Abstract: A system and process for compressing and decompressing multiple video streams depicting substantially the same dynamic scene from different viewpoints. Each frame in each contemporaneous set of video frames of the multiple streams is represented by at least a two layers—a main layer and a boundary layer. Compression of the main layers involves first designating one or more of these layers in each set of contemporaneous frames as keyframes. For each set of contemporaneous frames in time sequence order, the main layer of each keyframe is compressed using an inter-frame compression technique. In addition, the main layer of each non-keyframe within the frame set under consideration is compressed using a spatial prediction compression technique. Finally, the boundary layers of each frame in the current frame set are each compressed using an intra-frame compression technique. Decompression is generally the reverse of the compression process.

Abstract: A method and a system for self-calibrating a wide field-of-view camera (such as a catadioptric camera) using a sequence of omni-directional images of a scene obtained from the camera. The present invention uses the consistency of pairwise features tracked across at least a portion of the image collection and uses these tracked features to determine unknown calibration parameters based on the characteristics of catadioptric imaging. More specifically, the self-calibration method of the present invention generates a sequence of omni-directional images representing a scene and tracks features across the image sequence. An objective function is defined in terms of the tracked features and an error metric (an image-based error metric in a preferred embodiment). The catadioptric imaging characteristics are defined by calibration parameters, and determination of optimal calibration parameters is accomplished by minimizing the objective function using an optimizing technique.

Abstract: A system and process for determining the vignetting function of an image and using the function to correct for the vignetting is presented. The image can be any arbitrary image and no other images are required. The system and process is designed to handle both textured and untextured segments in order to maximize the use of available information. To extract vignetting information from an image, segmentation techniques are employed that locate image segments with reliable data for vignetting estimation. Within each image segment, the system and process capitalizes on frequency characteristics and physical properties of vignetting to distinguish it from other sources of intensity variation. The vignetting data acquired from segments are weighted according to a presented reliability measure to promote robustness in estimation.

Abstract: A technique for estimating the optical flow between images of a scene and a segmentation of the images is presented. This involves first establishing an initial segmentation of the images and an initial optical flow estimate for each segment of each images and its neighboring image or images. A refined optical flow estimate is computed for each segment of each image from the initial segmentation of that image and the initial optical flow of the segments of that image. Next, the segmentation of each image is refined from the last-computed optical flow estimates for each segment of the image. This process can continue in an iterative manner by further refining the optical flow estimates for the images using their respective last-computed segmentation, followed by further refining the segmentation of each image using their respective last-computed optical flow estimates, until a prescribed number of iterations have been completed.

Abstract: In the described embodiment, methods and systems for processing facial image data for use in animation are described. In one embodiment, a system is provided that illuminates a face with illumination that is sufficient to enable the simultaneous capture of both structure data, e.g. a range or depth map, and reflectance properties, e.g. the diffuse reflectance of a subject's face. This captured information can then be used for various facial animation operations, among which are included expression recognition and expression transformation.

Abstract: Game data is rendered in three dimensions in the GPU of a game console. A left camera view and a right camera view are generated from a single camera view. The left and right camera positions are derived as an offset from a default camera. The focal distance of the left and right cameras is infinity. A game developer does not have to encode dual images into a specific hardware format. When a viewer sees the two slightly offset images, the user's brain combines the two offset images into a single 3D image to give the illusion that objects either pop out from or recede into the display screen. In another embodiment, individual, private video is rendered, on a single display screen, for different viewers. Rather than rendering two similar offset images, two completely different images are rendered allowing each player to view only one of the images.