Abstract: A method factorizing a sequence of images acquired of a scene into lighting components. The scene is illuminated by a moving light source. An appearance profile is constructed for each pixel in the sequence of images. The appearance profile is a vector representing intensities of the pixel at instances in time. The appearance profiles are factorized into a shadow component, a skylight component, and a sunlight component.

Abstract: A method and system acquire and display light fields. A continuous light field is reconstructed from input samples of an input light field of a 3D scene acquired by cameras according to an acquisition parameterization. The continuous light is reparameterized according to a display parameterization and then prefiltering and sampled to produce output samples having the display parametrization. The output samples are displayed as an output light field using a 3D display device.

Abstract: A method and system acquire and display light fields. A continuous light field is reconstructed from input samples of an input light field of a 3D scene acquired by cameras according to an acquisition parameterization. The continuous light is reparameterized according to a display parameterization and then prefiltering and sampled to produce output samples having the display parametrization. The output samples are displayed as an output light field using a 3D display device.

Abstract: A system encodes videos acquired of a moving object in a scene by multiple fixed cameras. Camera calibration data of each camera are first determined. The camera calibration data of each camera are associated with the corresponding video. A segmentation mask for each frame of each video is determined. The segmentation mask identifies only foreground pixels in the frame associated with the object. A shape encoder then encodes the segmentation masks, a position encoder encodes a position of each pixel, and a color encoder encodes a color of each pixel. The encoded data can be combined into a single bitstream and transferred to a decoder. At the decoder, the bitstream is decoded to an output video having an arbitrary user selected viewpoint. A dynamic 3D point model defines a geometry of the moving object. Splat sizes and surface normals used during the rendering can be explicitly determined by the encoder, or explicitly by the decoder.

Abstract: A face is scanned to obtain a three-dimensional geometry of the face, images are also acquired of the face, and subsurface scattering of the face is measured. A translucency map is determined from the subsurface reflectance. A total surface reflectance and a normal map are estimated from the three-dimensional geometry and the images, and diffuse reflectance is estimated using the total reflectance. An albedo map is determined from the diffuse reflectance. The diffuse reflectance is subtracted from the total reflectance to obtain a surface reflectance. A set of bi-directional reflectance functions is fitted to the surface reflectance. Then, the set of bi-directional reflectance distribution functions, the albedo map, and the translucency map are combined to form a skin reflectance model of the face.

Abstract: A method renders a volume data set including a plurality of voxels. A sampling rate for each voxel in a volume data set is determined. Each voxel is filtered according to the sampling rate and pixels for an output image are generated from the filtered voxels.

Abstract: The invention provides a system and method for modeling small three-dimensional facial features, such as wrinkles and pores. A scan of a face is acquired. A polygon mesh is constructed from the scan. The polygon mesh is reparameterized to determine a base mesh and a displacement image. The displacement image is partitioned into a plurality of tiles. Statistics for each tile are measured. The statistics is modified to deform the displacement image and the deformed displacement image is combined with the base mesh to synthesize a novel face.

Abstract: A method and system acquire and display light fields. A continuous light field is reconstructed from input samples of an input light field of a 3D scene acquired by cameras according to an acquisition parameterization. The continuous light is reparameterized according to a display parameterization and then prefiltering and sampled to produce output samples having the display parametrization. The output samples are displayed as an output light field using a 3D display device.

Abstract: A method renders a model of an object by first acquiring, in an acquisition space, a reflectance field of the object. The reflectance field includes a set of reflectance images of the object and a point model of the object. The model is deformed in an object space to generate a deformed model. For each point of the deformed model in the object space, the set of the reflectance images is queried in the acquisition space to obtain reflectance coefficients for each point. Each point of the deformed model is then shaded according to the corresponding reflectance coefficients to generate an image of the object reflecting the deforming.

Abstract: A method and system estimates a reflectance function of an arbitrary scene. The scene is illuminated under various lighting condition. For each lighting condition there is an associated illumination image and an observed image. Multiple, non-overlapping kernels are determined for each pixel in a reflectance image from the pairs of illumination and observed images. A weight is then determined for each kernel to estimate the reflectance function represented as the reflectance image.

Abstract: A method renders a 3D model of a graphics object wherein the model includes discrete zero-dimensional points. A first opaque polygon is centered on each point, and the polygon is rendered to obtain depth values of a depth image in a z-buffer. A second polygon is centered on each point. The second polygons are adapted to associated object space EWA resampling filters, and the adapted second polygons are rendered as an image according to the depth values in the depth image.

Abstract: A method reconstructs or synthesizes heads from 3D models of heads and 2D silhouettes of heads. A 3D statistical model is generated from multiple real human heads. The 3D statistical model includes a model parameter in the form of basis vectors and corresponding coefficients. Multiple 2D silhouettes of a particular head are acquired using a camera for example. The 3D statistical model is fitted to multiple 2D silhouettes to determine a particular value of the model parameter corresponding to the plurality of 2D silhouettes. Then, the 3D statistical model is rendered according to the particular value of the model parameter to reconstruct the particular head.

Abstract: A method renders a model of a 3D object. A polygon model of the 3D object is generated, and a set of input images are acquired of the object. For each vertex of the polygon model, a set of visible views are located, and a set of closest views are selected from the visibility views. Blending weights are determined for each closest view. Then, the each polygon in the model is rendered into an output image. The rendering is performed by blending images of the set of input images corresponding to the set of closest views of each vertex of the polygon using the blending weights.

Abstract: A method recognizes a face in an image. A morphable model having shape and pose parameters is fitted to a face in an image to construct a three-dimensional model of the face. Texture is extracted from the face in the image using the three-dimensional model. The shape and texture are projected into a bilinear illumination model to generate illumination bases for the face in the image. The illumination bases for the face in the image are compared to illumination bases of each of a plurality of bilinear illumination models of known faces to identify the face in the image.

Abstract: A method generates a three-dimensional, bi-linear, illumination model for arbitrary faces. A large number of images are acquired of many different faces. For each face, multiple images are acquired with varying poses and varying illumination. A three-mode singular value decomposition is applied to the images to determine parameters of the model. The model can be fit to a probe image of an unknown face. Then, the model can be compared with models of a gallery of images of unknown faces to recognize the face in the probe image.

Abstract: A method constructs three-dimensional (3D) models of a scene from a set of two-dimensional (2D) input images. The 3D model can then be used to reconstruct the scene from arbitrary viewpoints. A user segments and labels a set of corresponding polygonal regions in each image using conventional photo-editing tools. The invention constructs the model so that the model has a maximum volume that is consistent with the set of labeled regions in the input images. The method according to the invention directly constructs the polygonal model.

Abstract: A face is scanned to obtain a three-dimensional geometry of the face, images are also acquired of the face, and subsurface scattering of the face is measured. A translucency map is determined from the subsurface reflectance. A total surface reflectance and a normal map are estimated from the three-dimensional geometry and the images, and diffuse reflectance is estimated using the total reflectance. An albedo map is determined from the diffuse reflectance. The diffuse reflectance is subtracted from the total reflectance to obtain a surface reflectance. A set of bi-directional reflectance functions is fitted to the surface reflectance. Then, the set of bi-directional reflectance distribution functions, the albedo map, and the translucency map are combined to form a skin reflectance model of the face.

Abstract: A method and system extracts a matte from images acquired of a scene. A foreground image focused at a foreground in a scene, a background image focused at a background in the scene, and a pinhole image focused on the entire scene are acquired. These three images can be acquired sequentially by a single camera, or simultaneous by three cameras. In the later case, foreground, background and pinhole sequences of images can be acquired. The pinhole image is compared to the foreground image and the background image to extract a matte representing the scene. The comparison classifies pixels in the images as foreground, background, or unknown pixels. An optimizer minimizes an error function in the form of Fourier image equations using a gradient descent method. The error function expresses pixel intensity differences.

Abstract: A camera system acquires multiple optical characteristics at multiple resolutions of a scene. The camera system includes multiple optical elements arranged as a tree having a multiple of nodes connected by edges. the nodes represent optical elements sharing a single optical center, and the edges representing light paths between the nodes. The tree has the following structure: a single root node acquiring a plenoptic field originating from a scene; nodes with a single child node represent filters, lenses, apertures, and shutters; nodes with multiple child nodes represent beam splitters and leaf nodes represent imaging sensors. Furthermore, a length of the light paths from the roof node to each leaf nodes can be equal.

Abstract: A method renders a volume data set including a plurality of voxels. A sampling rate for each voxel in a volume data set is determined. Each voxel is filtered according to the sampling rate and pixels for an output image are generated from the filtered voxels.