Abstract: Automated layer extraction from 2D images making up a 3D scene, and automated image pixel assignment to layers, to provide for scene modeling, is disclosed. In one embodiment, a computer-implemented method determines a number of planes, or layers, and assigns pixels to the planes. The method can determine the number of planes by first determining the high-entropy pixels of the images, and then determining a 1-plane through a predetermined n-plane estimation, such as via a robust estimation, and a most likely x-plane estimation, where x is between 1 and n, such as via a Bayesian approach. Furthermore, the method can assign pixels via an iterative EM approach based on classifying criteria.

Abstract: The present invention is embodied in systems and methods for determining structure and motion of a three-dimensional (3D) object using two-dimensional (2D) images of the object obtained from multiple sets of views with different projection models, such as from a full perspective view and a weak perspective views. A novel fundamental matrix is derived that embodies the epipolar geometry between a full perspective view and a weak perspective view. The systems and methods of the present invention preferably uses the derived fundamental matrix together with the 2D image information of the full and weak perspective views to digitally reconstruct the 3D object and produce results with multi-resolution processing techniques. These techniques include recovering and refining motion parameters and recovering and refining structure parameters of the fundamental matrix. The results can include, for example, 3D positions of points, camera position between different views, texture maps, and the like.

Abstract: A system and method for extracting structure from stereo that represents the scene as a collection of planar layers. Each layer optimally has an explicit 3D plane equation, a colored image with per-pixel opacity, and a per-pixel depth value relative to the plane. Initial estimates of the layers are recovered using techniques from parametric motion estimation. The combination of a global model (the plane) with a local correction to it (the per-pixel relative depth value) imposes enough local consistency to allow the recovery of shape in both textured and untextured regions.

Abstract: A system and method for extracting structure from stereo that represents the scene as a collection of planar layers. Each layer optimally has an explicit 3D plane equation, a colored image with per-pixel opacity, and a per-pixel depth value relative to the plane. Initial estimates of the layers are made and then refined using a re-synthesis step which takes into account both occlusions and mixed pixels. Reasoning about these effects allows the recovery of depth and color information with high accuracy, even in partially occluded regions. Moreover, the combination of a global model (the plane) with a local correction to it (the per-pixel relative depth value) imposes enough local consistency to allow the recovery of shape in both textured and untextured regions.

Abstract: The invention is embodied in a process for synthesizing a new image representing a new viewpoint of a scene from at least two existing images of the scene taken from different respective viewspoints. The process begins by choosing a planar surface visible in the at least two of the existing images and transforming the at least two existing images relative to one another so as to bring the planar surface into perspective alignment in the at least two existing images, and then choosing a reference frame and computing parallax vectors between the two images of the projection of common scene points on the reference frame. Preferably, the reference frame comprises an image plane of a first one of the existing images. Preferably, the reference frame is co-planar with the planar surface.