Abstract: One embodiment in accordance with the invention is a video system that can include a display screen and a plurality of video capturing devices located behind the display screen. It is noted that each of the plurality of video capturing devices is for capturing a video stream through the display screen.

Abstract: A method for processing video using depth sensor information, comprising the steps of: dividing the image area into a number of bins roughly equal to the depth sensor resolution, with each bin corresponding to a number of adjacent image pixels; adding each depth measurement to the bin representing the portion of the image area to which the depth measurement corresponds; averaging the value of the depth measurement for each bin to determine a single average value for each bin; and applying a threshold to each bin of the registered depth map to produce a threshold image.

Abstract: The present invention provides a system and method for rendering a video conference. The method includes the steps of: creating a visual representation of a plurality of participants in the video conference, wherein the plurality of participants are divided into two or more subsets; and creating a compact version of the visual representation of at least one of the two or more subsets, wherein the choice of which subsets have a compact version of the visual representation created and displayed is based on the head position of a local participant.

Abstract: A method includes generating a first plurality of meshes configured to map a first domain associated with a display surface having a plurality of fiducial marks to a second domain associated with a image capture device configured to capture a first image of the display surface, generating a second plurality of meshes configured to map the second domain to a third domain associated with a first projector configured to display a second image onto the display surface, and generating a first mesh configured to map the first domain to the third domain in conjunction with a second mesh using the first plurality of meshes and the second plurality of meshes.

Abstract: A method for clustering data streams. Specifically, one embodiment of the present invention discloses a method for clustering data streams for use in a virtual environment. The embodiment of the method begins by determining a cluster of receiving nodes in the virtual environment. Each of the cluster of receiving nodes have associated values for at least one clustering parameter that as a set satisfies a test. Then, the embodiment continues by generating a common data stream based on the clustering parameter. Thereafter, the embodiment sends the common data stream from a sending node to the cluster of receiving nodes.

Abstract: A method of communicating gaze. Specifically, one embodiment of the present invention discloses a method of communicating gaze in an immersive virtual environment. The method begins by representing an immersive virtual environment on a monitor that is viewable by a viewing participant. The monitor displays a plurality of objects in the virtual environment. The physical gaze of the viewing participant is tracked to determine a physical direction of the physical gaze within a physical environment including the viewing participant. Thereafter, a viewed object is determined at which the viewing participant is gazing. Then, a virtual direction is determined between the viewing participant and the viewed object in the immersive virtual environment. A model of the viewing participant is rotated based on the physical and virtual directions to render a view of the viewing participant such that the viewing participant is facing the viewed object in the immersive virtual environment.

Abstract: A method and system for videoconferencing between parties at N sites. Specifically, a virtual space arrangement of N sites is created. Video sequences of N?1 remote sites are simultaneously displayed on a non-planar display arranged around a local party at a local site consistent with the virtual space arrangement. The displayed images of the remote parties are scaled consistently with their true physical sizes at their distances from the local party in the virtual space arrangement. Real-time video streams of the local party are captured from a plurality of viewpoints and are used to generate N?1 video view streams representing views of the local party from approximately the local display locations of the N?1 remote parties.

Abstract: Methods and systems for utilizing a parallel conversational thread during a remote collaboration are described. In one embodiment, a managing entity is utilized to establish a first communications path for a remote collaboration of at least two parties. Then, a request for a parallel conversation is received from at least one of the at least two parties during the remote collaboration. The managing entity then establishes a second communications path for the parallel conversation between the at least two parties. Wherein, the enabling of the parallel conversation on the second communications path is performed without inhibiting the first communications path, such that a portion of a first party can have a parallel conversation with a portion of a second party on the second communications path while the first communications path for the remote collaboration remains active.

Abstract: One embodiment in accordance with the invention is a video system that can include a display screen and a plurality of video capturing devices located behind the display screen. It is noted that each of the plurality of video capturing devices is for capturing a video stream through the display screen.

Abstract: A method of displaying images includes performing geometric processing and photometric processing on a plurality of image frames, thereby generating a plurality of processed image frames, projecting the plurality of processed image frames on a non-planar display surface with at least one projector, wherein the geometric processing includes applying a first plurality of meshes to the plurality of image frames, wherein the first plurality of meshes defines 2D mappings between the display surface and the at least one projector.

Abstract: A method includes generating a first plurality of meshes configured to map a first domain associated with a display surface having a plurality of fiducial marks to a second domain associated with a image capture device configured to capture a first image of the display surface, generating a second plurality of meshes configured to map the second domain to a third domain associated with a first projector configured to display a second image onto the display surface, and generating a first mesh configured to map the first domain to the third domain in conjunction with a second mesh using the first plurality of meshes and the second plurality of meshes.

Abstract: Disclosed is a method for producing a model of an object, comprising receiving at least one image of the object in a computing device, processing one or more of the at least one image to produce a model of the object, generating feedback information corresponding to the model of the object, and imparting the feedback information to a user of the computing device concurrently with the processing such that a determination is enabled to be made whether the model of the object is of a desired quality while the computing device is disposed to receive a further image of the object. The feedback information can be an image of the model which enables a user to determine whether an additional image is necessary or the feedback information can be direction to acquire an additional image of the object.

Abstract: A method of operating a data processing system to generate a three-dimensional model of a space from a plurality of measured images of the space. Each measured image includes a view of the space from a corresponding viewpoint. The method divides the space into a plurality of voxels, each voxel being characterized by a location and a color or an indication that the voxel is clear. The method defines a reconstruction of the space by assigning colors and clear values to a set of the voxels. The set is based on at least one linked list of voxels. The reconstruction is characterized by an error value related to the difference between each of the measured images and an image that would be produced by the set of voxels from the corresponding viewpoint as that of the measured image. The colors and clear values are chosen to reduce the error value below a cutoff value.

Abstract: A method of generating a three-dimensional model of a scene from a plurality of photographs of the scene. The method includes voxels in an interior region and an exterior region of the scene in question and then examines the voxels to determine if they are empty or part of the objects in the scene. The interior region is divided into a plurality of interior voxels, that are preferably of the same size. The exterior region surrounds the interior space and is divided into exterior voxels having different sizes. The size of the exterior voxels increases as a function of the distance between the voxel and the interior region. The voxels that are furthest from the interior region are preferably infinite in size.

Abstract: A method for real-time video communication. Specifically, one embodiment of the present invention discloses a method of video conferencing that captures a plurality of real-time video streams of a local participant from a plurality of sample viewpoints. From the plurality of video streams, a new view synthesis technique can be applied to generate a video image stream in real-time of the local participant rendered from a second location of a second participant with respect to a first location of the local participant in a coordinate space of a virtual environment. A change in either of the locations leads to the modifying of the video image stream, thereby enabling real-time video communication from the local participant to the second participant.

Abstract: Disclosed is a method for producing a model of an object, comprising receiving at least one image of the object in a computing device, processing one or more of the at least one image to produce a model of the object, generating feedback information corresponding to the model of the object, and imparting the feedback information to a user of the computing device concurrently with the processing such that a determination is enabled to be made whether the model of the object is of a desired quality while the computing device is disposed to receive a further image of the object. The feedback information can be an image of the model which enables a user to determine whether an additional image is necessary or the feedback information can be direction to acquire an additional image of the object.

Abstract: A method for real-time video communication. Specifically, one embodiment of the present invention discloses a method of video conferencing that captures a plurality of real-time video streams of a local participant from a plurality of sample viewpoints. From the plurality of video streams, a new view synthesis technique can be applied to generate a video image stream in real-time of the local participant rendered from a second location of a second participant with respect to a first location of the local participant in a coordinate space of a virtual environment. A change in either of the locations leads to the modifying of the video image stream, thereby enabling real-time video communication from the local participant to the second participant.

Abstract: A method of operating a data processing system to generate a three-dimensional model of a space from a plurality of measured images of the space. Each measured image includes a view of the space from a corresponding viewpoint. The method divides the space into a plurality of voxels, each voxel being characterized by a location and a color or an indication that the voxel is clear. The method defines a reconstruction of the space by assigning colors and clear values to a set of the voxels. The reconstruction is characterized by an error value related to the difference between each of the measured images and an image that would be produced by the set of voxels from the corresponding view point as that of the measured image. The colors and clear values are chosen to reduce the error value below a cutoff value.

Abstract: A method of generating a three-dimensional model of a scene from a plurality of photographs of the scene. The method includes voxels in an interior region and an exterior region of the scene in question and then examines the voxels to determine if they are empty or part of the objects in the scene. The interior region is divided into a plurality of interior voxels, that are preferably of the same size. The exterior region surrounds the interior space and is divided into exterior voxels having different sizes. The size of the exterior voxels increases as a function of the distance between the voxel and the interior region. The voxels that are furthest from the interior region are preferably infinite in size.

Abstract: A three-dimensional model of a scene is reconstructed from a plurality of images of the scene taken from different viewpoints. A set of voxels that include the scene is defined, and each of the voxels in the set of voxels is processed. Each voxel is projected onto a set of pixels in each of the images. A first color-standard-deviation of not-occluded pixels is determined across all images. A second color-standard-deviation of not-occluded pixels is determined for each individual image. A mean of the second color-standard-deviations is determined across all images. A coloring threshold that is a function of the mean is established. If the first color-standard-deviation is less than the coloring threshold, the voxel is colored. Otherwise, the voxel is left transparent. The coloring threshold is established adaptively and depends on the color variation across the projection of the voxel in each individual image.