Abstract: A method of generating a composite image includes capturing a video image of a physical scene with a camera, identifying a green-screen region within the video image, identifying a viewpoint and a position and/or orientation of the green-screen region relative to the viewpoint, and generating a modified video image rendered from the viewpoint onto a display surface in which the green-screen region is replaced with an image of a virtual object. The image of the virtual object is generated by projection rendering of a model of the virtual object based on the position and/or orientation of the green-screen region relative to the viewpoint such that the virtual object is constrained within the green-screen region.

Abstract: A stereoscopic display includes a headset having a housing shaped to fit on the front of a user's face, right and left displays secured to the housing so as to be positioned before the right and left eyes of the user when the headset is worn, right and left throw lenses positioned before the left and right displays to throw images of the right and left displays to a throw distance, and right and left prism lenses positioned before the right and left displays. The right and left lenses are shaped to shift images of the right and left displays as seen by the right and left eyes inwardly toward a midline such that the vergence of the left/right eyes converge towards the center at the throw distance of the thrown display plane.

Abstract: A system, method or compute program product for restricting convergence and divergence when viewing stereo 3D imagery. The technique includes receiving data representing a 3D virtual scene, receiving data indicating an eyepoint and a three-dimensional frustum rendering region with the three dimensional frustum rendering region identifying a position and orientation of a render plane, receiving data indicating a user stereo comfort region, rendering virtual objects in a first portion of the 3D virtual scene that is in the three-dimensional frustum rendering region and within the user stereo comfort region onto the render plane according to a first type of projection, and rendering virtual objects in a second portion of the 3D virtual scene that is in the three-dimensional frustum rendering region and outside the user stereo comfort region onto the render plane according to a different second type of projection.

Abstract: A method of generating a composite image includes capturing a video image of a physical scene with a camera, identifying a green-screen region within the video image, identifying a viewpoint and a position and/or orientation of the green-screen region relative to the viewpoint, and generating a modified video image rendered from the viewpoint onto a display surface in which the green-screen region is replaced with an image of a virtual object. The image of the virtual object is generated by projection rendering of a model of the virtual object based on the position and/or orientation of the green-screen region relative to the viewpoint such that the virtual object is constrained within the green-screen region.

Abstract: A system, method or compute program product for generating composite images. One of the systems includes a capture device to capture an image of a physical environment; and one or more storage devices storing instructions that are operable, when executed by one or more processors of the system, to cause the one or more processors to: obtain an image of the physical environment as captured by the capture device, identify a visually-demarked region on a surface in the physical environment as depicted in the image, process the image to generate a composite image of the physical environment that includes a depiction of a virtual object, wherein a location of the depiction of the virtual object in the composite image is based on a location of the depiction of the visually-demarked region in the image, and cause the composite image to be displayed for a user.

Abstract: A system, method or compute program product for integrating two or more displays. One of the systems includes a stereoscopic first display; a monoscopic second display; and one or more computers configured to perform operations including receiving first data representing a 3D scene including at least one virtual 3D object, receiving second data related to the at least one virtual 3D object, obtaining third data representing the position and/or orientation of the second display relative to position and/or orientation of the first display, based on the first data, rendering the 3D scene including the at least one virtual object as a stereoscopic image on the stereoscopic first display, and based on the second data, rendering a 2D object on the monoscopic second display with the rendering varying based on the position and orientation of the second display relative to the first display provided by the third data.

Abstract: A system, method or compute program product for displaying a first image based on first image data in a display area of a first display device, receiving at least one camera-captured second image of an environment with the second image capturing at least a portion of the first image displayed in the display area, determining a location and orientation of the first display device relative to the camera, determining a portion of the second image that corresponds to the portion of the first image displayed in the display area, generating a third image that corresponds to the portion of the first image displayed on the first display device as viewed from a point of view of the camera from the first image data, and generating a composite image of the environment by replacing at least a portion of the second image with the third image, and displaying the composite image.

Abstract: A system, method or compute program product for restricting convergence and divergence when viewing stereo 3D imagery. The technique includes receiving data representing a 3D virtual scene, receiving data indicating an eyepoint and a three-dimensional frustum rendering region with the three dimensional frustum rendering region identifying a position and orientation of a render plane, receiving data indicating a user stereo comfort region, rendering virtual objects in a first portion of the 3D virtual scene that is in the three-dimensional frustum rendering region and within the user stereo comfort region onto the render plane according to a first type of projection, and rendering virtual objects in a second portion of the 3D virtual scene that is in the three-dimensional frustum rendering region and outside the user stereo comfort region onto the render plane according to a different second type of projection.

Abstract: A method of generating a composite image includes capturing a video image of a physical scene with a camera, identifying a green-screen region within the video image, identifying a viewpoint and a position and/or orientation of the green-screen region relative to the viewpoint, and generating a modified video image rendered from the viewpoint onto a display surface in which the green-screen region is replaced with an image of a virtual object. The image of the virtual object is generated by projection rendering of a model of the virtual object based on the position and/or orientation of the green-screen region relative to the viewpoint such that the virtual object is constrained within the green-screen region.

Abstract: A system, method or compute program product for generating composite images. One of the systems includes a capture device to capture an image of a physical environment; and one or more storage devices storing instructions that are operable, when executed by one or more processors of the system, to cause the one or more processors to: obtain an image of the physical environment as captured by the capture device, identify a visually-demarked region on a surface in the physical environment as depicted in the image, process the image to generate a composite image of the physical environment that includes a depiction of a virtual object, wherein a location of the depiction of the virtual object in the composite image is based on a location of the depiction of the visually-demarked region in the image, and cause the composite image to be displayed for a user.

Abstract: A system, method or compute program product for restricting convergence and divergence when viewing stereo 3D imagery. The technique includes receiving data representing a 3D virtual scene, receiving data indicating an eyepoint and a three-dimensional frustum rendering region with the three dimensional frustum rendering region identifying a position and orientation of a render plane, receiving data indicating a user stereo comfort region, rendering virtual objects in a first portion of the 3D virtual scene that is in the three-dimensional frustum rendering region and within the user stereo comfort region onto the render plane according to a first type of projection, and rendering virtual objects in a second portion of the 3D virtual scene that is in the three-dimensional frustum rendering region and outside the user stereo comfort region onto the render plane according to a different second type of projection.

Abstract: A system, method or compute program product for restricting convergence and divergence when viewing stereo 3D imagery. The technique includes receiving data representing a 3D virtual scene, receiving data indicating an eyepoint and a three-dimensional frustrum rendering region with the three dimensional frustum rendering region identifying a position and orientation of a render plane, receiving data indicating a user stereo comfort region, rendering virtual objects in a first portion of the 3D virtual scene that is in the three-dimensional frustrum rendering region and within the user stereo comfort region onto the render plane according to a first type of projection, and rendering virtual objects in a second portion of the 3D virtual scene that is in the three-dimensional frustrum rendering region and outside the user stereo comfort region onto the render plane according to a different second type of projection.

Abstract: Embodiments of the present invention generally relate to interacting with a virtual scene at a perspective which is independent from the perspective of the user. Methods and systems can include either tracking and defining a perspective of the user based on the position and orientation of the user in the physical space, projecting a virtual scene for the user perspective to a virtual plane, tracking and defining a perspective of the a freehand user input device based on the position and orientation of the a freehand user input device, identifying a mark in the virtual scene which corresponds to the position and orientation of the device in the physical space, creating a virtual segment from the mark and interacting with virtual objects in the virtual scene at the end point of the virtual segment, as controlled using the device.

Abstract: Presenting a view based on a virtual viewpoint in a three dimensional (3D) scene. The 3D scene may be presented by at least one display, which includes displaying at least one stereoscopic image of the 3D scene by the display(s). The 3D scene may be presented according to a first viewpoint. A virtual viewpoint may be determined within the 3D scene that is different than the first viewpoint. The view of the 3D scene may be presented on the display(s) according to the virtual viewpoint and/or the first view point. The presentation of the view of the 3D scene is performed concurrently with presenting the 3D scene.

Abstract: System and methods for user interface elements for use within a 3D scene. The 3D scene may be presented by at least one display, which includes displaying at least one stereoscopic image of the 3D scene by the display(s). The 3D scene may be presented according to a first viewpoint. One or more user interface elements may be used. The 3D scene may be updated in response to the use of the user interface elements.

Abstract: Modifying perspective of stereoscopic images provided by one or more displays based on changes in user viewpoint. The one or more displays may include a first display that is provided substantially horizontal for displaying 3D horizontal perspective images and/or a second display that is provided substantially vertical for displaying text or conventional images such as 2D images, or 3D vertical perspective images. The horizontal display surface may be typically positioned directly in front of the user, and at a height of about a desktop surface so that the user can have about a 45° looking angle. The vertical display surface may be positioned in front of the user and preferably behind and above the horizontal display surface.

Abstract: Remote collaboration of a subject and a graphics object in a same view of a 3D scene. In one embodiment, one or more cameras of a collaboration system may be configured to capture images of a subject and track the subject (e.g., head of a user, other physical object). The images may be processed and provided to another collaboration system along with a determined viewpoint of the user. The other collaboration system may be configured to render and project the captured images and a graphics object in the same view of a 3D scene.

Abstract: Modifying perspective of stereoscopic images provided by one or more displays based on changes in user viewpoint. The one or more displays may include a first display that is provided substantially horizontal for displaying 3D horizontal perspective images and/or a second display that is provided substantially vertical for displaying text or conventional images such as 2D images, or 3D vertical perspective images. The horizontal display surface may be typically positioned directly in front of the user, and at a height of about a desktop surface so that the user can have about a 45° looking angle. The vertical display surface may be positioned in front of the user and preferably behind and above the horizontal display surface.

Abstract: System and methods for user interface elements for use within a 3D scene. The 3D scene may be presented by at least one display, which includes displaying at least one stereoscopic image of the 3D scene by the display(s). The 3D scene may be presented according to a first viewpoint. One or more user interface elements may be used. The 3D scene may be updated in response to the use of the user interface elements.

Abstract: Embodiments of the present invention generally relate to interacting with a virtual scene at a perspective which is independent from the perspective of the user. Methods and systems can include either tracking and defining a perspective of the user based on the position and orientation of the user in the physical space, projecting a virtual scene for the user perspective to a virtual plane, tracking and defining a perspective of the a freehand user input device based on the position and orientation of the a freehand user input device, identifying a mark in the virtual scene which corresponds to the position and orientation of the device in the physical space, creating a virtual segment from the mark and interacting with virtual objects in the virtual scene at the end point of the virtual segment, as controlled using the device.