Abstract: The system enables conversion of black and white images to color images and/or two-dimensional images into three-dimensional images based on adding color and/or depth to images using masks for regions in the images, as well as reshaping of masks to cover objects that have moved and changed shape as the objects move in a sequence of images. Also, includes motion picture project management system for reviewers, coordinators and artists. Artists utilize image analysis and image enhancement and computer graphics processing for example to convert two-dimensional images into three-dimensional images or otherwise create or alter motion pictures. Enables the efficient management of projects related to motion pictures to enable enterprises to manage assets, control costs, predict budgets and profit margins, reduce archival storage and otherwise provide displays tailored to specific roles to increase worker efficiency.

Abstract: Machine learning method that learns to convert 2D video to 3D video from a set of training examples. Uses machine learning to perform any or all of the 2D to 3D conversion steps of identifying and locating objects, masking objects, modeling object depth, generating stereoscopic image pairs, and filling gaps created by pixel displacement for depth effects. Training examples comprise inputs and outputs for the conversion steps. The machine learning system generates transformation functions that generate the outputs from the inputs; these functions may then be used on new 2D videos to automate or semi-automate the conversion process. Operator input may be used to augment the results of the machine learning system. Illustrative representations for conversion data in the training examples include object tags to identify objects and locate their features, Bézier curves to mask object regions, and point clouds or geometric shapes to model object depth.

Abstract: Machine learning method that learns to convert 2D video to 3D video from a set of training examples. Uses machine learning to perform any or all of the 2D to 3D conversion steps of identifying and locating objects, masking objects, modeling object depth, generating stereoscopic image pairs, and filling gaps created by pixel displacement for depth effects. Training examples comprise inputs and outputs for the conversion steps. The machine learning system generates transformation functions that generate the outputs from the inputs; these functions may then be used on new 2D videos to automate or semi-automate the conversion process. Operator input may be used to augment the results of the machine learning system. Illustrative representations for conversion data in the training examples include object tags to identify objects and locate their features, Bézier curves to mask object regions, and point clouds or geometric shapes to model object depth.

Abstract: Method for converting 2D video to 3D video using 3D object models. Embodiments of the invention obtain a 3D object model for one or more objects in a 2D video scene, such as a character. Object models may for example be derived from 3D scanner data; planes, polygons, or surfaces may be fit to this data to generate a 3D model. In each frame in which a modeled object appears, the location and orientation of the 3D model may be determined in the frame, and a depth map for the object may be generated from the model. 3D video may be generated using the depth map. Embodiments may use feature tracking to automatically determine object location and orientation. Embodiments may use rigged 3D models with degrees of freedom to model objects with parts that move relative to one another.

Abstract: A method that enables creation of a 3D virtual reality environment from a series of 2D images of a scene. Embodiments map 2D images onto a sphere to create a composite spherical image, divide the composite image into regions, and add depth information to the regions. Depth information may be generated by mapping regions onto flat or curved surfaces, and positioning these surfaces in 3D space. Some embodiments enable inserting, removing, or extending objects in the scene, adding or modifying depth information as needed. The final composite image and depth information are projected back onto one or more spheres, and then projected onto left and right eye image planes to form a 3D stereoscopic image for a viewer of the virtual reality environment. Embodiments enable 3D images to be generated dynamically for the viewer in response to changes in the viewer's position and orientation in the virtual reality environment.

Abstract: The system enables conversion of black and white images to color images and/or two-dimensional images into three-dimensional images based on adding color and/or depth to images using masks for regions in the images, as well as reshaping of masks to cover objects that have moved and changed shape as the objects move in a sequence of images. Also, includes motion picture project management system for reviewers, coordinators and artists. Artists utilize image analysis and image enhancement and computer graphics processing for example to convert two-dimensional images into three-dimensional images or otherwise create or alter motion pictures. Enables the efficient management of projects related to motion pictures to enable enterprises to manage assets, control costs, predict budgets and profit margins, reduce archival storage and otherwise provide displays tailored to specific roles to increase worker efficiency.

Abstract: The system enables conversion of black and white images to color images and/or two-dimensional images into three-dimensional images based on adding color and/or depth to images using masks for regions in the images, as well as reshaping of masks to cover objects that have moved and changed shape as the objects move in a sequence of images. Also, includes motion picture project management system for reviewers, coordinators and artists. Artists utilize image analysis and image enhancement and computer graphics processing for example to convert two-dimensional images into three-dimensional images or otherwise create or alter motion pictures. Enables the efficient management of projects related to motion pictures to enable enterprises to manage assets, control costs, predict budgets and profit margins, reduce archival storage and otherwise provide displays tailored to specific roles to increase worker efficiency.

Abstract: Enables real-time depth modifications to stereo images of a 3D virtual reality environment locally and for example without iterative workflow involving region designers or depth artists that includes re-rendering these images from the original 3D model. Embodiments generate a spherical translation map from the 3D model of the virtual environment; this spherical translation map is a function of the pixel shifts between left and right stereo images for each point of the sphere surrounding the viewer of the virtual environment. Modifications may be made directly to the spherical translation map, and applied directly to the stereo images, without requiring re-rendering of the scene from the complete 3D model. This process enables depth modifications to be viewed in real-time, greatly improving the efficiency of the 3D model creation, review, and update cycle.

Abstract: A system that coordinates multiple simultaneous reviews of a 3D model, potentially from different viewpoints. Embodiments support multiple reviewers using review stations that render images of the model based on the pose of the reviewer. For example, multiple reviewers may use virtual reality headsets to observe a 3D virtual environment from different orientations. A coordinator uses a coordinator station to observe the entire 3D model and the viewpoints of each of the reviewers in this 3D model. Embodiments may support reviewer designation of regions within the model that require review or modification; these designated regions are also displayed on the coordinator station. Embodiments may support real time updates to the 3D model and propagation of updated images to the coordinator and to the multiple viewers.

Abstract: Method for converting 2D video to 3D video using 3D object models. Embodiments of the invention obtain a 3D object model for one or more objects in a 2D video scene, such as a character. Object models may for example be derived from 3D scanner data; planes, polygons, or surfaces may be fit to this data to generate a 3D model. In each frame in which a modeled object appears, the location and orientation of the 3D model may be determined in the frame, and a depth map for the object may be generated from the model. 3D video may be generated using the depth map. Embodiments may use feature tracking to automatically determine object location and orientation. Embodiments may use rigged 3D models with degrees of freedom to model objects with parts that move relative to one another.

Abstract: Enables real-time depth modifications to stereo images of a 3D virtual reality environment locally and for example without iterative workflow involving region designers or depth artists that includes re-rendering these images from the original 3D model. Embodiments generate a spherical translation map from the 3D model of the virtual environment; this spherical translation map is a function of the pixel shifts between left and right stereo images for each point of the sphere surrounding the viewer of the virtual environment. Modifications may be made directly to the spherical translation map, and applied directly to the stereo images, without requiring re-rendering of the scene from the complete 3D model. This process enables depth modifications to be viewed in real-time, greatly improving the efficiency of the 3D model creation, review, and update cycle.

Abstract: A system that coordinates multiple simultaneous reviews of a 3D model, potentially from different viewpoints. Embodiments support multiple reviewers using review stations that render images of the model based on the pose of the reviewer. For example, multiple reviewers may use virtual reality headsets to observe a 3D virtual environment from different orientations. A coordinator uses a coordinator station to observe the entire 3D model and the viewpoints of each of the reviewers in this 3D model. Embodiments may support reviewer designation of regions within the model that require review or modification; these designated regions are also displayed on the coordinator station. Embodiments may support real time updates to the 3D model and propagation of updated images to the coordinator and to the multiple viewers.

Abstract: A method that enables creation of a 3D virtual reality environment from a series of 2D images of a scene. Embodiments map 2D images onto a sphere to create a composite spherical image, divide the composite image into regions, and add depth information to the regions. Depth information may be generated by mapping regions onto flat or curved surfaces, and positioning these surfaces in 3D space. Some embodiments enable inserting, removing, or extending objects in the scene, adding or modifying depth information as needed. The final composite image and depth information are projected back onto one or more spheres, and then projected onto left and right eye image planes to form a 3D stereoscopic image for a viewer of the virtual reality environment. Embodiments enable 3D images to be generated dynamically for the viewer in response to changes in the viewer's position and orientation in the virtual reality environment.

Abstract: The system enables conversion of black and white images to color images and/or two-dimensional images into three-dimensional images based on adding color and/or depth to images using masks for regions in the images, as well as reshaping of masks to cover objects that have moved and changed shape as the objects move in a sequence of images. Also, includes motion picture project management system for reviewers, coordinators and artists. Artists utilize image analysis and image enhancement and computer graphics processing for example to convert two-dimensional images into three-dimensional images or otherwise create or alter motion pictures. Enables the efficient management of projects related to motion pictures to enable enterprises to manage assets, control costs, predict budgets and profit margins, reduce archival storage and otherwise provide displays tailored to specific roles to increase worker efficiency.

Abstract: A line depth augmentation system and method for conversion of 2D images to 3D images. Enables adding depth to regions by altering depth of lines in the regions, for example in cell animation images or regions of limited color range. Eliminates creation of wireframe or other depth models and complex modeling of regions to match the depth of lines therein. Enables rapid conversion of two-dimensional images to three-dimensional images by enabling stereographers to quickly add/alter line depth without artifacts in images for example lines in monochrome regions. Embodiments may output a stereoscopic image pair of images with lines having desired depth, or any other three-dimensional viewing enabled image, such as an anaglyph image. Although the lines may be of a different depth than the region they appear in, the human mind interprets the monochromatic region as having depth associated with the line.

Abstract: Movies to be colorized/depth enhanced (2D?3D) are broken into backgrounds/sets or motion/onscreen-action. Background and motion elements are combined into composite frame which becomes a visual reference database that includes data for all frame offsets used later for the computer controlled application of masks within a sequence of frames. Masks are applied to subsequent frames of motion objects based on various differentiating image processing methods, including automated mask fitting/reshaping. Colors/depths are automatically applied with masks throughout a scene from the composite background and to motion objects. Areas never exposed by motion or foreground objects may be partially or fully realistically drawn/rendered/applied to the occluded areas and applied throughout the images to generate artifact-free secondary viewpoints during 2D?3D conversion.

Abstract: Motion picture scenes to be colorized/depth enhanced (2D?3D) are broken into separate elements, backgrounds/sets or motion/onscreen-action. Background and motion elements are combined into composite frame which becomes a visual reference database that includes data for all frame offsets used later for the computer controlled application of masks within a sequence of frames. Masks are applied to subsequent frames of motion objects based on various differentiating image processing methods, including automated mask fitting/reshaping. Colors and/or depths are automatically applied to masks throughout a scene from the composite background, translucent, motion objects.

Abstract: A line depth augmentation system and method for conversion of 2D images to 3D images. Enables adding depth to regions by altering depth of lines in the regions, for example in cell animation images or regions of limited color range. Eliminates creation of wireframe or other depth models and complex modeling of regions to match the depth of lines therein. Enables rapid conversion of two-dimensional images to three-dimensional images by enabling stereographers to quickly add/alter line depth without artifacts in images for example lines in monochrome regions. Embodiments may output a stereoscopic image pair of images with lines having desired depth, or any other three-dimensional viewing enabled image, such as an anaglyph image. Although the lines may be of a different depth than the region they appear in, the human mind interprets the monochromatic region as having depth associated with the line.

Abstract: Motion picture scenes to be colorized/depth enhanced (2D->3D) are broken into separate elements, backgrounds/sets or motion/onscreen-action. Background and motion elements are combined into composite frame which becomes a visual reference database that includes data for all frame offsets used later for the computer controlled application of masks within a sequence of frames. Masks are applied to subsequent frames of motion objects based on various differentiating image processing methods, including automated mask fitting/reshaping. Colors and/or depths are automatically applied to masks throughout a scene from the composite background and to motion objects.

Abstract: Motion picture scenes to be colorized/depth enhanced (2D->3D) are broken into separate elements, backgrounds/sets or motion/onscreen-action. Background and motion elements are combined into composite frame which becomes a visual reference database that includes data for all frame offsets used later for the computer controlled application of masks within a sequence of frames. Masks are applied to subsequent frames of motion objects based on various differentiating image processing methods, including automated mask fitting/reshaping. Colors and/or depths are automatically applied to masks throughout a scene from the composite background and to motion objects.