Abstract: The system enables conversion of black and white images to color images and/or two-dimensional images into three-dimensional images based on adding color and/or depth to images using masks for regions in the images, as well as reshaping of masks to cover objects that have moved and changed shape as the objects move in a sequence of images. Also, includes motion picture project management system for reviewers, coordinators and artists. Artists utilize image analysis and image enhancement and computer graphics processing for example to convert two-dimensional images into three-dimensional images or otherwise create or alter motion pictures. Enables the efficient management of projects related to motion pictures to enable enterprises to manage assets, control costs, predict budgets and profit margins, reduce archival storage and otherwise provide displays tailored to specific roles to increase worker efficiency.

Abstract: Machine learning method that learns to convert 2D video to 3D video from a set of training examples. Uses machine learning to perform any or all of the 2D to 3D conversion steps of identifying and locating objects, masking objects, modeling object depth, generating stereoscopic image pairs, and filling gaps created by pixel displacement for depth effects. Training examples comprise inputs and outputs for the conversion steps. The machine learning system generates transformation functions that generate the outputs from the inputs; these functions may then be used on new 2D videos to automate or semi-automate the conversion process. Operator input may be used to augment the results of the machine learning system. Illustrative representations for conversion data in the training examples include object tags to identify objects and locate their features, Bézier curves to mask object regions, and point clouds or geometric shapes to model object depth.

Abstract: Machine learning method that learns to convert 2D video to 3D video from a set of training examples. Uses machine learning to perform any or all of the 2D to 3D conversion steps of identifying and locating objects, masking objects, modeling object depth, generating stereoscopic image pairs, and filling gaps created by pixel displacement for depth effects. Training examples comprise inputs and outputs for the conversion steps. The machine learning system generates transformation functions that generate the outputs from the inputs; these functions may then be used on new 2D videos to automate or semi-automate the conversion process. Operator input may be used to augment the results of the machine learning system. Illustrative representations for conversion data in the training examples include object tags to identify objects and locate their features, Bézier curves to mask object regions, and point clouds or geometric shapes to model object depth.

Abstract: Multi-stage production pipeline system that may be utilized in conjunction with a motion picture project management system. The multi-stage production pipeline system includes a computer and a database. The database includes metadata associated with at least one shot or associated with regions within the plurality of images in the at least one shot, or both. The computer includes a grouping tool interface for presenting user interface elements and accepting input of the metadata associated with the at least one shot or regions within the plurality of images in the at least one shot, or both. The system enables a large studio workforce to work non-linearly on a film while maintaining a unified vision driven by key creative figures, allowing for more consistent, higher quality, faster, less expensive work product and more efficient project management techniques. The system also enables reuse of project files, masks and other production elements across projects.

Abstract: Embodiments enable a three-dimensional annotation system and method that accepts desired depths for regions of input images and annotates two-dimensional/three-dimensional images with three-dimensional annotations for viewing at the desired depth(s) in any three-dimensional manner. Enables rapid and intuitive specification of desired depth and application of depth to regions in the two-dimensional images, or when editing three-dimensional images, as annotated by the three-dimensional annotations having at least one depth associated with the annotation. Enables rapid and intuitive depth augmentation or editing of an input image.

Abstract: Multi-stage production pipeline system that may be utilized in conjunction with a motion picture project management system. The multi-stage production pipeline system includes a computer and a database. The database includes metadata associated with at least one shot or associated with regions within the plurality of images in the at least one shot, or both. The computer includes a grouping tool interface for presenting user interface elements and accepting input of the metadata associated with the at least one shot or regions within the plurality of images in the at least one shot, or both. The system enables a large studio workforce to work non-linearly on a film while maintaining a unified vision driven by key creative figures, allowing for more consistent, higher quality, faster, less expensive work product and more efficient project management techniques. The system also enables reuse of project files, masks and other production elements across projects.

Abstract: Multi-stage production pipeline system that may be utilized in conjunction with a motion picture project management system. The multi-stage production pipeline system includes a computer and a database. The database includes metadata associated with at least one shot or associated with regions within the plurality of images in the at least one shot, or both. The computer includes a grouping tool interface for presenting user interface elements and accepting input of the metadata associated with the at least one shot or regions within the plurality of images in the at least one shot, or both. The system enables a large studio workforce to work non-linearly on a film while maintaining a unified vision driven by key creative figures, allowing for more consistent, higher quality, faster, less expensive work product and more efficient project management techniques. The system also enables reuse of project files, masks and other production elements across projects.

Abstract: Method for converting 2D video to 3D video using 3D object models. Embodiments of the invention obtain a 3D object model for one or more objects in a 2D video scene, such as a character. Object models may for example be derived from 3D scanner data; planes, polygons, or surfaces may be fit to this data to generate a 3D model. In each frame in which a modeled object appears, the location and orientation of the 3D model may be determined in the frame, and a depth map for the object may be generated from the model. 3D video may be generated using the depth map. Embodiments may use feature tracking to automatically determine object location and orientation. Embodiments may use rigged 3D models with degrees of freedom to model objects with parts that move relative to one another.

Abstract: A method that enables creation of a 3D virtual reality environment from a series of 2D images of a scene. Embodiments map 2D images onto a sphere to create a composite spherical image, divide the composite image into regions, and add depth information to the regions. Depth information may be generated by mapping regions onto flat or curved surfaces, and positioning these surfaces in 3D space. Some embodiments enable inserting, removing, or extending objects in the scene, adding or modifying depth information as needed. The final composite image and depth information are projected back onto one or more spheres, and then projected onto left and right eye image planes to form a 3D stereoscopic image for a viewer of the virtual reality environment. Embodiments enable 3D images to be generated dynamically for the viewer in response to changes in the viewer's position and orientation in the virtual reality environment.

Abstract: The system enables conversion of black and white images to color images and/or two-dimensional images into three-dimensional images based on adding color and/or depth to images using masks for regions in the images, as well as reshaping of masks to cover objects that have moved and changed shape as the objects move in a sequence of images. Also, includes motion picture project management system for reviewers, coordinators and artists. Artists utilize image analysis and image enhancement and computer graphics processing for example to convert two-dimensional images into three-dimensional images or otherwise create or alter motion pictures. Enables the efficient management of projects related to motion pictures to enable enterprises to manage assets, control costs, predict budgets and profit margins, reduce archival storage and otherwise provide displays tailored to specific roles to increase worker efficiency.

Abstract: Multi-stage production pipeline system that may be utilized in conjunction with a motion picture project management system. The multi-stage production pipeline system includes a computer and a database. The database includes metadata associated with at least one shot or associated with regions within the plurality of images in the at least one shot, or both. The computer includes a grouping tool interface for presenting user interface elements and accepting input of the metadata associated with the at least one shot or regions within the plurality of images in the at least one shot, or both. The system enables a large studio workforce to work non-linearly on a film while maintaining a unified vision driven by key creative figures, allowing for more consistent, higher quality, faster, less expensive work product and more efficient project management techniques. The system also enables reuse of project files, masks and other production elements across projects.

Abstract: Enables real-time depth modifications to stereo images of a 3D virtual reality environment locally and for example without iterative workflow involving region designers or depth artists that includes re-rendering these images from the original 3D model. Embodiments generate a spherical translation map from the 3D model of the virtual environment; this spherical translation map is a function of the pixel shifts between left and right stereo images for each point of the sphere surrounding the viewer of the virtual environment. Modifications may be made directly to the spherical translation map, and applied directly to the stereo images, without requiring re-rendering of the scene from the complete 3D model. This process enables depth modifications to be viewed in real-time, greatly improving the efficiency of the 3D model creation, review, and update cycle.

Abstract: The system enables conversion of black and white images to color images and/or two-dimensional images into three-dimensional images based on adding color and/or depth to images using masks for regions in the images, as well as reshaping of masks to cover objects that have moved and changed shape as the objects move in a sequence of images. Also, includes motion picture project management system for reviewers, coordinators and artists. Artists utilize image analysis and image enhancement and computer graphics processing for example to convert two-dimensional images into three-dimensional images or otherwise create or alter motion pictures. Enables the efficient management of projects related to motion pictures to enable enterprises to manage assets, control costs, predict budgets and profit margins, reduce archival storage and otherwise provide displays tailored to specific roles to increase worker efficiency.

Abstract: A system that coordinates multiple simultaneous reviews of a 3D model, potentially from different viewpoints. Embodiments support multiple reviewers using review stations that render images of the model based on the pose of the reviewer. For example, multiple reviewers may use virtual reality headsets to observe a 3D virtual environment from different orientations. A coordinator uses a coordinator station to observe the entire 3D model and the viewpoints of each of the reviewers in this 3D model. Embodiments may support reviewer designation of regions within the model that require review or modification; these designated regions are also displayed on the coordinator station. Embodiments may support real time updates to the 3D model and propagation of updated images to the coordinator and to the multiple viewers.

Abstract: Removes undesirable camera rotation from a panoramic video. Video frames are mapped to a sphere and then projected to a forward-looking planar video. Camera tracking or other techniques are used to derive camera position and orientation in each frame. From the camera orientation, a correcting rotation is derived and is applied to the sphere onto which the panoramic video was mapped. Re-projecting video from the rotation-corrected sphere reduces or eliminates camera rotation artifacts from the panoramic video. Rotation may be re-introduced and other visual effects may be added to the stabilized video. May be utilized in 2D virtual reality and augmented reality displays. Depth may be accepted by the system for objects in the video and utilized to create 3D stereoscopic virtual reality and augmented reality displays.

Abstract: Multi-stage production pipeline system that may be utilized in conjunction with a motion picture project management system. The multi-stage production pipeline system includes a computer and a database. The database includes metadata associated with at least one shot or associated with regions within the plurality of images in the at least one shot, or both. The computer includes a grouping tool interface for presenting user interface elements and accepting input of the metadata associated with the at least one shot or regions within the plurality of images in the at least one shot, or both. The system enables a large studio workforce to work non-linearly on a film while maintaining a unified vision driven by key creative figures, allowing for more consistent, higher quality, faster, less expensive work product and more efficient project management techniques. The system also enables reuse of project files, masks and other production elements across projects.

Abstract: External depth map transformation method for conversion of two-dimensional images to stereoscopic images that provides increased artistic and technical flexibility and rapid conversion of movies for stereoscopic viewing. Embodiments of the invention convert a large set of highly granular depth information inherent in a depth map associated with a two-dimensional image to a smaller set of rotated planes associated with masked areas in the image. This enables the planes to be manipulated independently or as part of a group and eliminates many problems associated with importing external depth maps including minimization of errors that frequently exist in external depth maps.

Abstract: Method for converting 2D video to 3D video using 3D object models. Embodiments of the invention obtain a 3D object model for one or more objects in a 2D video scene, such as a character. Object models may for example be derived from 3D scanner data; planes, polygons, or surfaces may be fit to this data to generate a 3D model. In each frame in which a modeled object appears, the location and orientation of the 3D model may be determined in the frame, and a depth map for the object may be generated from the model. 3D video may be generated using the depth map. Embodiments may use feature tracking to automatically determine object location and orientation. Embodiments may use rigged 3D models with degrees of freedom to model objects with parts that move relative to one another.

Abstract: A system that coordinates multiple simultaneous reviews of a 3D model, potentially from different viewpoints. Embodiments support multiple reviewers using review stations that render images of the model based on the pose of the reviewer. For example, multiple reviewers may use virtual reality headsets to observe a 3D virtual environment from different orientations. A coordinator uses a coordinator station to observe the entire 3D model and the viewpoints of each of the reviewers in this 3D model. Embodiments may support reviewer designation of regions within the model that require review or modification; these designated regions are also displayed on the coordinator station. Embodiments may support real time updates to the 3D model and propagation of updated images to the coordinator and to the multiple viewers.

Abstract: Enables real-time depth modifications to stereo images of a 3D virtual reality environment locally and for example without iterative workflow involving region designers or depth artists that includes re-rendering these images from the original 3D model. Embodiments generate a spherical translation map from the 3D model of the virtual environment; this spherical translation map is a function of the pixel shifts between left and right stereo images for each point of the sphere surrounding the viewer of the virtual environment. Modifications may be made directly to the spherical translation map, and applied directly to the stereo images, without requiring re-rendering of the scene from the complete 3D model. This process enables depth modifications to be viewed in real-time, greatly improving the efficiency of the 3D model creation, review, and update cycle.