Patents Assigned to Legend3D, Inc.
  • Patent number: 9615082
    Abstract: The system enables conversion of black and white images to color images and/or two-dimensional images into three-dimensional images based on adding color and/or depth to images using masks for regions in the images, as well as reshaping of masks to cover objects that have moved and changed shape as the objects move in a sequence of images. Also, includes motion picture project management system for reviewers, coordinators and artists. Artists utilize image analysis and image enhancement and computer graphics processing for example to convert two-dimensional images into three-dimensional images or otherwise create or alter motion pictures. Enables the efficient management of projects related to motion pictures to enable enterprises to manage assets, control costs, predict budgets and profit margins, reduce archival storage and otherwise provide displays tailored to specific roles to increase worker efficiency.
    Type: Grant
    Filed: March 14, 2016
    Date of Patent: April 4, 2017
    Assignee: LEGEND3D, Inc.
    Inventors: Barry Sandrew, Anthony Lopez, Jared Sandrew, Tony Baldridge
  • Patent number: 9609307
    Abstract: Machine learning method that learns to convert 2D video to 3D video from a set of training examples. Uses machine learning to perform any or all of the 2D to 3D conversion steps of identifying and locating objects, masking objects, modeling object depth, generating stereoscopic image pairs, and filling gaps created by pixel displacement for depth effects. Training examples comprise inputs and outputs for the conversion steps. The machine learning system generates transformation functions that generate the outputs from the inputs; these functions may then be used on new 2D videos to automate or semi-automate the conversion process. Operator input may be used to augment the results of the machine learning system. Illustrative representations for conversion data in the training examples include object tags to identify objects and locate their features, Bézier curves to mask object regions, and point clouds or geometric shapes to model object depth.
    Type: Grant
    Filed: December 14, 2015
    Date of Patent: March 28, 2017
    Assignee: LEGEND3D, INC.
    Inventors: Anthony Lopez, Jacqueline McFarland, Tony Baldridge
  • Publication number: 20170085863
    Abstract: Machine learning method that learns to convert 2D video to 3D video from a set of training examples. Uses machine learning to perform any or all of the 2D to 3D conversion steps of identifying and locating objects, masking objects, modeling object depth, generating stereoscopic image pairs, and filling gaps created by pixel displacement for depth effects. Training examples comprise inputs and outputs for the conversion steps. The machine learning system generates transformation functions that generate the outputs from the inputs; these functions may then be used on new 2D videos to automate or semi-automate the conversion process. Operator input may be used to augment the results of the machine learning system. Illustrative representations for conversion data in the training examples include object tags to identify objects and locate their features, Bézier curves to mask object regions, and point clouds or geometric shapes to model object depth.
    Type: Application
    Filed: December 14, 2015
    Publication date: March 23, 2017
    Applicant: LEGEND3D, INC.
    Inventors: Anthony LOPEZ, Jacqueline McFARLAND, Tony BALDRIDGE
  • Patent number: 9595296
    Abstract: Multi-stage production pipeline system that may be utilized in conjunction with a motion picture project management system. The multi-stage production pipeline system includes a computer and a database. The database includes metadata associated with at least one shot or associated with regions within the plurality of images in the at least one shot, or both. The computer includes a grouping tool interface for presenting user interface elements and accepting input of the metadata associated with the at least one shot or regions within the plurality of images in the at least one shot, or both. The system enables a large studio workforce to work non-linearly on a film while maintaining a unified vision driven by key creative figures, allowing for more consistent, higher quality, faster, less expensive work product and more efficient project management techniques. The system also enables reuse of project files, masks and other production elements across projects.
    Type: Grant
    Filed: September 12, 2016
    Date of Patent: March 14, 2017
    Assignee: LEGEND3D, Inc.
    Inventors: Barry Sandrew, Jared Sandrew, Nancy Wang, Craig Cesareo, James Prola, Jill Hunt, Anthony Lopez
  • Patent number: 9547937
    Abstract: Embodiments enable a three-dimensional annotation system and method that accepts desired depths for regions of input images and annotates two-dimensional/three-dimensional images with three-dimensional annotations for viewing at the desired depth(s) in any three-dimensional manner. Enables rapid and intuitive specification of desired depth and application of depth to regions in the two-dimensional images, or when editing three-dimensional images, as annotated by the three-dimensional annotations having at least one depth associated with the annotation. Enables rapid and intuitive depth augmentation or editing of an input image.
    Type: Grant
    Filed: November 30, 2012
    Date of Patent: January 17, 2017
    Assignee: LEGEND3D, INC.
    Inventors: Jared Sandrew, Jill Hunt
  • Publication number: 20160379683
    Abstract: Multi-stage production pipeline system that may be utilized in conjunction with a motion picture project management system. The multi-stage production pipeline system includes a computer and a database. The database includes metadata associated with at least one shot or associated with regions within the plurality of images in the at least one shot, or both. The computer includes a grouping tool interface for presenting user interface elements and accepting input of the metadata associated with the at least one shot or regions within the plurality of images in the at least one shot, or both. The system enables a large studio workforce to work non-linearly on a film while maintaining a unified vision driven by key creative figures, allowing for more consistent, higher quality, faster, less expensive work product and more efficient project management techniques. The system also enables reuse of project files, masks and other production elements across projects.
    Type: Application
    Filed: September 12, 2016
    Publication date: December 29, 2016
    Applicant: LEGEND3D, Inc.
    Inventors: Barry SANDREW, Jared SANDREW, Nancy WANG, Craig CESAREO, James PROLA, Jill HUNT, Anthony LOPEZ
  • Patent number: 9443555
    Abstract: Multi-stage production pipeline system that may be utilized in conjunction with a motion picture project management system. The multi-stage production pipeline system includes a computer and a database. The database includes metadata associated with at least one shot or associated with regions within the plurality of images in the at least one shot, or both. The computer includes a grouping tool interface for presenting user interface elements and accepting input of the metadata associated with the at least one shot or regions within the plurality of images in the at least one shot, or both. The system enables a large studio workforce to work non-linearly on a film while maintaining a unified vision driven by key creative figures, allowing for more consistent, higher quality, faster, less expensive work product and more efficient project management techniques. The system also enables reuse of project files, masks and other production elements across projects.
    Type: Grant
    Filed: February 22, 2016
    Date of Patent: September 13, 2016
    Assignee: LEGEND3D, Inc.
    Inventors: Barry Sandrew, Jared Sandrew, Nancy Wang, Craig Cesareo, James Prola, Jill Hunt, Anthony Lopez
  • Patent number: 9438878
    Abstract: Method for converting 2D video to 3D video using 3D object models. Embodiments of the invention obtain a 3D object model for one or more objects in a 2D video scene, such as a character. Object models may for example be derived from 3D scanner data; planes, polygons, or surfaces may be fit to this data to generate a 3D model. In each frame in which a modeled object appears, the location and orientation of the 3D model may be determined in the frame, and a depth map for the object may be generated from the model. 3D video may be generated using the depth map. Embodiments may use feature tracking to automatically determine object location and orientation. Embodiments may use rigged 3D models with degrees of freedom to model objects with parts that move relative to one another.
    Type: Grant
    Filed: September 17, 2015
    Date of Patent: September 6, 2016
    Assignee: LEGEND3D, Inc.
    Inventors: Vicente Niebla, Jr., Tony Baldridge, Thomas Schad, Scott Jones
  • Patent number: 9407904
    Abstract: A method that enables creation of a 3D virtual reality environment from a series of 2D images of a scene. Embodiments map 2D images onto a sphere to create a composite spherical image, divide the composite image into regions, and add depth information to the regions. Depth information may be generated by mapping regions onto flat or curved surfaces, and positioning these surfaces in 3D space. Some embodiments enable inserting, removing, or extending objects in the scene, adding or modifying depth information as needed. The final composite image and depth information are projected back onto one or more spheres, and then projected onto left and right eye image planes to form a 3D stereoscopic image for a viewer of the virtual reality environment. Embodiments enable 3D images to be generated dynamically for the viewer in response to changes in the viewer's position and orientation in the virtual reality environment.
    Type: Grant
    Filed: May 11, 2015
    Date of Patent: August 2, 2016
    Assignee: LEGEND3D, INC.
    Inventors: Jared Sandrew, Tony Baldridge, Jacqueline McFarland, Scott Jones, Thomas Schad
  • Publication number: 20160198142
    Abstract: The system enables conversion of black and white images to color images and/or two-dimensional images into three-dimensional images based on adding color and/or depth to images using masks for regions in the images, as well as reshaping of masks to cover objects that have moved and changed shape as the objects move in a sequence of images. Also, includes motion picture project management system for reviewers, coordinators and artists. Artists utilize image analysis and image enhancement and computer graphics processing for example to convert two-dimensional images into three-dimensional images or otherwise create or alter motion pictures. Enables the efficient management of projects related to motion pictures to enable enterprises to manage assets, control costs, predict budgets and profit margins, reduce archival storage and otherwise provide displays tailored to specific roles to increase worker efficiency.
    Type: Application
    Filed: March 14, 2016
    Publication date: July 7, 2016
    Applicant: LEGEND3D, Inc.
    Inventors: Barry SANDREW, Anthony LOPEZ, Jared SANDREW, Tony BALDRIDGE
  • Publication number: 20160172002
    Abstract: Multi-stage production pipeline system that may be utilized in conjunction with a motion picture project management system. The multi-stage production pipeline system includes a computer and a database. The database includes metadata associated with at least one shot or associated with regions within the plurality of images in the at least one shot, or both. The computer includes a grouping tool interface for presenting user interface elements and accepting input of the metadata associated with the at least one shot or regions within the plurality of images in the at least one shot, or both. The system enables a large studio workforce to work non-linearly on a film while maintaining a unified vision driven by key creative figures, allowing for more consistent, higher quality, faster, less expensive work product and more efficient project management techniques. The system also enables reuse of project files, masks and other production elements across projects.
    Type: Application
    Filed: February 22, 2016
    Publication date: June 16, 2016
    Applicant: LEGEND3D, Inc.
    Inventors: Barry SANDREW, Jared SANDREW, Nancy WANG, Craig CESAREO, James PROLA, Jill HUNT, Anthony LOPEZ
  • Patent number: 9286941
    Abstract: The system enables conversion of black and white images to color images and/or two-dimensional images into three-dimensional images based on adding color and/or depth to images using masks for regions in the images, as well as reshaping of masks to cover objects that have moved and changed shape as the objects move in a sequence of images. Also, includes motion picture project management system for reviewers, coordinators and artists. Artists utilize image analysis and image enhancement and computer graphics processing for example to convert two-dimensional images into three-dimensional images or otherwise create or alter motion pictures. Enables the efficient management of projects related to motion pictures to enable enterprises to manage assets, control costs, predict budgets and profit margins, reduce archival storage and otherwise provide displays tailored to specific roles to increase worker efficiency.
    Type: Grant
    Filed: May 11, 2015
    Date of Patent: March 15, 2016
    Assignee: LEGEND3D, Inc.
    Inventors: Barry Sandrew, Tony Baldridge, Jared Sandrew, Anthony Lopez
  • Patent number: 9288476
    Abstract: Enables real-time depth modifications to stereo images of a 3D virtual reality environment locally and for example without iterative workflow involving region designers or depth artists that includes re-rendering these images from the original 3D model. Embodiments generate a spherical translation map from the 3D model of the virtual environment; this spherical translation map is a function of the pixel shifts between left and right stereo images for each point of the sphere surrounding the viewer of the virtual environment. Modifications may be made directly to the spherical translation map, and applied directly to the stereo images, without requiring re-rendering of the scene from the complete 3D model. This process enables depth modifications to be viewed in real-time, greatly improving the efficiency of the 3D model creation, review, and update cycle.
    Type: Grant
    Filed: August 17, 2015
    Date of Patent: March 15, 2016
    Assignee: LEGEND3D, INC.
    Inventors: Jared Sandrew, Tony Baldridge, Anthony Lopez, Jacqueline McFarland, Scott Jones, Thomas Schad
  • Patent number: 9282321
    Abstract: A system that coordinates multiple simultaneous reviews of a 3D model, potentially from different viewpoints. Embodiments support multiple reviewers using review stations that render images of the model based on the pose of the reviewer. For example, multiple reviewers may use virtual reality headsets to observe a 3D virtual environment from different orientations. A coordinator uses a coordinator station to observe the entire 3D model and the viewpoints of each of the reviewers in this 3D model. Embodiments may support reviewer designation of regions within the model that require review or modification; these designated regions are also displayed on the coordinator station. Embodiments may support real time updates to the 3D model and propagation of updated images to the coordinator and to the multiple viewers.
    Type: Grant
    Filed: August 17, 2015
    Date of Patent: March 8, 2016
    Assignee: LEGEND3D, INC.
    Inventors: Jared Sandrew, Tony Baldridge, Anthony Lopez, Jacqueline McFarland, Scott Jones, Thomas Schad
  • Patent number: 9277122
    Abstract: Removes undesirable camera rotation from a panoramic video. Video frames are mapped to a sphere and then projected to a forward-looking planar video. Camera tracking or other techniques are used to derive camera position and orientation in each frame. From the camera orientation, a correcting rotation is derived and is applied to the sphere onto which the panoramic video was mapped. Re-projecting video from the rotation-corrected sphere reduces or eliminates camera rotation artifacts from the panoramic video. Rotation may be re-introduced and other visual effects may be added to the stabilized video. May be utilized in 2D virtual reality and augmented reality displays. Depth may be accepted by the system for objects in the video and utilized to create 3D stereoscopic virtual reality and augmented reality displays.
    Type: Grant
    Filed: August 13, 2015
    Date of Patent: March 1, 2016
    Assignee: LEGEND3D, INC.
    Inventors: Ken Imura, Ryan Cummins
  • Patent number: 9270965
    Abstract: Multi-stage production pipeline system that may be utilized in conjunction with a motion picture project management system. The multi-stage production pipeline system includes a computer and a database. The database includes metadata associated with at least one shot or associated with regions within the plurality of images in the at least one shot, or both. The computer includes a grouping tool interface for presenting user interface elements and accepting input of the metadata associated with the at least one shot or regions within the plurality of images in the at least one shot, or both. The system enables a large studio workforce to work non-linearly on a film while maintaining a unified vision driven by key creative figures, allowing for more consistent, higher quality, faster, less expensive work product and more efficient project management techniques. The system also enables reuse of project files, masks and other production elements across projects.
    Type: Grant
    Filed: August 17, 2015
    Date of Patent: February 23, 2016
    Assignee: LEGEND 3D, INC.
    Inventors: Barry Sandrew, Jared Sandrew, Nancy Wang, Craig Cesareo, James Prola, Jill Hunt, Anthony Lopez
  • Patent number: 9241147
    Abstract: External depth map transformation method for conversion of two-dimensional images to stereoscopic images that provides increased artistic and technical flexibility and rapid conversion of movies for stereoscopic viewing. Embodiments of the invention convert a large set of highly granular depth information inherent in a depth map associated with a two-dimensional image to a smaller set of rotated planes associated with masked areas in the image. This enables the planes to be manipulated independently or as part of a group and eliminates many problems associated with importing external depth maps including minimization of errors that frequently exist in external depth maps.
    Type: Grant
    Filed: May 1, 2013
    Date of Patent: January 19, 2016
    Assignee: LEGEND3D, INC.
    Inventors: Matthew DeJohn, Thomas Schad, Scott Jones
  • Publication number: 20160005228
    Abstract: Method for converting 2D video to 3D video using 3D object models. Embodiments of the invention obtain a 3D object model for one or more objects in a 2D video scene, such as a character. Object models may for example be derived from 3D scanner data; planes, polygons, or surfaces may be fit to this data to generate a 3D model. In each frame in which a modeled object appears, the location and orientation of the 3D model may be determined in the frame, and a depth map for the object may be generated from the model. 3D video may be generated using the depth map. Embodiments may use feature tracking to automatically determine object location and orientation. Embodiments may use rigged 3D models with degrees of freedom to model objects with parts that move relative to one another.
    Type: Application
    Filed: September 17, 2015
    Publication date: January 7, 2016
    Applicant: LEGEND3D, Inc.
    Inventors: Vicente NIEBLA, JR., Tony BALDRIDGE, Thomas SCHAD, Scott JONES
  • Publication number: 20150358612
    Abstract: Enables real-time depth modifications to stereo images of a 3D virtual reality environment locally and for example without iterative workflow involving region designers or depth artists that includes re-rendering these images from the original 3D model. Embodiments generate a spherical translation map from the 3D model of the virtual environment; this spherical translation map is a function of the pixel shifts between left and right stereo images for each point of the sphere surrounding the viewer of the virtual environment. Modifications may be made directly to the spherical translation map, and applied directly to the stereo images, without requiring re-rendering of the scene from the complete 3D model. This process enables depth modifications to be viewed in real-time, greatly improving the efficiency of the 3D model creation, review, and update cycle.
    Type: Application
    Filed: August 17, 2015
    Publication date: December 10, 2015
    Applicant: LEGEND3D, INC.
    Inventors: Jared SANDREW, Tony BALDRIDGE, Anthony LOPEZ, Jacqueline MCFARLAND, Scott JONES, Thomas SCHAD
  • Publication number: 20150358613
    Abstract: A system that coordinates multiple simultaneous reviews of a 3D model, potentially from different viewpoints. Embodiments support multiple reviewers using review stations that render images of the model based on the pose of the reviewer. For example, multiple reviewers may use virtual reality headsets to observe a 3D virtual environment from different orientations. A coordinator uses a coordinator station to observe the entire 3D model and the viewpoints of each of the reviewers in this 3D model. Embodiments may support reviewer designation of regions within the model that require review or modification; these designated regions are also displayed on the coordinator station. Embodiments may support real time updates to the 3D model and propagation of updated images to the coordinator and to the multiple viewers.
    Type: Application
    Filed: August 17, 2015
    Publication date: December 10, 2015
    Applicant: LEGEND3D, INC.
    Inventors: Jared SANDREW, Tony BALDRIDGE, Anthony LOPEZ, Jacqueline MCFARLAND, Scott JONES, Thomas SCHAD