Patents by Inventor Matthew Loper
Matthew Loper has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11461630Abstract: Disclosed are systems and techniques for extracting user body shape (e.g., a representation of the three-dimensional body surface) from user behavioral data. The behavioral data may not be explicitly body-shape-related, and can include shopping history, social media likes, or other recorded behaviors of the user within (or outside of) a networked content delivery environment. The determined body shape can be used, for example, to generate a virtual fitting room user interface.Type: GrantFiled: March 6, 2018Date of Patent: October 4, 2022Assignee: Max-Planck-Gesellschaft zur Förderung der Wisenschaften e.V.Inventors: Michael Julian Black, Eric Rachlin, Matthew Loper, Jonathan Robert Cilley, William John O'Farrell, Alexander Weiss, Jason Lawrence Gelman, Steven Douglas Hatch, Nicolas Heron, Javier Romero Gonzalez-Nicolas
-
Patent number: 11049332Abstract: A method of transferring a facial expression from a subject to a computer generated character that includes: receiving a plate with an image of the subject's facial expression and an estimate of intrinsic parameters of a camera used to film the plate; generating a three-dimensional parameterized deformable model of the subject's face where different facial expressions of the subject can be obtained by varying values of the model parameters; solving for the facial expression in the plate by executing a deformation solver to solve for at least some parameters of the deformable model with a differentiable renderer and shape-from-shading techniques, using as inputs, the three-dimensional parameterized deformable model, estimated intrinsic camera parameters, estimated lighting conditions and albedo estimates over a series of iterations to infer geometry of the facial expression and generate an intermediate facial; generating, from the intermediate facial mesh, refined albedo estimates for the deformable model; andType: GrantFiled: March 3, 2020Date of Patent: June 29, 2021Assignee: LUCASFILM ENTERTAINMENT COMPANY LTD.Inventors: Matthew Loper, Stéphane Grabli, Kiran Bhat
-
Patent number: 11017577Abstract: The invention comprises a learned model of human body shape and pose dependent shape variation that is more accurate than previous models and is compatible with existing graphics pipelines. Our Skinned Multi-Person Linear model (SMPL) is a skinned vertex based model that accurately represents a wide variety of body shapes in natural human poses. The parameters of the model are learned from data including the rest pose template, blend weights, pose-dependent blend shapes, identity-dependent blend shapes, and a regressor from vertices to joint locations. Unlike previous models, the pose-dependent blend shapes are a linear function of the elements of the pose rotation matrices. This simple formulation enables training the entire model from a relatively large number of aligned 3D meshes of different people in different poses. The invention quantitatively evaluates variants of SMPL using linear or dual quaternion blend skinning and show that both are more accurate than a BlendSCAPE model trained on the same data.Type: GrantFiled: August 14, 2019Date of Patent: May 25, 2021Assignee: Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V.Inventors: Michael J. Black, Matthew Loper, Naureen Mahmood, Gerard Pons-Moll, Javier Romero
-
Publication number: 20200286301Abstract: A method of transferring a facial expression from a subject to a computer generated character that includes: receiving a plate with an image of the subject's facial expression and an estimate of intrinsic parameters of a camera used to film the plate; generating a three-dimensional parameterized deformable model of the subject's face where different facial expressions of the subject can be obtained by varying values of the model parameters; solving for the facial expression in the plate by executing a deformation solver to solve for at least some parameters of the deformable model with a differentiable renderer and shape-from-shading techniques, using as inputs, the three-dimensional parameterized deformable model, estimated intrinsic camera parameters, estimated lighting conditions and albedo estimates over a series of iterations to infer geometry of the facial expression and generate an intermediate facial; generating, from the intermediate facial mesh, refined albedo estimates for the deformable model; andType: ApplicationFiled: March 3, 2020Publication date: September 10, 2020Applicant: LUCASFILM ENTERTAINMENT COMPANY LTD.Inventors: Matthew Loper, Stéphane Grabli, Kiran Bhat
-
Patent number: 10755464Abstract: Present application refers to a method, a model generation unit and a computer program (product) for generating trained models (M) of moving persons, based on physically measured person scan data (S). The approach is based on a common template (T) for the respective person and on the measured person scan data (S) in different shapes and different poses. Scan data are measured with a 3D laser scanner. A generic personal model is used for co-registering a set of person scan data (S) aligning the template (T) to the set of person scans (S) while simultaneously training the generic personal model to become a trained person model (M) by constraining the generic person model to be scan-specific, person-specific and pose-specific and providing the trained model (M), based on the co-registering of the measured object scan data (S).Type: GrantFiled: February 16, 2018Date of Patent: August 25, 2020Assignee: Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V.Inventors: Michael Black, David L. Hirshberg, Matthew Loper, Eric Rachlin, Alex Weiss
-
Patent number: 10679046Abstract: Disclosed is a method including receiving an input image including a human, predicting, based on a convolutional neural network that is trained using examples consisting of pairs of sensor data, a corresponding body shape of the human and utilizing the corresponding body shape predicted from the convolutional neural network as input to another convolutional neural network to predict additional body shape metrics.Type: GrantFiled: November 29, 2017Date of Patent: June 9, 2020Assignee: Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V.Inventors: Michael Black, Eric Rachlin, Nicolas Heron, Matthew Loper, Alexander Weiss, Xiaochen Hu, Theodora Hinkle, Martin Kristiansen
-
Patent number: 10529137Abstract: Disclosed is a method including receiving visual input comprising a human within a scene, detecting a pose associated with the human using a trained machine learning model that detects human poses to yield a first output, estimating a shape (and optionally a motion) associated with the human using a trained machine learning model associated that detects shape (and optionally motion) to yield a second output, recognizing the scene associated with the visual input using a trained convolutional neural network which determines information about the human and other objects in the scene to yield a third output, and augmenting reality within the scene by leveraging one or more of the first output, the second output, and the third output to place 2D and/or 3D graphics in the scene.Type: GrantFiled: November 29, 2017Date of Patent: January 7, 2020Assignee: Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V.Inventors: Michael Black, Eric Rachlin, Evan Lee, Nicolas Heron, Matthew Loper, Alexander Weiss, David Smith
-
Publication number: 20190392626Abstract: The invention comprises a learned model of human body shape and pose dependent shape variation that is more accurate than previous models and is compatible with existing graphics pipelines. Our Skinned Multi-Person Linear model (SMPL) is a skinned vertex based model that accurately represents a wide variety of body shapes in natural human poses. The parameters of the model are learned from data including the rest pose template, blend weights, pose-dependent blend shapes, identity-dependent blend shapes, and a regressor from vertices to joint locations. Unlike previous models, the pose-dependent blend shapes are a linear function of the elements of the pose rotation matrices. This simple formulation enables training the entire model from a relatively large number of aligned 3D meshes of different people in different poses. The invention quantitatively evaluates variants of SMPL using linear or dual quaternion blend skinning and show that both are more accurate than a BlendSCAPE model trained on the same data.Type: ApplicationFiled: August 14, 2019Publication date: December 26, 2019Inventors: Michael J. Black, Matthew Loper, Naureen Mahmood, Gerard Pons-Moll, Javier Romero
-
Patent number: 10417818Abstract: A method for providing a three-dimensional body model which may be applied for an animation, based on a moving body, wherein the method comprises providing a parametric three-dimensional body model, which allows shape and pose variations; applying a standard set of body markers; optimizing the set of body markers by generating an additional set of body markers and applying the same for providing 3D coordinate marker signals for capturing shape and pose of the body and dynamics of soft tissue; and automatically providing an animation by processing the 3D coordinate marker signals in order to provide a personalized three-dimensional body model, based on estimated shape and an estimated pose of the body by means of predicted marker locations.Type: GrantFiled: June 19, 2017Date of Patent: September 17, 2019Assignee: Max-Planck-Gesellschaft zur Foerderung der Wissenschaften e.V.Inventors: Matthew Loper, Naureen Mahmood, Michael Black
-
Patent number: 10395411Abstract: The invention comprises a learned model of human body shape and pose dependent shape variation that is more accurate than previous models and is compatible with existing graphics pipelines. Our Skinned Multi-Person Linear model (SMPL) is a skinned vertex based model that accurately represents a wide variety of body shapes in natural human poses. The parameters of the model are learned from data including the rest pose template, blend weights, pose-dependent blend shapes, identity-dependent blend shapes, and a regressor from vertices to joint locations. Unlike previous models, the pose-dependent blend shapes are a linear function of the elements of the pose rotation matrices. This simple formulation enables training the entire model from a relatively large number of aligned 3D meshes of different people in different poses. The invention quantitatively evaluates variants of SMPL using linear or dual-quaternion blend skinning and show that both are more accurate than a Blend SCAPE model trained on the same data.Type: GrantFiled: June 23, 2016Date of Patent: August 27, 2019Assignee: Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V.Inventors: Michael J. Black, Matthew Loper, Naureen Mahmood, Gerard Pons-Moll, Javier Romero
-
Publication number: 20180315230Abstract: The invention comprises a learned model of human body shape and pose dependent shape variation that is more accurate than previous models and is compatible with existing graphics pipelines. Our Skinned Multi-Person Linear model (SMPL) is a skinned vertex based model that accurately represents a wide variety of body shapes in natural human poses. The parameters of the model are learned from data including the rest pose template, blend weights, pose-dependent blend shapes, identity-dependent blend shapes, and a regressor from vertices to joint locations. Unlike previous models, the pose-dependent blend shapes are a linear function of the elements of the pose rotation matrices. This simple formulation enables training the entire model from a relatively large number of aligned 3D meshes of different people in different poses. The invention quantitatively evaluates variants of SMPL using linear or dual-quaternion blend skinning and show that both are more accurate than a Blend SCAPE model trained on the same data.Type: ApplicationFiled: June 23, 2016Publication date: November 1, 2018Inventors: Michael J. BLACK, Matthew LOPER, Naureen MAHMOOD, Gerard PONS-MOLL, Javier ROMERO
-
Publication number: 20180247444Abstract: Present application refers to a method, a model generation unit and a computer program (product) for generating trained models (M) of moving persons, based on physically measured person scan data (S). The approach is based on a common template (T) for the respective person and on the measured person scan data (S) in different shapes and different poses. Scan data are measured with a 3D laser scanner. A generic personal model is used for co-registering a set of person scan data (S) aligning the template (T) to the set of person scans (S) while simultaneously training the generic personal model to become a trained person model (M) by constraining the generic person model to be scan-specific, person-specific and pose-specific and providing the trained model (M), based on the co-registering of the measured object scan data (S).Type: ApplicationFiled: February 16, 2018Publication date: August 30, 2018Inventors: Michael Black, David L. Hirshberg, Matthew Loper, Eric Rachlin, Alex Weiss
-
Patent number: 9898848Abstract: Present application refers to a method, a model generation unit and a computer program (product) for generating trained models (M) of moving persons, based on physically measured person scan data (S). The approach is based on a common template (T) for the respective person and on the measured person scan data (S) in different shapes and different poses. Scan data are measured with a 3D laser scanner. A generic personal model is used for co-registering a set of person scan data (S) aligning the template (T) to the set of person scans (S) while simultaneously training the generic personal model to become a trained person model (M) by constraining the generic person model to be scan-specific, person-specific and pose-specific and providing the trained model (M), based on the co registering of the measured object scan data (S).Type: GrantFiled: December 14, 2012Date of Patent: February 20, 2018Assignee: Max-Planck-Gesellschaft zur Foerderung der Wissenschaften e. V.Inventors: Michael Black, David L. Hirshberg, Matthew Loper, Eric Rachlin, Alex Weiss
-
Publication number: 20170287213Abstract: A method for providing a three-dimensional body model which may be applied for an animation, based on a moving body, wherein the method comprises providing a parametric three-dimensional body model, which allows shape and pose variations; applying a standard set of body markers; optimizing the set of body markers by generating an additional set of body markers and applying the same for providing 3D coordinate marker signals for capturing shape and pose of the body and dynamics of soft tissue; and automatically providing an animation by processing the 3D coordinate marker signals in order to provide a personalized three-dimensional body model, based on estimated shape and an estimated pose of the body by means of predicted marker locations.Type: ApplicationFiled: June 19, 2017Publication date: October 5, 2017Inventors: Matthew LOPER, Naureen MAHMOOD, Michael BLACK
-
Patent number: 9710964Abstract: A method for providing a three-dimensional body model which may be applied for an animation, based on a moving body, wherein the method comprises providing a parametric three-dimensional body model, which allows shape and pose variations; applying a standard set of body markers; optimizing the set of body markers by generating an additional set of body markers and applying the same for providing 3D coordinate marker signals for capturing shape and pose of the body and dynamics of soft tissue; and automatically providing an animation by processing the 3D coordinate marker signals in order to provide a personalized three-dimensional body model, based on estimated shape and an estimated pose of the body by means of predicted marker locations.Type: GrantFiled: January 22, 2015Date of Patent: July 18, 2017Assignee: Max-Planck-Gesellschaft zur Foerderung der Wissenschaften e.V.Inventors: Matthew Loper, Naureen Mahmood, Michael Black
-
Publication number: 20150262405Abstract: Present application refers to a method, a model generation unit and a computer program (product) for generating trained models (M) of moving persons, based on physically measured person scan data (S). The approach is based on a common template (T) for the respective person and on the measured person scan data (S) in different shapes and different poses. Scan data are measured with a 3D laser scanner. A generic personal model is used for co-registering a set of person scan data (S) aligning the template (T) to the set of person scans (S) while simultaneously training the generic personal model to become a trained person model (M) by constraining the generic person model to be scan-specific, person-specific and pose-specific and providing the trained model (M), based on the co registering of the measured object scan data (S).Type: ApplicationFiled: December 14, 2012Publication date: September 17, 2015Applicant: Max-Planck-Gesellschaft zur Foerderung der Wissenschaften e. V.Inventors: Michael Black, David L. Hirshberg, Matthew Loper, Eric Rachlin, Alex Weiss
-
Publication number: 20150206341Abstract: A method for providing a three-dimensional body model which may be applied for an animation, based on a moving body, wherein the method comprises providing a parametric three-dimensional body model, which allows shape and pose variations; applying a standard set of body markers; optimizing the set of body markers by generating an additional set of body markers and applying the same for providing 3D coordinate marker signals for capturing shape and pose of the body and dynamics of soft tissue; and automatically providing an animation by processing the 3D coordinate marker signals in order to provide a personalized three-dimensional body model, based on estimated shape and an estimated pose of the body by means of predicted marker locations.Type: ApplicationFiled: January 22, 2015Publication date: July 23, 2015Inventors: Matthew LOPER, Naureen MAHMOOD, Michael BLACK
-
Publication number: 20050285860Abstract: A method and system estimates a reflectance function of an arbitrary scene. The scene is illuminated under various lighting condition. For each lighting condition there is an associated illumination image and an observed image. Multiple, non-overlapping kernels are determined for each pixel in a reflectance image from the pairs of illumination and observed images. A weight is then determined for each kernel to estimate the reflectance function represented as the reflectance image.Type: ApplicationFiled: June 18, 2004Publication date: December 29, 2005Inventors: Hanspeter Pfister, Wojciech Matusik, Matthew Loper