Patents by Inventor Aaron Hertzmann
Aaron Hertzmann has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11941746Abstract: Embodiments are disclosed for computing accurate smooth occluding contours. In one embodiment, a method of computing accurate smooth occluding contours includes projecting a boundary polygon associated with a first region of a three-dimensional (3D) object to a two-dimensional (2D) image plane, the boundary polygon comprising a plurality of contour vertices and edges connecting the plurality of contour vertices, triangulating the first region in the 2D image plane to generate a 2D triangulation, and generating a 3D mesh for the first region by mapping the 2D triangulation to the 3D object.Type: GrantFiled: September 3, 2021Date of Patent: March 26, 2024Assignee: Adobe Inc.Inventors: Aaron Hertzmann, Shayan Hoshyari, Chenxi Liu
-
Patent number: 11880913Abstract: Techniques for generating a stylized drawing of three-dimensional (3D) shapes using neural networks are disclosed. A processing device generates a set of vector curve paths from a viewpoint of a 3D shape; extracts, using a first neural network of a plurality of neural networks of a machine learning model, surface geometry features of the 3D shape based on geometric properties of surface points of the 3D shape; determines, using a second neural network of the plurality of neural networks of the machine learning model, a set of at least one predicted stroke attribute based on the surface geometry features and a predetermined drawing style; generates, based on the at least one predicted stroke attribute, a set of vector stroke paths corresponding to the set of vector curve paths; and outputs a two-dimensional (2D) stylized stroke drawing of the 3D shape based at least on the set of vector stroke paths.Type: GrantFiled: October 27, 2021Date of Patent: January 23, 2024Assignees: Adobe Inc., University of MassachusettsInventors: Aaron Hertzmann, Matthew Fisher, Difan Liu, Evangelos Kalogerakis
-
Patent number: 11823391Abstract: The present disclosure includes methods and systems for identifying and manipulating a segment of a three-dimensional digital model based on soft classification of the three-dimensional digital model. In particular, one or more embodiments of the disclosed systems and methods identify a soft classification of a digital model and utilize the soft classification to tune segmentation algorithms. For example, the disclosed systems and methods can utilize a soft classification to select a segmentation algorithm from a plurality of segmentation algorithms, to combine segmentation parameters from a plurality of segmentation algorithms, and/or to identify input parameters for a segmentation algorithm. The disclosed systems and methods can utilize the tuned segmentation algorithms to accurately and efficiently identify a segment of a three-dimensional digital model.Type: GrantFiled: March 17, 2022Date of Patent: November 21, 2023Assignee: Adobe Inc.Inventors: Vladimir Kim, Aaron Hertzmann, Mehmet Yumer
-
Patent number: 11721056Abstract: In some embodiments, a model training system obtains a set of animation models. For each of the animation models, the model training system renders the animation model to generate a sequence of video frames containing a character using a set of rendering parameters and extracts joint points of the character from each frame of the sequence of video frames. The model training system further determines, for each frame of the sequence of video frames, whether a subset of the joint points are in contact with a ground plane in a three-dimensional space and generates contact labels for the subset of the joint points. The model training system trains a contact estimation model using training data containing the joint points extracted from the sequences of video frames and the generated contact labels. The contact estimation model can be used to refine a motion model for a character.Type: GrantFiled: January 12, 2022Date of Patent: August 8, 2023Assignee: Adobe Inc.Inventors: Jimei Yang, Davis Rempe, Bryan Russell, Aaron Hertzmann
-
Publication number: 20230109732Abstract: Techniques for generating a stylized drawing of three-dimensional (3D) shapes using neural networks are disclosed. A processing device generates a set of vector curve paths from a viewpoint of a 3D shape; extracts, using a first neural network of a plurality of neural networks of a machine learning model, surface geometry features of the 3D shape based on geometric properties of surface points of the 3D shape; determines, using a second neural network of the plurality of neural networks of the machine learning model, a set of at least one predicted stroke attribute based on the surface geometry features and a predetermined drawing style; generates, based on the at least one predicted stroke attribute, a set of vector stroke paths corresponding to the set of vector curve paths; and outputs a two-dimensional (2D) stylized stroke drawing of the 3D shape based at least on the set of vector stroke paths.Type: ApplicationFiled: October 27, 2021Publication date: April 13, 2023Inventors: Aaron Hertzmann, Matthew Fisher, Difan Liu, Evangelos Kalogerakis
-
Publication number: 20230074094Abstract: Embodiments are disclosed for computing accurate smooth occluding contours. In one embodiment, a method of computing accurate smooth occluding contours includes projecting a boundary polygon associated with a first region of a three-dimensional (3D) object to a two-dimensional (2D) image plane, the boundary polygon comprising a plurality of contour vertices and edges connecting the plurality of contour vertices, triangulating the first region in the 2D image plane to generate a 2D triangulation, and generating a 3D mesh for the first region by mapping the 2D triangulation to the 3D object.Type: ApplicationFiled: September 3, 2021Publication date: March 9, 2023Inventors: Aaron HERTZMANN, Shayan HOSHYARI, Chenxi LIU
-
Publication number: 20230037339Abstract: One example method involves a processing device that performs operations that include receiving a request to retarget a source motion into a target object. Operations further include providing the target object to a contact-aware motion retargeting neural network trained to retarget the source motion into the target object. The contact-aware motion retargeting neural network is trained by accessing training data that includes a source object performing the source motion. The contact-aware motion retargeting neural network generates retargeted motion for the target object, based on a self-contact having a pair of input vertices. The retargeted motion is subject to motion constraints that: (i) preserve a relative location of the self-contact and (ii) prevent self-penetration of the target object.Type: ApplicationFiled: July 26, 2021Publication date: February 9, 2023Inventors: Ruben Villegas, Jun Saito, Jimei Yang, Duygu Ceylan Aksit, Aaron Hertzmann
-
Publication number: 20220207749Abstract: The present disclosure includes methods and systems for identifying and manipulating a segment of a three-dimensional digital model based on soft classification of the three-dimensional digital model. In particular, one or more embodiments of the disclosed systems and methods identify a soft classification of a digital model and utilize the soft classification to tune segmentation algorithms. For example, the disclosed systems and methods can utilize a soft classification to select a segmentation algorithm from a plurality of segmentation algorithms, to combine segmentation parameters from a plurality of segmentation algorithms, and/or to identify input parameters for a segmentation algorithm. The disclosed systems and methods can utilize the tuned segmentation algorithms to accurately and efficiently identify a segment of a three-dimensional digital model.Type: ApplicationFiled: March 17, 2022Publication date: June 30, 2022Inventors: Vladimir Kim, Aaron Hertzmann, Mehmet Yumer
-
Publication number: 20220139019Abstract: In some embodiments, a model training system obtains a set of animation models. For each of the animation models, the model training system renders the animation model to generate a sequence of video frames containing a character using a set of rendering parameters and extracts joint points of the character from each frame of the sequence of video frames. The model training system further determines, for each frame of the sequence of video frames, whether a subset of the joint points are in contact with a ground plane in a three-dimensional space and generates contact labels for the subset of the joint points. The model training system trains a contact estimation model using training data containing the joint points extracted from the sequences of video frames and the generated contact labels. The contact estimation model can be used to refine a motion model for a character.Type: ApplicationFiled: January 12, 2022Publication date: May 5, 2022Inventors: Jimei Yang, Davis Rempe, Bryan Russell, Aaron Hertzmann
-
Patent number: 11315255Abstract: The present disclosure includes methods and systems for identifying and manipulating a segment of a three-dimensional digital model based on soft classification of the three-dimensional digital model. In particular, one or more embodiments of the disclosed systems and methods identify a soft classification of a digital model and utilize the soft classification to tune segmentation algorithms. For example, the disclosed systems and methods can utilize a soft classification to select a segmentation algorithm from a plurality of segmentation algorithms, to combine segmentation parameters from a plurality of segmentation algorithms, and/or to identify input parameters for a segmentation algorithm. The disclosed systems and methods can utilize the tuned segmentation algorithms to accurately and efficiently identify a segment of a three-dimensional digital model.Type: GrantFiled: June 22, 2020Date of Patent: April 26, 2022Assignee: Adobe Inc.Inventors: Vladimir Kim, Aaron Hertzmann, Mehmet Yumer
-
Patent number: 11238634Abstract: In some embodiments, a motion model refinement system receives an input video depicting a human character and an initial motion model describing motions of individual joint points of the human character in a three-dimensional space. The motion model refinement system identifies foot joint points of the human character that are in contact with a ground plane using a trained contact estimation model. The motion model refinement system determines the ground plane based on the foot joint points and the initial motion model and constructs an optimization problem for refining the initial motion model. The optimization problem minimizes the difference between the refined motion model and the initial motion model under a set of plausibility constraints including constraints on the contact foot joint points and a time-dependent inertia tensor-based constraint. The motion model refinement system obtains the refined motion model by solving the optimization problem.Type: GrantFiled: April 28, 2020Date of Patent: February 1, 2022Assignee: Adobe Inc.Inventors: Jimei Yang, Davis Rempe, Bryan Russell, Aaron Hertzmann
-
Patent number: 11216170Abstract: The present disclosure is directed toward systems and methods that enable simultaneous viewing and editing of audio-visual content within a virtual-reality environment (i.e., while wearing a virtual-reality device). For example, the virtual-reality editing system allows for editing of audio-visual content while viewing the audio-visual content via a virtual-reality device. In particular, the virtual-reality editing system provides an editing interface over a display of audio-visual content provided via a virtual-reality device (e.g., a virtual-reality headset) that allows for editing of the audio-visual content.Type: GrantFiled: July 31, 2020Date of Patent: January 4, 2022Assignee: ADOBE INC.Inventors: Stephen DiVerdi, Aaron Hertzmann, Cuong Nguyen
-
Patent number: 11189066Abstract: Embodiments disclosed herein describe systems, methods, and products that train one or more neural networks and execute the trained neural network across various applications. The one or more neural networks are trained to optimize a loss function comprising a pixel-level comparison between the outputs generated by the neural networks and the ground truth dataset generated from a bubble view methodology or an explicit importance maps methodology. Each of these methodologies may be more efficient than and may closely approximate the more expensive but accurate human eye gaze measurements. The embodiments herein leverage an existing process for training neural networks to generate importance maps of a plurality of graphic objects to offer interactive applications for graphics designs and data visualizations.Type: GrantFiled: November 13, 2018Date of Patent: November 30, 2021Assignee: Adobe Inc.Inventors: Zoya Bylinskii, Aaron Hertzmann, Bryan Russell
-
Publication number: 20210335028Abstract: In some embodiments, a motion model refinement system receives an input video depicting a human character and an initial motion model describing motions of individual joint points of the human character in a three-dimensional space. The motion model refinement system identifies foot joint points of the human character that are in contact with a ground plane using a trained contact estimation model. The motion model refinement system determines the ground plane based on the foot joint points and the initial motion model and constructs an optimization problem for refining the initial motion model. The optimization problem minimizes the difference between the refined motion model and the initial motion model under a set of plausibility constraints including constraints on the contact foot joint points and a time-dependent inertia tensor-based constraint. The motion model refinement system obtains the refined motion model by solving the optimization problem.Type: ApplicationFiled: April 28, 2020Publication date: October 28, 2021Inventors: Jimei Yang, Davis Rempe, Bryan Russell, Aaron Hertzmann
-
Patent number: 10957063Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating modified video content to reduce depth conflicts between user interface elements and video objects. For example, the disclosed systems can analyze an input video to identify feature points that designate objects within the input video and to determine the depths of the identified feature points. In addition, the disclosed systems can compare the depths of the feature points with a depth of a user interface element to determine whether there are any depth conflicts. In response to detecting a depth conflict, the disclosed systems can modify the depth of the user interface element to reduce or avoid the depth conflict. Furthermore, the disclosed systems can apply a blurring effect to an area around a user interface element to reduce the effect of depth conflicts.Type: GrantFiled: March 26, 2018Date of Patent: March 23, 2021Assignee: ADOBE INC.Inventors: Stephen DiVerdi, Cuong Nguyen, Aaron Hertzmann, Feng Liu
-
Publication number: 20200363940Abstract: The present disclosure is directed toward systems and methods that enable simultaneous viewing and editing of audio-visual content within a virtual-reality environment (i.e., while wearing a virtual-reality device). For example, the virtual-reality editing system allows for editing of audio-visual content while viewing the audio-visual content via a virtual-reality device. In particular, the virtual-reality editing system provides an editing interface over a display of audio-visual content provided via a virtual-reality device (e.g., a virtual-reality headset) that allows for editing of the audio-visual content.Type: ApplicationFiled: July 31, 2020Publication date: November 19, 2020Inventors: Stephen DiVerdi, Aaron Hertzmann, Cuong Nguyen
-
Publication number: 20200320715Abstract: The present disclosure includes methods and systems for identifying and manipulating a segment of a three-dimensional digital model based on soft classification of the three-dimensional digital model. In particular, one or more embodiments of the disclosed systems and methods identify a soft classification of a digital model and utilize the soft classification to tune segmentation algorithms. For example, the disclosed systems and methods can utilize a soft classification to select a segmentation algorithm from a plurality of segmentation algorithms, to combine segmentation parameters from a plurality of segmentation algorithms, and/or to identify input parameters for a segmentation algorithm. The disclosed systems and methods can utilize the tuned segmentation algorithms to accurately and efficiently identify a segment of a three-dimensional digital model.Type: ApplicationFiled: June 22, 2020Publication date: October 8, 2020Inventors: Vladimir Kim, Aaron Hertzmann, Mehmet Yumer
-
Patent number: 10754529Abstract: The present disclosure is directed toward systems and methods that enable simultaneous viewing and editing of audio-visual content within a virtual-reality environment (i.e., while wearing a virtual-reality device). For example, the virtual-reality editing system allows for editing of audio-visual content while viewing the audio-visual content via a virtual-reality device. In particular, the virtual-reality editing system provides an editing interface over a display of audio-visual content provided via a virtual-reality device (e.g., a virtual-reality headset) that allows for editing of the audio-visual content.Type: GrantFiled: October 28, 2016Date of Patent: August 25, 2020Assignee: ADOBE INC.Inventors: Stephen DiVerdi, Aaron Hertzmann, Cuong Nguyen
-
Patent number: 10748324Abstract: This disclosure relates to methods, non-transitory computer readable media, and systems that integrate (or embed) a non-photorealistic rendering (“NPR”) generator with a style-transfer-neural network to generate stylized images that both correspond to a source image and resemble a stroke style. By integrating an NPR generator with a style-transfer-neural network, the disclosed methods, non-transitory computer readable media, and systems can accurately capture a stroke style resembling one or both of stylized edges or stylized shadings. When training such a style-transfer-neural network, the integrated NPR generator can enable the disclosed methods, non-transitory computer readable media, and systems to use real-stroke drawings (instead of conventional paired-ground-truth drawings) for training the network to accurately portray a stroke style.Type: GrantFiled: November 8, 2018Date of Patent: August 18, 2020Assignee: ADOBE INC.Inventors: Elya Shechtman, Yijun Li, Chen Fang, Aaron Hertzmann
-
Patent number: 10706554Abstract: The present disclosure includes methods and systems for identifying and manipulating a segment of a three-dimensional digital model based on soft classification of the three-dimensional digital model. In particular, one or more embodiments of the disclosed systems and methods identify a soft classification of a digital model and utilize the soft classification to tune segmentation algorithms. For example, the disclosed systems and methods can utilize a soft classification to select a segmentation algorithm from a plurality of segmentation algorithms, to combine segmentation parameters from a plurality of segmentation algorithms, and/or to identify input parameters for a segmentation algorithm. The disclosed systems and methods can utilize the tuned segmentation algorithms to accurately and efficiently identify a segment of a three-dimensional digital model.Type: GrantFiled: April 14, 2017Date of Patent: July 7, 2020Assignee: ADOBE INC.Inventors: Vladimir Kim, Aaron Hertzmann, Mehmet Yumer