Patents by Inventor Yangtuanfeng Wang

Yangtuanfeng Wang has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20240428491
    Abstract: The present disclosure relates to a system that utilizes neural networks to generate looping animations from still images. The system fits a 3D model to a pose of a person in a digital image. The system receives a 3D animation sequence that transitions between a starting pose and an ending pose. The system generates, utilizing an animation transition neural network, first and second 3D animation transition sequences that respectively transition between the pose of the person and the starting pose and between the ending pose and the pose of the person. The system modifies each of the 3D animation sequence, the first 3D animation transition sequence, and the second 3D animation transition sequence by applying a texture map. The system generates a looping 3D animation by combining the modified 3D animation sequence, the modified first 3D animation transition sequence, and the modified second 3D animation transition sequence.
    Type: Application
    Filed: June 23, 2023
    Publication date: December 26, 2024
    Inventors: Jae Shin Yoon, Duygu Ceylan Aksit, Yangtuanfeng Wang, Jingwan Lu, Jimei Yang, Zhixin Shu, Chengan He, Yi Zhou, Jun Saito, James Zachary
  • Patent number: 12165260
    Abstract: Systems and methods are described for rendering garments. The system includes a first machine learning model trained to generate coarse garment templates of a garment and a second machine learning model trained to render garment images. The first machine learning model generates a coarse garment template based on position data. The system produces a neural texture for the garment, the neural texture comprising a multi-dimensional feature map characterizing detail of the garment. The system provides the coarse garment template and the neural texture to the second machine learning model trained to render garment images. The second machine learning model generates a rendered garment image of the garment based on the coarse garment template of the garment and the neural texture.
    Type: Grant
    Filed: April 7, 2022
    Date of Patent: December 10, 2024
    Assignees: Adobe Inc., UCL Business Ltd.
    Inventors: Duygu Ceylan Aksit, Yangtuanfeng Wang, Niloy J. Mitra, Meng Zhang
  • Publication number: 20240404155
    Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that utilizes neural networks to generate cinemagraphs from single RGB images. For example, the cyclic animation system includes a cyclic animation neural network trained with synthetic data, wherein different wind effects can be replicated using physically based simulations to create cyclic videos more efficiently. More specifically, the cyclic animation system generalizes a solution by operating in the gradient domain and using surface normal maps. Because normal maps are invariant to appearance (color, texture, illumination, etc.), the gap between synthetic and real data distribution in the normal map space is smaller than in the RGB space. The cyclic animation system performs a reshading approach that synthesizes RGB pixels from the original image and the animated normal maps to create plausible changes to the real image to create the cinemagraph.
    Type: Application
    Filed: May 30, 2023
    Publication date: December 5, 2024
    Inventors: Duygu Ceylan Aksit, Hugo Bertiche Argila, Niloy Jyoti Mitra, Kuldeep Kulkarni, Chun Hao Huang, Yangtuanfeng Wang
  • Publication number: 20240378809
    Abstract: Decal application techniques as implemented by a computing device are described to perform decaling of a digital image. In one example, learned features of a digital image using machine learning are used by a computing device as a basis to predict the surface geometry of an object in the digital image. Once the surface geometry of the object is predicted, machine learning techniques are then used by the computing device to configure an overlay object to be applied onto the digital image according to the predicted surface geometry of the overlaid object.
    Type: Application
    Filed: May 12, 2023
    Publication date: November 14, 2024
    Applicant: Adobe Inc.
    Inventors: Yangtuanfeng Wang, Yi Zhou, Yasamin Jafarian, Nathan Aaron Carr, Jimei Yang, Duygu Ceylan Aksit
  • Patent number: 12067659
    Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and method that utilize a character animation neural network informed by motion and pose signatures to generate a digital video through person-specific appearance modeling and motion retargeting. In particular embodiments, the disclosed systems implement a character animation neural network that includes a pose embedding model to encode a pose signature into spatial pose features. The character animation neural network further includes a motion embedding model to encode a motion signature into motion features. In some embodiments, the disclosed systems utilize the motion features to refine per-frame pose features and improve temporal coherency. In certain implementations, the disclosed systems also utilize the motion features to demodulate neural network weights used to generate an image frame of a character in motion based on the refined pose features.
    Type: Grant
    Filed: October 15, 2021
    Date of Patent: August 20, 2024
    Assignee: Adobe Inc.
    Inventors: Yangtuanfeng Wang, Duygu Ceylan Aksit, Krishna Kumar Singh, Niloy J Mitra
  • Publication number: 20240169553
    Abstract: Techniques for modeling secondary motion based on three-dimensional models are described as implemented by a secondary motion modeling system, which is configured to receive a plurality of three-dimensional object models representing an object. Based on the three-dimensional object models, the secondary motion modeling system determines three-dimensional motion descriptors of a particular three-dimensional object model using one or more machine learning models. Based on the three-dimensional motion descriptors, the secondary motion modeling system models at least one feature subjected to secondary motion using the one or more machine learning models. The particular three-dimensional object model having the at least one feature is rendered by the secondary motion modeling system.
    Type: Application
    Filed: November 21, 2022
    Publication date: May 23, 2024
    Applicant: Adobe Inc.
    Inventors: Jae shin Yoon, Zhixin Shu, Yangtuanfeng Wang, Jingwan Lu, Jimei Yang, Duygu Ceylan Aksit
  • Patent number: 11978144
    Abstract: Embodiments are disclosed for using machine learning models to perform three-dimensional garment deformation due to character body motion with collision handling. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving an input, the input including character body shape parameters and character body pose parameters defining a character body, and garment parameters. The disclosed systems and methods further comprise generating, by a first neural network, a first set of garment vertices defining deformations of a garment with the character body based on the input. The disclosed systems and methods further comprise determining, by a second neural network, that the first set of garment vertices includes a second set of garment vertices penetrating the character body. The disclosed systems and methods further comprise modifying, by a third neural network, each garment vertex in the second set of garment vertices to positions outside the character body.
    Type: Grant
    Filed: July 27, 2022
    Date of Patent: May 7, 2024
    Assignee: Adobe Inc.
    Inventors: Yi Zhou, Yangtuanfeng Wang, Xin Sun, Qingyang Tan, Duygu Ceylan Aksit
  • Publication number: 20240144520
    Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify two-dimensional images via scene-based editing using three-dimensional representations of the two-dimensional images. For instance, in one or more embodiments, the disclosed systems utilize three-dimensional representations of two-dimensional images to generate and modify shadows in the two-dimensional images according to various shadow maps. Additionally, the disclosed systems utilize three-dimensional representations of two-dimensional images to modify humans in the two-dimensional images. The disclosed systems also utilize three-dimensional representations of two-dimensional images to provide scene scale estimation via scale fields of the two-dimensional images. In some embodiments, the disclosed systems utilizes three-dimensional representations of two-dimensional images to generate and visualize 3D planar surfaces for modifying objects in two-dimensional images.
    Type: Application
    Filed: April 20, 2023
    Publication date: May 2, 2024
    Inventors: Giorgio Gori, Yi Zhou, Yangtuanfeng Wang, Yang Zhou, Krishna Kumar Singh, Jae Shin Yoon, Duygu Ceylan Aksit
  • Publication number: 20240144623
    Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify two-dimensional images via scene-based editing using three-dimensional representations of the two-dimensional images. For instance, in one or more embodiments, the disclosed systems utilize three-dimensional representations of two-dimensional images to generate and modify shadows in the two-dimensional images according to various shadow maps. Additionally, the disclosed systems utilize three-dimensional representations of two-dimensional images to modify humans in the two-dimensional images. The disclosed systems also utilize three-dimensional representations of two-dimensional images to provide scene scale estimation via scale fields of the two-dimensional images. In some embodiments, the disclosed systems utilizes three-dimensional representations of two-dimensional images to generate and visualize 3D planar surfaces for modifying objects in two-dimensional images.
    Type: Application
    Filed: April 20, 2023
    Publication date: May 2, 2024
    Inventors: Giorgio Gori, Yi Zhou, Yangtuanfeng Wang, Yang Zhou, Krishna Kumar Singh, Jae Shin Yoon, Duygu Ceylan Aksit
  • Publication number: 20240135511
    Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.
    Type: Application
    Filed: March 27, 2023
    Publication date: April 25, 2024
    Inventors: Krishna Kumar Singh, Yijun Li, Jingwan Lu, Duygu Ceylan Aksit, Yangtuanfeng Wang, Jimei Yang, Tobias Hinz, Qing Liu, Jianming Zhang, Zhe Lin
  • Publication number: 20240135572
    Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.
    Type: Application
    Filed: March 27, 2023
    Publication date: April 25, 2024
    Inventors: Krishna Kumar Singh, Yijun Li, Jingwan Lu, Duygu Ceylan Aksit, Yangtuanfeng Wang, Jimei Yang, Tobias Hinz
  • Publication number: 20240135512
    Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.
    Type: Application
    Filed: March 27, 2023
    Publication date: April 25, 2024
    Inventors: Krishna Kumar Singh, Yijun Li, Jingwan Lu, Duygu Ceylan Aksit, Yangtuanfeng Wang, Jimei Yang, Tobias Hinz, Qing Liu, Jianming Zhang, Zhe Lin
  • Publication number: 20240135513
    Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.
    Type: Application
    Filed: March 27, 2023
    Publication date: April 25, 2024
    Inventors: Krishna Kumar Singh, Yijun Li, Jingwan Lu, Duygu Ceylan Aksit, Yangtuanfeng Wang, Jimei Yang, Tobias Hinz
  • Publication number: 20240037827
    Abstract: Embodiments are disclosed for using machine learning models to perform three-dimensional garment deformation due to character body motion with collision handling. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving an input, the input including character body shape parameters and character body pose parameters defining a character body, and garment parameters. The disclosed systems and methods further comprise generating, by a first neural network, a first set of garment vertices defining deformations of a garment with the character body based on the input. The disclosed systems and methods further comprise determining, by a second neural network, that the first set of garment vertices includes a second set of garment vertices penetrating the character body. The disclosed systems and methods further comprise modifying, by a third neural network, each garment vertex in the second set of garment vertices to positions outside the character body.
    Type: Application
    Filed: July 27, 2022
    Publication date: February 1, 2024
    Applicant: Adobe Inc.
    Inventors: Yi ZHOU, Yangtuanfeng WANG, Xin SUN, Qingyang TAN, Duygu CEYLAN AKSIT
  • Publication number: 20230326137
    Abstract: Systems and methods are described for rendering garments. The system includes a first machine learning model trained to generate coarse garment templates of a garment and a second machine learning model trained to render garment images. The first machine learning model generates a coarse garment template based on position data. The system produces a neural texture for the garment, the neural texture comprising a multi-dimensional feature map characterizing detail of the garment. The system provides the coarse garment template and the neural texture to the second machine learning model trained to render garment images. The second machine learning model generates a rendered garment image of the garment based on the coarse garment template of the garment and the neural texture.
    Type: Application
    Filed: April 7, 2022
    Publication date: October 12, 2023
    Inventors: Duygu Ceylan Aksit, Yangtuanfeng Wang, Niloy J. Mitra, Meng Zhang
  • Patent number: 11776189
    Abstract: In implementations of systems for generating digital objects to animate sketches, a computing device implements a sketch system to receive input data describing a user sketched digital object having a pose and a non-photorealistic style. The sketch system generates a latent vector representation of the user defined non-photorealistic style using an encoder of a generative adversarial network. A digital object is generated having the pose and a non-photorealistic style using a generator of the generative adversarial network based on the latent vector representation of the user defined non-photorealistic style. The sketch system modifies the latent vector representation of the user defined non-photorealistic style based on a comparison between the user defined non-photorealistic style and the non-photorealistic style.
    Type: Grant
    Filed: October 22, 2021
    Date of Patent: October 3, 2023
    Assignee: Adobe Inc.
    Inventors: Zeyu Wang, Yangtuanfeng Wang
  • Publication number: 20230126250
    Abstract: In implementations of systems for generating digital objects to animate sketches, a computing device implements a sketch system to receive input data describing a user sketched digital object having a pose and a non-photorealistic style. The sketch system generates a latent vector representation of the user defined non-photorealistic style using an encoder of a generative adversarial network. A digital object is generated having the pose and a non-photorealistic style using a generator of the generative adversarial network based on the latent vector representation of the user defined non-photorealistic style. The sketch system modifies the latent vector representation of the user defined non-photorealistic style based on a comparison between the user defined non-photorealistic style and the non-photorealistic style.
    Type: Application
    Filed: October 22, 2021
    Publication date: April 27, 2023
    Applicant: Adobe Inc.
    Inventors: Zeyu Wang, Yangtuanfeng Wang
  • Publication number: 20230123820
    Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and method that utilize a character animation neural network informed by motion and pose signatures to generate a digital video through person-specific appearance modeling and motion retargeting. In particular embodiments, the disclosed systems implement a character animation neural network that includes a pose embedding model to encode a pose signature into spatial pose features. The character animation neural network further includes a motion embedding model to encode a motion signature into motion features. In some embodiments, the disclosed systems utilize the motion features to refine per-frame pose features and improve temporal coherency. In certain implementations, the disclosed systems also utilize the motion features to demodulate neural network weights used to generate an image frame of a character in motion based on the refined pose features.
    Type: Application
    Filed: October 15, 2021
    Publication date: April 20, 2023
    Inventors: Yangtuanfeng Wang, Duygu Ceylan Aksit, Krishna Kumar Singh, Niloy J Mitra
  • Publication number: 20190279414
    Abstract: Systems and techniques provide a user interface within an application to enable users to designate a folded object image of a folded object, as well as a superimposed image of a superimposed object to be added to the folded object image. Within the user interface, the user may simply place the superimposed image over the folded object image to obtain the desired modified image. If the user places the superimposed image over one or more folds of the folded object image, portions of the superimposed image will be removed to create the illusion in the modified image that the removed portions are obscured by one or more folds.
    Type: Application
    Filed: March 8, 2018
    Publication date: September 12, 2019
    Inventors: Duygu Ceylan Aksit, Yangtuanfeng Wang, Niloy Jyoti Mitra, Mehmet Ersin Yumer, Jovan Popovic
  • Patent number: 10410400
    Abstract: Systems and techniques provide a user interface within an application to enable users to designate a folded object image of a folded object, as well as a superimposed image of a superimposed object to be added to the folded object image. Within the user interface, the user may simply place the superimposed image over the folded object image to obtain the desired modified image. If the user places the superimposed image over one or more folds of the folded object image, portions of the superimposed image will be removed to create the illusion in the modified image that the removed portions are obscured by one or more folds.
    Type: Grant
    Filed: March 8, 2018
    Date of Patent: September 10, 2019
    Assignee: Adobe Inc.
    Inventors: Duygu Ceylan Aksit, Yangtuanfeng Wang, Niloy Jyoti Mitra, Mehmet Ersin Yumer, Jovan Popovic