Patents by Inventor Songyao Jiang

Songyao Jiang has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20230126178
    Abstract: Embodiments identify joints of a multi-limb body in an image. One such embodiment unifies depth of a plurality of multi-scale feature maps generated from an image of a multi-limb body to create a plurality of feature maps each having a same depth. In turn, for each of the plurality of feature maps having the same depth, an initial indication of one or more joints in the image is generated. The one or more joints are located at an interconnection of a limb to the multi-limb body or at an interconnection of a limb to another limb. To continue, a final indication of the one or more joints in the image is generated using each generated initial indication of the one or more joints.
    Type: Application
    Filed: February 10, 2021
    Publication date: April 27, 2023
    Inventors: Yun Fu, Songyao Jiang, Bin Sun
  • Publication number: 20220386759
    Abstract: The present disclosure provides systems and methods for virtual facial makeup simulation through virtual makeup removal and virtual makeup add-ons, virtual end effects and simulated textures. In one aspect, the present disclosure provides a method for virtually removing facial makeup, the method comprising providing a facial image of a user with makeups being applied thereto, locating facial landmarks from the facial image of the user in one or more regions, decomposing some regions into first channels which are fed to histogram matching to obtain a first image without makeup in that region and transferring other regions into color channels which are fed into histogram matching under different lighting conditions to obtain a second image without makeup in that region, and combining the images to form a resultant image with makeups removed in the facial regions.
    Type: Application
    Filed: May 29, 2022
    Publication date: December 8, 2022
    Applicant: Shiseido Company, Limited
    Inventors: Yun Fu, Shuyang Wang, Sumin Lee, Songyao Jiang, Bin Sun, Haiyi Mao, Kai Ho Edgar Cheung
  • Publication number: 20220254157
    Abstract: Embodiments provide functionality for identifying joints and limbs in frames of video that use indications of joints and limbs from a previous frame. One such embodiment processes a current frame of video to determine initial predictions of joint and limb locations in the current frame. In turn, indications of the joint and limb locations in the current frame are generated by refining the initial predictions of the joint and limb locations based on indications of respective joint and limb locations from a previous frame. Embodiments provide results that are insensitive to occlusions and results that have less shaking and vibration.
    Type: Application
    Filed: May 13, 2020
    Publication date: August 11, 2022
    Inventors: Yun Fu, Songyao Jiang
  • Patent number: 10825219
    Abstract: Embodiments provide methods and systems for image generation through use of adversarial networks. An embodiment trains an image generator comprising (i) a generator implemented with a first neural network configured to generate a fake image based on a target segmentation, (ii) a discriminator implemented with a second neural network configured to distinguish a real image from a fake image and output a discrimination result as a function thereof and (iii) a segmentor implemented with a third neural network configured to generate a segmentation from the fake image. The training includes (i) operating the generator to output the fake image to the discriminator and the segmentor and (ii) iteratively operating the generator, discriminator, and segmentor during a training period, whereby the discriminator and generator train in an adversarial relationship with each other and the generator and segmentor train in a collaborative relationship with each other.
    Type: Grant
    Filed: March 22, 2019
    Date of Patent: November 3, 2020
    Assignee: Northeastern University
    Inventors: Yun Fu, Songyao Jiang
  • Publication number: 20200015575
    Abstract: The present disclosure provides systems and methods for virtual facial makeup simulation through virtual makeup removal and virtual makeup add-ons, virtual end effects and simulated textures. In one aspect, the present disclosure provides a method for virtually removing facial makeup, the method comprising providing a facial image of a user with makeups being applied thereto, locating facial landmarks from the facial image of the user in one or more regions, decomposing some regions into first channels which are fed to histogram matching to obtain a first image without makeup in that region and transferring other regions into color channels which are fed into histogram matching under different lighting conditions to obtain a second image without makeup in that region, and combining the images to form a resultant image with makeups removed in the facial regions.
    Type: Application
    Filed: September 26, 2019
    Publication date: January 16, 2020
    Applicant: Shiseido Americas Corporation
    Inventors: Yun Fu, Shuyang Wang, Sumin Lee, Songyao Jiang, Bin Sun, Haiyi Mao, Kai Ho Edgar Cheung
  • Publication number: 20190289986
    Abstract: The present disclosure provides systems and methods for virtual facial makeup simulation through virtual makeup removal and virtual makeup add-ons, virtual end effects and simulated textures. In one aspect, the present disclosure provides a method for virtually removing facial makeup, the method comprising providing a facial image of a user with makeups being applied thereto, locating facial landmarks from the facial image of the user in one or more regions, decomposing some regions into first channels which are fed to histogram matching to obtain a first image without makeup in that region and transferring other regions into color channels which are fed into histogram matching under different lighting conditions to obtain a second image without makeup in that region, and combining the images to form a resultant image with makeups removed in the facial regions.
    Type: Application
    Filed: June 11, 2019
    Publication date: September 26, 2019
    Applicant: Shiseido Americas Corporation
    Inventors: Yun Fu, Shuyang Wang, Sumin Lee, Songyao Jiang, Bin Sun, Haiyi Mao, Kai Ho Edgar Cheung
  • Publication number: 20190295302
    Abstract: Embodiments provide methods and systems for image generation through use of adversarial networks. An embodiment trains an image generator comprising (i) a generator implemented with a first neural network configured to generate a fake image based on a target segmentation, (ii) a discriminator implemented with a second neural network configured to distinguish a real image from a fake image and output a discrimination result as a function thereof and (iii) a segmentor implemented with a third neural network configured to generate a segmentation from the fake image. The training includes (i) operating the generator to output the fake image to the discriminator and the segmentor and (ii) iteratively operating the generator, discriminator, and segmentor during a training period, whereby the discriminator and generator train in an adversarial relationship with each other and the generator and segmentor train in a collaborative relationship with each other.
    Type: Application
    Filed: March 22, 2019
    Publication date: September 26, 2019
    Inventors: Yun Fu, Songyao Jiang
  • Publication number: 20190269224
    Abstract: The present disclosure provides systems and methods for virtual facial makeup simulation through virtual makeup removal and virtual makeup add-ons, virtual end effects and simulated textures. In one aspect, the present disclosure provides a method for virtually removing facial makeup, the method comprising providing a facial image of a user with makeups being applied thereto, locating facial landmarks from the facial image of the user in one or more regions, decomposing some regions into first channels which are fed to histogram matching to obtain a first image without makeup in that region and transferring other regions into color channels which are fed into histogram matching under different lighting conditions to obtain a second image without makeup in that region, and combining the images to form a resultant image with makeups removed in the facial regions.
    Type: Application
    Filed: May 3, 2019
    Publication date: September 5, 2019
    Applicant: Shiseido Americas Corporation
    Inventors: Yun Fu, Shuyang Wang, Sumin Lee, Songyao Jiang, Bin Sun, Haiyi Mao, Kai Ho Edgar Cheung
  • Publication number: 20190014884
    Abstract: The present disclosure provides systems and methods for virtual facial makeup simulation through virtual makeup removal and virtual makeup add-ons, virtual end effects and simulated textures. In one aspect, the present disclosure provides a method for virtually removing facial makeup, the method comprising providing a facial image of a user with makeups being applied thereto, locating facial landmarks from the facial image of the user in one or more regions, decomposing some regions into first channels which are fed to histogram matching to obtain a first image without makeup in that region and transferring other regions into color channels which are fed into histogram matching under different lighting conditions to obtain a second image without makeup in that region, and combining the images to form a resultant image with makeups removed in the facial regions.
    Type: Application
    Filed: July 13, 2018
    Publication date: January 17, 2019
    Inventors: Yun Fu, Shuyang Wang, Sumin Lee, Songyao Jiang, Bin Sun, Haiyi Mao, Kai Ho Edgar Cheung