Patents by Inventor Wei-An Lin

Wei-An Lin has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 12254594
    Abstract: Methods, systems, and non-transitory computer readable media are disclosed for intelligently enhancing details in edited images. The disclosed system iteratively updates residual detail latent code for segments in edited images where detail has been lost through the editing process. More particularly, the disclosed system enhances an edited segment in an edited image based on details in a detailed segment of an image. Additionally, the disclosed system may utilize a detail neural network encoder to project the detailed segment and a corresponding segment of the edited image into a residual detail latent code. In some embodiments, the disclosed system generates a refined edited image based on the residual detail latent code and a latent vector of the edited image.
    Type: Grant
    Filed: April 1, 2022
    Date of Patent: March 18, 2025
    Assignee: Adobe Inc.
    Inventors: Hui Qu, Jingwan Lu, Saeid Motiian, Shabnam Ghadar, Wei-An Lin, Elya Shechtman
  • Patent number: 12254597
    Abstract: An item recommendation system receives a set of recommendable items and a request to select, from the set of recommendable items, a contrast group. The item recommendation system selects a contrast group from the set of recommendable items by applying a image modification model to the set of recommendable items. The image modification model includes an item selection model configured to determine an unbiased conversion rate for each item of the set of recommendable items and select a recommended item from the set of recommendable items having a greatest unbiased conversion rate. The image modification model includes a contrast group selection model configured to select, for the recommended item, a contrast group comprising the recommended item and one or more contrast items. The item recommendation system transmits the contrast group responsive to the request.
    Type: Grant
    Filed: March 30, 2022
    Date of Patent: March 18, 2025
    Assignee: Adobe Inc.
    Inventors: Cameron Smith, Wei-An Lin, Timothy M. Converse, Shabnam Ghadar, Ratheesh Kalarot, John Nack, Jingwan Lu, Hui Qu, Elya Shechtman, Baldo Faieta
  • Publication number: 20250069299
    Abstract: One or more aspects of a method, apparatus, and non-transitory computer readable medium include obtaining an input latent vector for an image generation network and a target lighting representation. A modified latent vector is generated based on the input latent vector and the target lighting representation, and an image generation network generates an image based on the modified latent vector using.
    Type: Application
    Filed: August 21, 2023
    Publication date: February 27, 2025
    Inventors: Kevin Duarte, Wei-An Lin, Ratheesh Kalarot, Shabnam Ghadar, Jingwan Lu, Elya Shechtman
  • Publication number: 20240412429
    Abstract: Systems and methods for editing multiple attributes of an image are described. Embodiments are configured to receive input comprising an image of a face and a target value of an attribute of the face to be modified; encode the image using an encoder of an image generation neural network to obtain an image embedding; and generate a modified image of the face having the target value of the attribute based on the image embedding using a decoder of the image generation neural network. The image generation neural network is trained using a plurality of training images generated by a separate training image generation neural network, and the plurality of training images include a first synthetic image having a first value of the attribute and a second synthetic image depicting a same face as the first synthetic image with a second value of the attribute.
    Type: Application
    Filed: June 9, 2023
    Publication date: December 12, 2024
    Inventors: Wei-An Lin, Hui Qu, Siavash Khodadadeh, Kevin Duarte, Surabhi Sinha, Ratheesh Kalarot, Shabnam Ghadar
  • Publication number: 20240404012
    Abstract: Systems and methods generate paired image data comprising synthesized eyeglass reflections and use the paired image data to train a machine learning model for reflection removal. A training dataset is generated that includes image pairs. Each image pair comprises a first version of a face image with eyeglasses not having a reflection and a second version of the face image with eyeglasses having a reflection. A first image pair in the training dataset is generated by: obtaining a first face image with eyeglasses not having a reflection, obtaining a reflection image, and generating a composite image using the first face image and the reflection image. Once generated, the training dataset is used to train a machine learning model to provide a trained machine learning model that performs reflection removal on input face images with eyeglass reflections.
    Type: Application
    Filed: June 2, 2023
    Publication date: December 5, 2024
    Inventors: Hui QU, Wei-An LIN, Ratheesh KALAROT
  • Patent number: 12014452
    Abstract: The present disclosure describes systems, methods, and non-transitory computer readable media for detecting user interactions to edit a digital image from a client device and modify the digital image for the client device by using a web-based intermediary that modifies a latent vector of the digital image and an image modification neural network to generate a modified digital image from the modified latent vector. In response to user interaction to modify a digital image, for instance, the disclosed systems modify a latent vector extracted from the digital image to reflect the requested modification. The disclosed systems further use a latent vector stream renderer (as an intermediary device) to generate an image delta that indicates a difference between the digital image and the modified digital image. The disclosed systems then provide the image delta as part of a digital stream to a client device to quickly render the modified digital image.
    Type: Grant
    Filed: August 14, 2023
    Date of Patent: June 18, 2024
    Assignee: Adobe Inc.
    Inventors: Akhilesh Kumar, Baldo Faieta, Piotr Walczyszyn, Ratheesh Kalarot, Archie Bagnall, Shabnam Ghadar, Wei-An Lin, Cameron Smith, Christian Cantrell, Patrick Hebron, Wilson Chan, Jingwan Lu, Holger Winnemoeller, Sven Olsen
  • Patent number: 11983628
    Abstract: Systems and methods dynamically adjust an available range for editing an attribute in an image. An image editing system computes a metric for an attribute in an input image as a function of a latent space representation of the input image and a filtering vector for editing the input image. The image editing system compares the metric to a threshold. If the metric exceeds the threshold, then the image editing system selects a first range for editing the attribute in the input image. If the metric does not exceed the threshold, a second range is selected. The image editing system causes display of a user interface for editing the input image comprising an interface element for editing the attribute within the selected range.
    Type: Grant
    Filed: September 7, 2021
    Date of Patent: May 14, 2024
    Assignee: Adobe Inc.
    Inventors: Wei-An Lin, Baldo Faieta, Cameron Smith, Elya Shechtman, Jingwan Lu, Jun-Yan Zhu, Niloy Mitra, Ratheesh Kalarot, Richard Zhang, Shabnam Ghadar, Zhixin Shu
  • Patent number: 11941727
    Abstract: Systems and methods for facial image generation are described. One aspect of the systems and methods includes receiving an image depicting a face, wherein the face has an identity non-related attribute and a first identity-related attribute; encoding the image to obtain an identity non-related attribute vector in an identity non-related attribute vector space, wherein the identity non-related attribute vector represents the identity non-related attribute; selecting an identity-related vector from an identity-related vector space, wherein the identity-related vector represents a second identity-related attribute different from the first identity-related attribute; generating a modified latent vector in a latent vector space based on the identity non-related attribute vector and the identity-related vector; and generating a modified image based on the modified latent vector, wherein the modified image depicts a face that has the identity non-related attribute and the second identity-related attribute.
    Type: Grant
    Filed: July 21, 2022
    Date of Patent: March 26, 2024
    Assignee: ADOBE INC.
    Inventors: Saeid Motiian, Wei-An Lin, Shabnam Ghadar
  • Patent number: 11900519
    Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure encode features of a source image to obtain a source appearance encoding that represents inherent attributes of a face in the source image; encode features of a target image to obtain a target non-appearance encoding that represents contextual attributes of the target image; combine the source appearance encoding and the target non-appearance encoding to obtain combined image features; and generate a modified target image based on the combined image features, wherein the modified target image includes the inherent attributes of the face in the source image together with the contextual attributes of the target image.
    Type: Grant
    Filed: November 17, 2021
    Date of Patent: February 13, 2024
    Assignee: ADOBE INC.
    Inventors: Kevin Duarte, Wei-An Lin, Ratheesh Kalarot, Shabnam Ghadar, Jingwan Lu, Elya Shechtman, John Thomas Nack
  • Publication number: 20240037805
    Abstract: Systems and methods for facial image generation are described. One aspect of the systems and methods includes receiving an image depicting a face, wherein the face has an identity non-related attribute and a first identity-related attribute; encoding the image to obtain an identity non-related attribute vector in an identity non-related attribute vector space, wherein the identity non-related attribute vector represents the identity non-related attribute; selecting an identity-related vector from an identity-related vector space, wherein the identity-related vector represents a second identity-related attribute different from the first identity-related attribute; generating a modified latent vector in a latent vector space based on the identity non-related attribute vector and the identity-related vector; and generating a modified image based on the modified latent vector, wherein the modified image depicts a face that has the identity non-related attribute and the second identity-related attribute.
    Type: Application
    Filed: July 21, 2022
    Publication date: February 1, 2024
    Inventors: Saeid Motiian, Wei-An Lin, Shabnam Ghadar
  • Patent number: 11880766
    Abstract: An improved system architecture uses a pipeline including a Generative Adversarial Network (GAN) including a generator neural network and a discriminator neural network to generate an image. An input image in a first domain and information about a target domain are obtained. The domains correspond to image styles. An initial latent space representation of the input image is produced by encoding the input image. An initial output image is generated by processing the initial latent space representation with the generator neural network. Using the discriminator neural network, a score is computed indicating whether the initial output image is in the target domain. A loss is computed based on the computed score. The loss is minimized to compute an updated latent space representation. The updated latent space representation is processed with the generator neural network to generate an output image in the target domain.
    Type: Grant
    Filed: July 23, 2021
    Date of Patent: January 23, 2024
    Assignee: Adobe Inc.
    Inventors: Cameron Smith, Ratheesh Kalarot, Wei-An Lin, Richard Zhang, Niloy Mitra, Elya Shechtman, Shabnam Ghadar, Zhixin Shu, Yannick Hold-Geoffrey, Nathan Carr, Jingwan Lu, Oliver Wang, Jun-Yan Zhu
  • Patent number: 11875221
    Abstract: Systems and methods generate a filtering function for editing an image with reduced attribute correlation. An image editing system groups training data into bins according to a distribution of a target attribute. For each bin, the system samples a subset of the training data based on a pre-determined target distribution of a set of additional attributes in the training data. The system identifies a direction in the sampled training data corresponding to the distribution of the target attribute to generate a filtering vector for modifying the target attribute in an input image, obtains a latent space representation of an input image, applies the filtering vector to the latent space representation of the input image to generate a filtered latent space representation of the input image, and provides the filtered latent space representation as input to a neural network to generate an output image with a modification to the target attribute.
    Type: Grant
    Filed: September 7, 2021
    Date of Patent: January 16, 2024
    Assignee: Adobe Inc.
    Inventors: Wei-An Lin, Baldo Faieta, Cameron Smith, Elya Shechtman, Jingwan Lu, Jun-Yan Zhu, Niloy Mitra, Ratheesh Kalarot, Richard Zhang, Shabnam Ghadar, Zhixin Shu
  • Publication number: 20230386114
    Abstract: The present disclosure describes systems, methods, and non-transitory computer readable media for detecting user interactions to edit a digital image from a client device and modify the digital image for the client device by using a web-based intermediary that modifies a latent vector of the digital image and an image modification neural network to generate a modified digital image from the modified latent vector. In response to user interaction to modify a digital image, for instance, the disclosed systems modify a latent vector extracted from the digital image to reflect the requested modification. The disclosed systems further use a latent vector stream renderer (as an intermediary device) to generate an image delta that indicates a difference between the digital image and the modified digital image. The disclosed systems then provide the image delta as part of a digital stream to a client device to quickly render the modified digital image.
    Type: Application
    Filed: August 14, 2023
    Publication date: November 30, 2023
    Inventors: Akhilesh Kumar, Baldo Faieta, Piotr Walczyszyn, Ratheesh Kalarot, Archie Bagnall, Shabnam Ghadar, Wei-An Lin, Cameron Smith, Christian Cantrell, Patrick Hebron, Wilson Chan, Jingwan Lu, Holger Winnemoeller, Sven Olsen
  • Patent number: 11823490
    Abstract: Systems and methods for image processing are described. One or more embodiments of the present disclosure identify a latent vector representing an image of a face, identify a target attribute vector representing a target attribute for the image, generate a modified latent vector using a mapping network that converts the latent vector and the target attribute vector into a hidden representation having fewer dimensions than the latent vector, wherein the modified latent vector is generated based on the hidden representation, and generate a modified image based on the modified latent vector, wherein the modified image represents the face with the target attribute.
    Type: Grant
    Filed: June 8, 2021
    Date of Patent: November 21, 2023
    Assignee: ADOBE, INC.
    Inventors: Ratheesh Kalarot, Siavash Khodadadeh, Baldo Faieta, Shabnam Ghadar, Saeid Motiian, Wei-An Lin, Zhe Lin
  • Publication number: 20230316475
    Abstract: An item recommendation system receives a set of recommendable items and a request to select, from the set of recommendable items, a contrast group. The item recommendation system selects a contrast group from the set of recommendable items by applying a image modification model to the set of recommendable items. The image modification model includes an item selection model configured to determine an unbiased conversion rate for each item of the set of recommendable items and select a recommended item from the set of recommendable items having a greatest unbiased conversion rate. The image modification model includes a contrast group selection model configured to select, for the recommended item, a contrast group comprising the recommended item and one or more contrast items. The item recommendation system transmits the contrast group responsive to the request.
    Type: Application
    Filed: March 30, 2022
    Publication date: October 5, 2023
    Inventors: Cameron Smith, Wei-An Lin, Timothy M. Converse, Shabnam Ghadar, Ratheesh Kalarot, John Nack, Jingwan Lu, Hui Qu, Elya Shechtman, Baldo Faieta
  • Publication number: 20230316606
    Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for latent-based editing of digital images using a generative neural network. In particular, in one or more embodiments, the disclosed systems perform latent-based editing of a digital image by mapping a feature tensor and a set of style vectors for the digital image into a joint feature style space. In one or more implementations, the disclosed systems apply a joint feature style perturbation and/or modification vectors within the joint feature style space to determine modified style vectors and a modified feature tensor. Moreover, in one or more embodiments the disclosed systems generate a modified digital image utilizing a generative neural network from the modified style vectors and the modified feature tensor.
    Type: Application
    Filed: March 21, 2022
    Publication date: October 5, 2023
    Inventors: Hui Qu, Baldo Faieta, Cameron Smith, Elya Shechtman, Jingwan Lu, Ratheesh Kalarot, Richard Zhang, Saeid Motiian, Shabnam Ghadar, Wei-An Lin
  • Publication number: 20230316474
    Abstract: Methods, systems, and non-transitory computer readable media are disclosed for intelligently enhancing details in edited images. The disclosed system iteratively updates residual detail latent code for segments in edited images where detail has been lost through the editing process. More particularly, the disclosed system enhances an edited segment in an edited image based on details in a detailed segment of an image. Additionally, the disclosed system may utilize a detail neural network encoder to project the detailed segment and a corresponding segment of the edited image into a residual detail latent code. In some embodiments, the disclosed system generates a refined edited image based on the residual detail latent code and a latent vector of the edited image.
    Type: Application
    Filed: April 1, 2022
    Publication date: October 5, 2023
    Inventors: Hui Qu, Jingwan Lu, Saeid Motiian, Shabnam Ghadar, Wei-An Lin, Elya Shechtman
  • Patent number: 11727614
    Abstract: The present disclosure describes systems, methods, and non-transitory computer readable media for detecting user interactions to edit a digital image from a client device and modify the digital image for the client device by using a web-based intermediary that modifies a latent vector of the digital image and an image modification neural network to generate a modified digital image from the modified latent vector. In response to user interaction to modify a digital image, for instance, the disclosed systems modify a latent vector extracted from the digital image to reflect the requested modification. The disclosed systems further use a latent vector stream renderer (as an intermediary device) to generate an image delta that indicates a difference between the digital image and the modified digital image. The disclosed systems then provide the image delta as part of a digital stream to a client device to quickly render the modified digital image.
    Type: Grant
    Filed: February 23, 2021
    Date of Patent: August 15, 2023
    Assignee: Adobe Inc.
    Inventors: Akhilesh Kumar, Baldo Faieta, Piotr Walczyszyn, Ratheesh Kalarot, Archie Bagnall, Shabnam Ghadar, Wei-An Lin, Cameron Smith, Christian Cantrell, Patrick Hebron, Wilson Chan, Jingwan Lu, Holger Winnemoeller, Sven Olsen
  • Publication number: 20230154088
    Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure encode features of a source image to obtain a source appearance encoding that represents inherent attributes of a face in the source image; encode features of a target image to obtain a target non-appearance encoding that represents contextual attributes of the target image; combine the source appearance encoding and the target non-appearance encoding to obtain combined image features; and generate a modified target image based on the combined image features, wherein the modified target image includes the inherent attributes of the face in the source image together with the contextual attributes of the target image.
    Type: Application
    Filed: November 17, 2021
    Publication date: May 18, 2023
    Inventors: Kevin Duarte, Wei-An Lin, Ratheesh Kalarot, Shabnam Ghadar, Jingwan Lu, Elya Shechtman, John Thomas Nack
  • Publication number: 20220391611
    Abstract: Systems and methods for image processing are described. One or more embodiments of the present disclosure identify a latent vector representing an image of a face, identify a target attribute vector representing a target attribute for the image, generate a modified latent vector using a mapping network that converts the latent vector and the target attribute vector into a hidden representation having fewer dimensions than the latent vector, wherein the modified latent vector is generated based on the hidden representation, and generate a modified image based on the modified latent vector, wherein the modified image represents the face with the target attribute.
    Type: Application
    Filed: June 8, 2021
    Publication date: December 8, 2022
    Inventors: RATHEESH KALAROT, Siavash Khodadadeh, Baldo Faieta, Shabnam Ghadar, Saeid Motiian, Wei-An Lin, Zhe Lin