Patents by Inventor Jason Wen

Jason Wen has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20230252774
    Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure receive a training image and a caption for the training image, wherein the caption includes text describing an object in the training image; generate a pseudo mask for the object using a teacher network based on the text describing the object; generate a mask for the object using a student network; and update parameters of the student network based on the mask and the pseudo mask.
    Type: Application
    Filed: February 9, 2022
    Publication date: August 10, 2023
    Inventors: Jason Wen Yong Kuen, Dat Ba Huynh, Zhe Lin, Jiuxiang Gu
  • Publication number: 20230154185
    Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure receive an image having a plurality of object instances; encode the image to obtain image features; decode the image features to obtain object features; generate object detection information based on the object features using an object detection branch, wherein the object detection branch is trained based on a first training set using a detection loss; generate semantic segmentation information based on the object features using a semantic segmentation branch, wherein the semantic segmentation branch is trained based on a second training set different from the first training set using a semantic segmentation loss; and combine the object detection information and the semantic segmentation information to obtain panoptic segmentation information that indicates which pixels of the image correspond to each of the plurality of object instances.
    Type: Application
    Filed: November 12, 2021
    Publication date: May 18, 2023
    Inventors: Jason Wen Yong Kuen, Bo Sun, Zhe Lin, Simon Su Chen
  • Publication number: 20230154221
    Abstract: The technology described includes methods for pretraining a document encoder model based on multimodal self cross-attention. One method includes receiving image data that encodes a set of pretraining documents. A set of sentences is extracted from the image data. A bounding box for each sentence is generated. For each sentence, a set of predicted features is generated by using an encoder machine-learning model. The encoder model performs cross-attention between a set of masked-textual features for the sentence and a set of masked-visual features for the sentence. The set of masked-textual features is based on a masking function and the sentence. The set of masked-visual features is based on the masking function and the corresponding bounding box. A document-encoder model is pretrained based on the set of predicted features for each sentence and pretraining tasks. The pretraining tasks includes masked sentence modeling, visual contrastive learning, or visual-language alignment.
    Type: Application
    Filed: November 16, 2021
    Publication date: May 18, 2023
    Inventors: Jiuxiang Gu, Ani Nenkova Nenkova, Nikolaos Barmpalios, Vlad Ion Morariu, Tong Sun, Rajiv Bhawanji Jain, Jason wen yong Kuen, Handong Zhao
  • Publication number: 20230128792
    Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates object masks for digital objects portrayed in digital images utilizing a detection-masking neural network pipeline. In particular, in one or more embodiments, the disclosed systems utilize detection heads of a neural network to detect digital objects portrayed within a digital image. In some cases, each detection head is associated with one or more digital object classes that are not associated with the other detection heads. Further, in some cases, the detection heads implement multi-scale synchronized batch normalization to normalize feature maps across various feature levels. The disclosed systems further utilize a masking head of the neural network to generate one or more object masks for the detected digital objects. In some cases, the disclosed systems utilize post-processing techniques to filter out low-quality masks.
    Type: Application
    Filed: January 31, 2022
    Publication date: April 27, 2023
    Inventors: Jason Wen Yong Kuen, Su Chen, Scott Cohen, Zhe Lin, Zijun Wei, Jianming Zhang
  • Publication number: 20230104262
    Abstract: Various disclosed embodiments are directed to refining or correcting individual semantic segmentation/instance segmentation masks that have already been produced by baseline models in order to generate a final coherent panoptic segmentation map. Specifically, a refinement model, such as an encoder-decoder-based neural network, generates or predicts various data objects, such as foreground masks, bounding box offset maps, center maps, center offset maps, and coordinate convolution. This, among other functionality described herein, improves the inaccuracies and computing resource consumption of existing technologies.
    Type: Application
    Filed: October 6, 2021
    Publication date: April 6, 2023
    Inventors: Zhe Lin, Simon Su Chen, Jason Wen-youg Kuen, Bo Sun
  • Patent number: 11610393
    Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and efficiently learning parameters of a distilled neural network from parameters of a source neural network utilizing multiple augmentation strategies. For example, the disclosed systems can generate lightly augmented digital images and heavily augmented digital images. The disclosed systems can further learn parameters for a source neural network from the lightly augmented digital images. Moreover, the disclosed systems can learn parameters for a distilled neural network from the parameters learned for the source neural network. For example, the disclosed systems can compare classifications of heavily augmented digital images generated by the source neural network and the distilled neural network to transfer learned parameters from the source neural network to the distilled neural network via a knowledge distillation loss function.
    Type: Grant
    Filed: October 2, 2020
    Date of Patent: March 21, 2023
    Assignee: Adobe Inc.
    Inventors: Jason Wen Yong Kuen, Zhe Lin, Jiuxiang Gu
  • Publication number: 20220284321
    Abstract: Systems and methods for multi-modal representation learning are described. One or more embodiments provide a visual representation learning system trained using machine learning techniques. For example, some embodiments of the visual representation learning system are trained using cross-modal training tasks including a combination of intra-modal and inter-modal similarity preservation objectives. In some examples, the training tasks are based on contrastive learning techniques.
    Type: Application
    Filed: March 3, 2021
    Publication date: September 8, 2022
    Inventors: Xin Yuan, Zhe Lin, Jason Wen Yong Kuen, Jianming Zhang, Yilin Wang, Ajinkya Kale, Baldo Faieta
  • Publication number: 20220157054
    Abstract: In implementations of object detection in images, object detectors are trained using heterogeneous training datasets. A first training dataset is used to train an image tagging network to determine an attention map of an input image for a target concept. A second training dataset is used to train a conditional detection network that accepts as conditional inputs the attention map and a word embedding of the target concept. Despite the conditional detection network being trained with a training dataset having a small number of seen classes (e.g., classes in a training dataset), it generalizes to novel, unseen classes by concept conditioning, since the target concept propagates through the conditional detection network via the conditional inputs, thus influencing classification and region proposal. Hence, classes of objects that can be detected are expanded, without the need to scale training databases to include additional classes.
    Type: Application
    Filed: January 31, 2022
    Publication date: May 19, 2022
    Applicant: Adobe Inc.
    Inventors: Zhe Lin, Xiaohui Shen, Mingyang Ling, Jianming Zhang, Jason Wen Yong Kuen
  • Publication number: 20220147838
    Abstract: Methods and systems disclosed herein relate generally to systems and methods for generating visual relationship graphs that identify relationships between objects depicted in an image. A vision-language application uses transformer encoders to generate a graph structure, in which the graph structure represents a dependency between a first region and a second region of an image. The dependency indicates that a contextual representation of the first region was derived, at least in part, by processing the second region. The contextual representation identifies a predicted identity of an image object depicted in the first region. The predicted identity is determined at least in part by identifying a relationship between the first region and other data objects associated with various modalities.
    Type: Application
    Filed: November 9, 2020
    Publication date: May 12, 2022
    Inventors: Jiuxiang Gu, Vlad Ion Morariu, Tong Sun, Jason wen yong Kuen, Handong Zhao
  • Publication number: 20220108131
    Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and efficiently learning parameters of a distilled neural network from parameters of a source neural network utilizing multiple augmentation strategies. For example, the disclosed systems can generate lightly augmented digital images and heavily augmented digital images. The disclosed systems can further learn parameters for a source neural network from the lightly augmented digital images. Moreover, the disclosed systems can learn parameters for a distilled neural network from the parameters learned for the source neural network. For example, the disclosed systems can compare classifications of heavily augmented digital images generated by the source neural network and the distilled neural network to transfer learned parameters from the source neural network to the distilled neural network via a knowledge distillation loss function.
    Type: Application
    Filed: October 2, 2020
    Publication date: April 7, 2022
    Inventors: Jason Wen Yong Kuen, Zhe Lin, Jiuxiang Gu
  • Patent number: 11256918
    Abstract: In implementations of object detection in images, object detectors are trained using heterogeneous training datasets. A first training dataset is used to train an image tagging network to determine an attention map of an input image for a target concept. A second training dataset is used to train a conditional detection network that accepts as conditional inputs the attention map and a word embedding of the target concept. Despite the conditional detection network being trained with a training dataset having a small number of seen classes (e.g., classes in a training dataset), it generalizes to novel, unseen classes by concept conditioning, since the target concept propagates through the conditional detection network via the conditional inputs, thus influencing classification and region proposal. Hence, classes of objects that can be detected are expanded, without the need to scale training databases to include additional classes.
    Type: Grant
    Filed: May 14, 2020
    Date of Patent: February 22, 2022
    Assignee: Adobe Inc.
    Inventors: Zhe Lin, Xiaohui Shen, Mingyang Ling, Jianming Zhang, Jason Wen Yong Kuen
  • Publication number: 20210174571
    Abstract: Systems, apparatuses, and methods for implementing kernel software driven color remapping of rendered primary surfaces are disclosed. A system includes at least a general processor, a graphics processor, and a memory. The general processor executes a user-mode application, a user-mode driver, and a kernel-mode driver. A primary surface is rendered on the graphics processor on behalf of the user-mode application. The primary surface is stored in memory locations allocated for the primary surface by the user-mode driver and the kernel-mode driver is notified when the primary surface is ready to be displayed. Rather than displaying the primary surface, the kernel-mode driver causes the pixels of the primary surface to be remapped on the graphics processor using a selected lookup table (LUT) so as to generate a remapped surface which stored in memory locations allocated for the remapped surface by the user-mode driver. Then, the remapped surface is displayed.
    Type: Application
    Filed: December 12, 2019
    Publication date: June 10, 2021
    Inventors: Jason Wen-Tse Wu, Parimalkumar Patel, Jia Hui Li, Chao Zhan
  • Publication number: 20200272822
    Abstract: In implementations of object detection in images, object detectors are trained using heterogeneous training datasets. A first training dataset is used to train an image tagging network to determine an attention map of an input image for a target concept. A second training dataset is used to train a conditional detection network that accepts as conditional inputs the attention map and a word embedding of the target concept. Despite the conditional detection network being trained with a training dataset having a small number of seen classes (e.g., classes in a training dataset), it generalizes to novel, unseen classes by concept conditioning, since the target concept propagates through the conditional detection network via the conditional inputs, thus influencing classification and region proposal. Hence, classes of objects that can be detected are expanded, without the need to scale training databases to include additional classes.
    Type: Application
    Filed: May 14, 2020
    Publication date: August 27, 2020
    Applicant: Adobe Inc.
    Inventors: Zhe Lin, Xiaohui Shen, Mingyang Ling, Jianming Zhang, Jason Wen Yong Kuen
  • Patent number: 10755099
    Abstract: In implementations of object detection in images, object detectors are trained using heterogeneous training datasets. A first training dataset is used to train an image tagging network to determine an attention map of an input image for a target concept. A second training dataset is used to train a conditional detection network that accepts as conditional inputs the attention map and a word embedding of the target concept. Despite the conditional detection network being trained with a training dataset having a small number of seen classes (e.g., classes in a training dataset), it generalizes to novel, unseen classes by concept conditioning, since the target concept propagates through the conditional detection network via the conditional inputs, thus influencing classification and region proposal. Hence, classes of objects that can be detected are expanded, without the need to scale training databases to include additional classes.
    Type: Grant
    Filed: November 13, 2018
    Date of Patent: August 25, 2020
    Assignee: Adobe Inc.
    Inventors: Zhe Lin, Xiaohui Shen, Mingyang Ling, Jianming Zhang, Jason Wen Yong Kuen
  • Publication number: 20200151448
    Abstract: In implementations of object detection in images, object detectors are trained using heterogeneous training datasets. A first training dataset is used to train an image tagging network to determine an attention map of an input image for a target concept. A second training dataset is used to train a conditional detection network that accepts as conditional inputs the attention map and a word embedding of the target concept. Despite the conditional detection network being trained with a training dataset having a small number of seen classes (e.g., classes in a training dataset), it generalizes to novel, unseen classes by concept conditioning, since the target concept propagates through the conditional detection network via the conditional inputs, thus influencing classification and region proposal. Hence, classes of objects that can be detected are expanded, without the need to scale training databases to include additional classes.
    Type: Application
    Filed: November 13, 2018
    Publication date: May 14, 2020
    Applicant: Adobe Inc.
    Inventors: Zhe Lin, Xiaohui Shen, Mingyang Ling, Jianming Zhang, Jason Wen Yong Kuen
  • Publication number: 20030123988
    Abstract: A set of fan blades comprises a plurality of blades evenly distributed around a hub case, wherein each of the blades has an outer edge from which a projection extends away at an angle.
    Type: Application
    Filed: June 3, 2002
    Publication date: July 3, 2003
    Inventor: Jason Wen