Patents by Inventor Xiaohui Shen

Xiaohui Shen has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20210350504
    Abstract: Methods and systems are provided for generating enhanced image. A neural network system is trained where the training includes training a first neural network that generates enhanced images conditioned on content of an image undergoing enhancement and training a second neural network that designates realism of the enhanced images generated by the first neural network. The neural network system is trained by determine loss and accordingly adjusting the appropriate neural network(s). The trained neural network system is used to generate an enhanced aesthetic image from a selected image where the output enhanced aesthetic image has increased aesthetics when compared to the selected image.
    Type: Application
    Filed: July 19, 2021
    Publication date: November 11, 2021
    Inventors: Xiaohui Shen, Zhe Lin, Xin Lu, Sarah Aye Kong, I-Ming Pao, Yingcong Chen
  • Publication number: 20210325118
    Abstract: A heat exchange device includes a core and a housing. The core comprises a first collecting part and a second collecting part, and a flat tube part is provided between the two. The flat tube part comprises a first flat tube group and a second flat tube group. The first collecting part comprises first and second collecting portions, and a separator is formed between the two. Each flat tube of the first flat tube group is communicated with the collecting cavity of the first collecting portion. The collecting cavity of the first collecting portion is communicated with the collecting cavity of the second collecting portion by means of the first flat tube group, the collecting cavity of the second collecting part, and the second flat tube group.
    Type: Application
    Filed: November 27, 2019
    Publication date: October 21, 2021
    Inventors: Qie SHEN, Weixin JIANG, Xiaohui WU
  • Patent number: 11152032
    Abstract: The present disclosure is directed toward systems and methods for tracking objects in videos. For example, one or more embodiments described herein utilize various tracking methods in combination with an image search index made up of still video frames indexed from a video. One or more embodiments described herein utilize a backward and forward tracking method that is anchored by one or more key frames in order to accurately track an object through the frames of a video, even when the video is long and may include challenging conditions.
    Type: Grant
    Filed: April 25, 2019
    Date of Patent: October 19, 2021
    Assignee: ADOBE INC.
    Inventors: Zhihong Ding, Zhe Lin, Xiaohui Shen, Michael Kaplan, Jonathan Brandt
  • Patent number: 11069030
    Abstract: Methods and systems are provided for generating enhanced image. A neural network system is trained where the training includes training a first neural network that generates enhanced images conditioned on content of an image undergoing enhancement and training a second neural network that designates realism of the enhanced images generated by the first neural network. The neural network system is trained by determine loss and accordingly adjusting the appropriate neural network(s). The trained neural network system is used to generate an enhanced aesthetic image from a selected image where the output enhanced aesthetic image has increased aesthetics when compared to the selected image.
    Type: Grant
    Filed: March 22, 2018
    Date of Patent: July 20, 2021
    Assignee: Adobe, Inc.
    Inventors: Xiaohui Shen, Zhe Lin, Xin Lu, Sarah Aye Kong, I-Ming Pao, Yingcong Chen
  • Publication number: 20210201150
    Abstract: Various embodiments describe frame selection based on training and using a neural network. In an example, the neural network is a convolutional neural network trained with training pairs. Each training pair includes two training frames from a frame collection. The loss function relies on the estimated quality difference between the two training frames. Further, the definition of the loss function varies based on the actual quality difference between these two frames. In a further example, the neural network is trained by incorporating facial heatmaps generated from the training frames and facial quality scores of faces detected in the training frames. In addition, the training involves using a feature mean that represents an average of the features of the training frames belonging to the same frame collection. Once the neural network is trained, a frame collection is input thereto and a frame is selected based on generated quality scores.
    Type: Application
    Filed: March 17, 2021
    Publication date: July 1, 2021
    Inventors: Zhe Lin, Xiaohui Shen, Radomir Mech, Jian Ren
  • Patent number: 10990877
    Abstract: Various embodiments describe frame selection based on training and using a neural network. In an example, the neural network is a convolutional neural network trained with training pairs. Each training pair includes two training frames from a frame collection. The loss function relies on the estimated quality difference between the two training frames. Further, the definition of the loss function varies based on the actual quality difference between these two frames. In a further example, the neural network is trained by incorporating facial heatmaps generated from the training frames and facial quality scores of faces detected in the training frames. In addition, the training involves using a feature mean that represents an average of the features of the training frames belonging to the same frame collection. Once the neural network is trained, a frame collection is input thereto and a frame is selected based on generated quality scores.
    Type: Grant
    Filed: January 9, 2018
    Date of Patent: April 27, 2021
    Assignee: ADOBE INC.
    Inventors: Zhe Lin, Xiaohui Shen, Radomir Mech, Jian Ren
  • Publication number: 20210110589
    Abstract: Embodiments of the present invention are directed to facilitating region of interest preservation. In accordance with some embodiments of the present invention, a region of interest preservation score using adaptive margins is determined. The region of interest preservation score indicates an extent to which at least one region of interest is preserved in a candidate image crop associated with an image. A region of interest positioning score is determined that indicates an extent to which a position of the at least one region of interest is preserved in the candidate image crop associated with the image. The region of interest preservation score and/or the preserving score are used to select a set of one or more candidate image crops as image crop suggestions.
    Type: Application
    Filed: October 29, 2020
    Publication date: April 15, 2021
    Inventors: Jianming Zhang, Zhe Lin, Radomir Mech, Xiaohui Shen
  • Patent number: 10949744
    Abstract: Provided are systems and techniques that provide an output phrase describing an image. An example method includes creating, with a convolutional neural network, feature maps describing image features in locations in the image. The method also includes providing a skeletal phrase for the image by processing the feature maps with a first long short-term memory (LSTM) neural network trained based on a first set of ground truth phrases which exclude attribute words. Then, attribute words are provided by processing the skeletal phrase and the feature maps with a second LSTM neural network trained based on a second set of ground truth phrases including words for attributes. Then, the method combines the skeletal phrase and the attribute words to form the output phrase.
    Type: Grant
    Filed: July 10, 2019
    Date of Patent: March 16, 2021
    Assignee: ADOBE INC.
    Inventors: Zhe Lin, Yufei Wang, Scott Cohen, Xiaohui Shen
  • Patent number: 10878550
    Abstract: Systems and methods are disclosed for estimating aesthetic quality of digital images using deep learning. In particular, the disclosed systems and methods describe training a neural network to generate an aesthetic quality score digital images. In particular, the neural network includes a training structure that compares relative rankings of pairs of training images to accurately predict a relative ranking of a digital image. Additionally, in training the neural network, an image rating system can utilize content-aware and user-aware sampling techniques to identify pairs of training images that have similar content and/or that have been rated by the same or different users. Using content-aware and user-aware sampling techniques, the neural network can be trained to accurately predict aesthetic quality ratings that reflect subjective opinions of most users as well as provide aesthetic scores for digital images that represent the wide spectrum of aesthetic preferences of various users.
    Type: Grant
    Filed: October 31, 2019
    Date of Patent: December 29, 2020
    Assignee: ADOBE INC.
    Inventors: Xiaohui Shen, Zhe Lin, Shu Kong, Radomir Mech
  • Patent number: 10867422
    Abstract: Embodiments of the present invention are directed to facilitating region of interest preservation. In accordance with some embodiments of the present invention, a region of interest preservation score using adaptive margins is determined. The region of interest preservation score indicates an extent to which at least one region of interest is preserved in a candidate image crop associated with an image. A region of interest positioning score is determined that indicates an extent to which a position of the at least one region of interest is preserved in the candidate image crop associated with the image. The region of interest preservation score and/or the preserving score are used to select a set of one or more candidate image crops as image crop suggestions.
    Type: Grant
    Filed: June 12, 2017
    Date of Patent: December 15, 2020
    Assignee: ADOBE Inc.
    Inventors: Jianming Zhang, Zhe Lin, Radomir Mech, Xiaohui Shen
  • Patent number: 10867416
    Abstract: Methods and systems are provided for generating harmonized images for input composite images. A neural network system can be trained, where the training includes training a neural network that generates harmonized images for input composite images. This training is performed based on a comparison of a training harmonized image and a reference image, where the reference image is modified to generate a training input composite image used to generate the training harmonized image. In addition, a mask of a region can be input to limit the area of the input image that is to be modified. Such a trained neural network system can be used to input a composite image and mask pair for which the trained system will output a harmonized image.
    Type: Grant
    Filed: March 10, 2017
    Date of Patent: December 15, 2020
    Assignee: ADOBE INC.
    Inventors: Xiaohui Shen, Zhe Lin, Yi-Hsuan Tsai, Xin Lu, Kalyan K. Sunkavalli
  • Publication number: 20200372622
    Abstract: The present disclosure relates to training and utilizing an image exposure transformation network to generate a long-exposure image from a single short-exposure image (e.g., still image). In various embodiments, the image exposure transformation network is trained using adversarial learning, long-exposure ground truth images, and a multi-term loss function. In some embodiments, the image exposure transformation network includes an optical flow prediction network and/or an appearance guided attention network. Trained embodiments of the image exposure transformation network generate realistic long-exposure images from single short- exposure images without additional information.
    Type: Application
    Filed: August 4, 2020
    Publication date: November 26, 2020
    Inventors: Yilin Wang, Zhe Lin, Zhaowen Wang, Xin Lu, Xiaohui Shen, Chih-Yao Hsieh
  • Patent number: 10846870
    Abstract: Joint training technique for depth map generation implemented by depth prediction system as part of a computing device is described. The depth prediction system is configured to generate a candidate feature map from features extracted from training digital images, generate a candidate segmentation map and a candidate depth map from the generated candidate feature map, and jointly train portions of the depth prediction system using a loss function. Consequently, depth prediction system is able to generate a depth map that identifies depths of objects using ordinal depth information and accurately delineates object boundaries within a single digital image.
    Type: Grant
    Filed: November 29, 2018
    Date of Patent: November 24, 2020
    Assignee: Adobe Inc.
    Inventors: Jianming Zhang, Zhe Lin, Xiaohui Shen, Oliver Wang, Lijun Wang
  • Patent number: 10839575
    Abstract: Certain embodiments involve using an image completion neural network to perform user-guided image completion. For example, an image editing application accesses an input image having a completion region to be replaced with new image content. The image editing application also receives a guidance input that is applied to a portion of a completion region. The image editing application provides the input image and the guidance input to an image completion neural network that is trained to perform image-completion operations using guidance input. The image editing application produces a modified image by replacing the completion region of the input image with the new image content generated with the image completion network. The image editing application outputs the modified image having the new image content.
    Type: Grant
    Filed: March 15, 2018
    Date of Patent: November 17, 2020
    Assignee: ADOBE INC.
    Inventors: Zhe Lin, Xin Lu, Xiaohui Shen, Jimei Yang, Jiahui Yu
  • Publication number: 20200349189
    Abstract: Compositing aware digital image search techniques and systems are described that leverage machine learning. In one example, a compositing aware image search system employs a two-stream convolutional neural network (CNN) to jointly learn feature embeddings from foreground digital images that capture a foreground object and background digital images that capture a background scene. In order to train models of the convolutional neural networks, triplets of training digital images are used. Each triplet may include a positive foreground digital image and a positive background digital image taken from the same digital image. The triplet also contains a negative foreground or background digital image that is dissimilar to the positive foreground or background digital image that is also included as part of the triplet.
    Type: Application
    Filed: July 15, 2020
    Publication date: November 5, 2020
    Applicant: Adobe Inc.
    Inventors: Xiaohui Shen, Zhe Lin, Kalyan Krishna Sunkavalli, Hengshuang Zhao, Brian Lynn Price
  • Publication number: 20200342576
    Abstract: Digital image completion by learning generation and patch matching jointly is described. Initially, a digital image having at least one hole is received. This holey digital image is provided as input to an image completer formed with a dual-stage framework that combines a coarse image neural network and an image refinement network. The coarse image neural network generates a coarse prediction of imagery for filling the holes of the holey digital image. The image refinement network receives the coarse prediction as input, refines the coarse prediction, and outputs a filled digital image having refined imagery that fills these holes. The image refinement network generates refined imagery using a patch matching technique, which includes leveraging information corresponding to patches of known pixels for filtering patches generated based on the coarse prediction. Based on this, the image completer outputs the filled digital image with the refined imagery.
    Type: Application
    Filed: July 14, 2020
    Publication date: October 29, 2020
    Applicant: Adobe Inc.
    Inventors: Zhe Lin, Xin Lu, Xiaohui Shen, Jimei Yang, Jiahui Yu
  • Patent number: 10810721
    Abstract: Digital image defect identification and correction techniques are described. In one example, a digital medium environment is configured to identify and correct a digital image defect through identification of a defect in a digital image using machine learning. The identification includes generating a plurality of defect type scores using a plurality of defect type identification models, as part of machine learning, for a plurality of different defect types and determining the digital image includes the defect based on the generated plurality of defect type scores. A correction is generated for the identified defect and the digital image is output as included the generated correction.
    Type: Grant
    Filed: March 14, 2017
    Date of Patent: October 20, 2020
    Assignee: Adobe Inc.
    Inventors: Radomir Mech, Ning Yu, Xiaohui Shen, Zhe Lin
  • Patent number: 10810707
    Abstract: Techniques of generating depth-of-field blur effects on digital images by digital effect generation system of a computing device are described. The digital effect generation system is configured to generate depth-of-field blur effects on objects based on focal depth value that defines a depth plane in the digital image and a aperture value that defines an intensity of blur effect applied to the digital image. The digital effect generation system is also configured to improve the accuracy with which depth-of-field blur effects are generated by performing up-sampling operations and implementing a unique focal loss algorithm that minimizes the focal loss within digital images effectively.
    Type: Grant
    Filed: November 29, 2018
    Date of Patent: October 20, 2020
    Assignee: Adobe Inc.
    Inventors: Jianming Zhang, Zhe Lin, Xiaohui Shen, Oliver Wang, Lijun Wang
  • Patent number: 10783622
    Abstract: The present disclosure relates to training and utilizing an image exposure transformation network to generate a long-exposure image from a single short-exposure image (e.g., still image). In various embodiments, the image exposure transformation network is trained using adversarial learning, long-exposure ground truth images, and a multi-term loss function. In some embodiments, the image exposure transformation network includes an optical flow prediction network and/or an appearance guided attention network. Trained embodiments of the image exposure transformation network generate realistic long-exposure images from single short-exposure images without additional information.
    Type: Grant
    Filed: April 25, 2018
    Date of Patent: September 22, 2020
    Assignee: ADOBE INC.
    Inventors: Yilin Wang, Zhe Lin, Zhaowen Wang, Xin Lu, Xiaohui Shen, Chih-Yao Hsieh
  • Patent number: 10776671
    Abstract: Techniques are disclosed for blur classification. The techniques utilize an image content feature map, a blur map, and an attention map, thereby combining low-level blur estimation with a high-level understanding of important image content in order to perform blur classification. The techniques allow for programmatically determining if blur exists in an image, and determining what type of blur it is (e.g., high blur, low blur, middle or neutral blur, or no blur). According to one example embodiment, if blur is detected, an estimate of spatially-varying blur amounts is performed and blur desirability is categorized in terms of image quality.
    Type: Grant
    Filed: May 25, 2018
    Date of Patent: September 15, 2020
    Assignee: Adobe Inc.
    Inventors: Zhe Lin, Xiaohui Shen, Shanghang Zhang, Radomir Mech