Patents by Inventor I-Ming Pao

I-Ming Pao has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11776184
    Abstract: The present disclosure provides systems and methods for image editing. Embodiments of the present disclosure provide an image editing system for perform image object replacement or image region replacement (e.g., an image editing system for replacing an object or region of an image with an object or region from another image). For example, the image editing system may replace a sky portion of an image with a more desirable sky portion from a different replacement image. The original image and the replacement image (e.g., the image including a desirable object or region) include layers of masks. A sky from the replacement image may replace the sky of the image to produce an aesthetically pleasing composite image.
    Type: Grant
    Filed: March 17, 2021
    Date of Patent: October 3, 2023
    Assignee: ADOBE, INC.
    Inventors: Jianming Zhang, Alan Erickson, I-Ming Pao, Guotong Feng, Kalyan Sunkavalli, Frederick Mandia, Hyunghwan Byun, Betty Leong, Meredith Payne Stotzner, Yukie Takahashi, Quynn Megan Le, Sarah Kong
  • Patent number: 11645788
    Abstract: Systems and methods for image editing are described. Embodiments of the present disclosure provide an image editing system for performing image object replacement or image region replacement (e.g., an image editing system for replacing an object or region of an image with an object or region from another image). For example, the image editing system may replace a sky portion of an image with a more desirable sky portion from a different replacement image. According to some embodiments described herein, thumbnails, region location information, and image metadata from multiple preset images can be stored together and loaded for presentation and selection of a preset image for replacing a region of an image. Once a preset image (e.g., an image with a replacement sky) is selected, a high-resolution version of the image can be loaded and used to generate a composite image.
    Type: Grant
    Filed: April 19, 2021
    Date of Patent: May 9, 2023
    Assignee: ADOBE INC.
    Inventors: Guotong Feng, Alan Erickson, I-Ming Pao, Betty Leong, Hyunghwan Byun
  • Publication number: 20230037282
    Abstract: Systems and methods for image editing are described. Embodiments of the present disclosure provide an image editing system for performing image object replacement or image region replacement (e.g., an image editing system for replacing an object or region of an image with an object or region from another image). For example, the image editing system may replace a sky portion of an image with a more desirable sky portion from a different replacement image. According to some embodiments described herein, real-time color harmonization based on the visible sky region may be used to produce more natural colorization. In some examples, horizon-aware sky alignment and placement with advanced padding may also be used. For example, the horizons of the original image and the replacement image may be automatically detected and aligned, and color harmonization may be performed based on the aligned images.
    Type: Application
    Filed: April 16, 2021
    Publication date: February 2, 2023
    Inventors: Alan Erickson, Kalyan Sunkavalli, I-Ming Pao, Guotong Feng, Jianming Zhang, Frederick Mandia
  • Publication number: 20230037276
    Abstract: Systems and methods for image editing are described. Embodiments of the present disclosure provide an image editing system for performing image object replacement or image region replacement (e.g., an image editing system for replacing an object or region of an image with an object or region from another image). For example, the image editing system may replace a sky portion of an image with a more desirable sky portion from a different replacement image. According to some embodiments described herein, thumbnails, region location information, and image metadata from multiple preset images can be stored together and loaded for presentation and selection of a preset image for replacing a region of an image. Once a preset image (e.g., an image with a replacement sky) is selected, a high-resolution version of the image can be loaded and used to generate a composite image.
    Type: Application
    Filed: April 19, 2021
    Publication date: February 2, 2023
    Inventors: Guotong Feng, Alan Erickson, I-Ming Pao, Betty Leong, Hyunghwan Byun
  • Publication number: 20230005197
    Abstract: The present disclosure provides systems and methods for image editing. Embodiments of the present disclosure provide an image editing system for perform image object replacement or image region replacement (e.g., an image editing system for replacing an object or region of an image with an object or region from another image). For example, the image editing system may replace a sky portion of an image with a more desirable sky portion from a different replacement image. The original image and the replacement image (e.g., the image including a desirable object or region) include layers of masks. A sky from the replacement image may replace the sky of the image to produce an aesthetically pleasing composite image.
    Type: Application
    Filed: March 17, 2021
    Publication date: January 5, 2023
    Inventors: JIANMING ZHANG, Alan Erickson, I-Ming Pao, Guotong Feng, Kalyan Sunkavalli, Frederick Mandia, Hyunghwan Byun, Betty Leong, Meredith Payne Stotzner, Yukie Takahashi, Quynn Megan Le, Sarah Kong
  • Publication number: 20220335659
    Abstract: Systems and methods for image editing are described. Embodiments of the present disclosure provide an image editing system for performing image object replacement or image region replacement (e.g., an image editing system for replacing an object or region of an image with an object or region from another image). For example, the image editing system may replace a sky portion of an image with a more desirable sky portion from a different replacement image. According to some embodiments described herein, thumbnails, region location information, and image metadata from multiple preset images can be stored together and loaded for presentation and selection of a preset image for replacing a region of an image. Once a preset image (e.g., an image with a replacement sky) is selected, a high-resolution version of the image can be loaded and used to generate a composite image.
    Type: Application
    Filed: April 19, 2021
    Publication date: October 20, 2022
    Inventors: Guotong Feng, Alan Erickson, I-Ming Pao, Betty Leong, Hyunghwan Byun
  • Publication number: 20220335671
    Abstract: Systems and methods for image editing are described. Embodiments of the present disclosure provide an image editing system for performing image object replacement or image region replacement (e.g., an image editing system for replacing an object or region of an image with an object or region from another image). For example, the image editing system may replace a sky portion of an image with a more desirable sky portion from a different replacement image. According to some embodiments described herein, real-time color harmonization based on the visible sky region may be used to produce more natural colorization. In some examples, horizon-aware sky alignment and placement with advanced padding may also be used. For example, the horizons of the original image and the replacement image may be automatically detected and aligned, and color harmonization may be performed based on the aligned images.
    Type: Application
    Filed: April 16, 2021
    Publication date: October 20, 2022
    Inventors: Alan Erickson, Kalyan Sunkavalli, I-Ming Pao, Guotong Feng, Jianming Zhang, Frederick Mandia
  • Patent number: 11468614
    Abstract: Methods and systems are provided for presenting and using multiple masks based on a segmented image in editing the image. In particular, multiple masks can be presented to a user using a graphical user interface for easy selection and utilization in the editing of an image. The graphical user interface can include a display configured to display an image, a mask zone configured to display segmentations of the image using masks, and an edit zone configured to display edits to the image. Upon receiving segmentation for the image, the masks can be displayed in the mask zone where the masks are based on a selected segmentation detail level.
    Type: Grant
    Filed: March 28, 2021
    Date of Patent: October 11, 2022
    Assignee: Adobe Inc.
    Inventors: Sarah Aye Kong, I-Ming Pao, Hyunghwan Byun
  • Publication number: 20220301243
    Abstract: The present disclosure provides systems and methods for image editing. Embodiments of the present disclosure provide an image editing system for perform image object replacement or image region replacement (e.g., an image editing system for replacing an object or region of an image with an object or region from another image). For example, the image editing system may replace a sky portion of an image with a more desirable sky portion from a different replacement image. The original image and the replacement image (e.g., the image including a desirable object or region) include layers of masks. A sky from the replacement image may replace the sky of the image to produce an aesthetically pleasing composite image.
    Type: Application
    Filed: March 17, 2021
    Publication date: September 22, 2022
    Inventors: JIANMING ZHANG, Alan Erickson, I-Ming Pao, Guotong Feng, Kalyan Sunkavalli, Frederick Mandia, Hyunghwan Byun, Betty Leong, Meredith Payne Stotzner, Yukie Takahashi, Quynn Megan Le, Sarah Kong
  • Publication number: 20220148285
    Abstract: The present disclosure relates to systems, method, and computer readable media that iteratively apply a neural network to a digital image at a reduced resolution to automatically identify pixels of salient objects portrayed within the digital image. For example, the disclosed systems can generate a reduced-resolution digital image from an input digital image and apply a neural network to identify a region corresponding to a salient object. The disclosed systems can then iteratively apply the neural network to additional reduced-resolution digital images (based on the identified region) to generate one or more reduced-resolution segmentation maps that roughly indicate pixels of the salient object. In addition, the systems described herein can perform post-processing based on the reduced-resolution segmentation map(s) and the input digital image to accurately determine pixels that correspond to the salient object.
    Type: Application
    Filed: January 25, 2022
    Publication date: May 12, 2022
    Inventors: I-Ming Pao, Zhe Lin, Sarah Stuckey, Jianming Zhang, Betty Leong
  • Patent number: 11244195
    Abstract: The present disclosure relates to systems, method, and computer readable media that iteratively apply a neural network to a digital image at a reduced resolution to automatically identify pixels of salient objects portrayed within the digital image. For example, the disclosed systems can generate a reduced-resolution digital image from an input digital image and apply a neural network to identify a region corresponding to a salient object. The disclosed systems can then iteratively apply the neural network to additional reduced-resolution digital images (based on the identified region) to generate one or more reduced-resolution segmentation maps that roughly indicate pixels of the salient object. In addition, the systems described herein can perform post-processing based on the reduced-resolution segmentation map(s) and the input digital image to accurately determine pixels that correspond to the salient object.
    Type: Grant
    Filed: May 1, 2018
    Date of Patent: February 8, 2022
    Assignee: Adobe Inc.
    Inventors: I-Ming Pao, Zhe Lin, Sarah Stuckey, Jianming Zhang, Betty Leong
  • Publication number: 20210350504
    Abstract: Methods and systems are provided for generating enhanced image. A neural network system is trained where the training includes training a first neural network that generates enhanced images conditioned on content of an image undergoing enhancement and training a second neural network that designates realism of the enhanced images generated by the first neural network. The neural network system is trained by determine loss and accordingly adjusting the appropriate neural network(s). The trained neural network system is used to generate an enhanced aesthetic image from a selected image where the output enhanced aesthetic image has increased aesthetics when compared to the selected image.
    Type: Application
    Filed: July 19, 2021
    Publication date: November 11, 2021
    Inventors: Xiaohui Shen, Zhe Lin, Xin Lu, Sarah Aye Kong, I-Ming Pao, Yingcong Chen
  • Patent number: 11069030
    Abstract: Methods and systems are provided for generating enhanced image. A neural network system is trained where the training includes training a first neural network that generates enhanced images conditioned on content of an image undergoing enhancement and training a second neural network that designates realism of the enhanced images generated by the first neural network. The neural network system is trained by determine loss and accordingly adjusting the appropriate neural network(s). The trained neural network system is used to generate an enhanced aesthetic image from a selected image where the output enhanced aesthetic image has increased aesthetics when compared to the selected image.
    Type: Grant
    Filed: March 22, 2018
    Date of Patent: July 20, 2021
    Assignee: Adobe, Inc.
    Inventors: Xiaohui Shen, Zhe Lin, Xin Lu, Sarah Aye Kong, I-Ming Pao, Yingcong Chen
  • Publication number: 20210217216
    Abstract: Methods and systems are provided for presenting and using multiple masks based on a segmented image in editing the image. In particular, multiple masks can be presented to a user using a graphical user interface for easy selection and utilization in the editing of an image. The graphical user interface can include a display configured to display an image, a mask zone configured to display segmentations of the image using masks, and an edit zone configured to display edits to the image. Upon receiving segmentation for the image, the masks can be displayed in the mask zone where the masks are based on a selected segmentation detail level.
    Type: Application
    Filed: March 28, 2021
    Publication date: July 15, 2021
    Applicant: Adobe Inc.
    Inventors: Sarah Aye Kong, I-Ming Pao, Hyunghwan Byun
  • Patent number: 11042990
    Abstract: Systems and techniques for automatic object replacement in an image include receiving an original image and a preferred image. The original image is automatically segmented into an original image foreground region and an original image object region. The preferred image is automatically segmented into a preferred image foreground region and a preferred image object region. A composite image is automatically composed by replacing the original image object region with the preferred image object region such that the composite image includes the original image foreground region and the preferred image object region. An attribute of the composite image is automatically adjusted.
    Type: Grant
    Filed: October 31, 2018
    Date of Patent: June 22, 2021
    Assignee: ADOBE INC.
    Inventors: I-Ming Pao, Sarah Aye Kong, Alan Lee Erickson, Kalyan Sunkavalli, Hyunghwan Byun
  • Patent number: 11036915
    Abstract: Embodiments of the present invention are directed at providing a font similarity system. In one embodiment, a new font is detected on a computing device. In response to the detection of the new font, a pre-computed font list is checked to determine whether the new font is included therein. The pre-computed font list including feature representations, generated independently of the computing device, for corresponding fonts. In response to a determination that the new font is absent from the pre-computed font list, a feature representation for the new font is generated. The generated feature representation capable of being utilized for a similarity analysis of the new font. The feature representation is then stored in a supplemental font list to enable identification of one or more fonts installed on the computing device that are similar to the new font. Other embodiments may be described and/or claimed.
    Type: Grant
    Filed: March 10, 2016
    Date of Patent: June 15, 2021
    Assignee: Adobe Inc.
    Inventors: I-Ming Pao, Zhaowen Wang, Hailin Jin, Alan Lee Erickson
  • Patent number: 10977844
    Abstract: Methods and systems are provided for presenting and using multiple masks based on a segmented image in editing the image. In particular, multiple masks can be presented to a user using a graphical user interface for easy selection and utilization in the editing of an image. The graphical user interface can include a display configured to display an image, a mask zone configured to display segmentations of the image using masks, and an edit zone configured to display edits to the image. Upon receiving segmentation for the image, the masks can be displayed in the mask zone where the masks are based on a selected segmentation detail level.
    Type: Grant
    Filed: November 29, 2017
    Date of Patent: April 13, 2021
    Assignee: Adobe Inc.
    Inventors: Sarah Aye Kong, I-Ming Pao, Hyunghwan Byun
  • Patent number: 10783408
    Abstract: Systems and techniques for identification of fonts include receiving a selection of an area of an image including text, where the selection is received from within an application. The selected area of the image is input to a font matching module within the application. The font matching module identifies one or more fonts similar to the text in the selected area using a convolutional neural network. The one or more fonts similar to the text are displayed within the application and the selection and use of the one or more fonts is enabled within the application.
    Type: Grant
    Filed: June 19, 2017
    Date of Patent: September 22, 2020
    Assignee: ADOBE INC.
    Inventors: Zhaowen Wang, Sarah Aye Kong, I-Ming Pao, Hailin Jin, Alan Lee Erickson
  • Patent number: 10692221
    Abstract: A digital medium environment is described to automatically generate a trimap and segment a digital image, independent of any user intervention. An image processing system receives an image and a low-resolution mask for the image, which provides a probability map indicating a likelihood that a pixel in the image mask corresponds to a foreground object in the image. The image processing system analyzes the image to identify content in the image's foreground and background portions, and adaptively generates a trimap for the image based on differences between the identified foreground and background content. By identifying content of the image prior to generating the trimap, the techniques described herein can be applied to a wide range of images, such as images where foreground content is visually similar to background content, and vice versa. Thus, the image processing system can automatically generate trimaps for images having diverse visual characteristics.
    Type: Grant
    Filed: July 13, 2018
    Date of Patent: June 23, 2020
    Assignee: Adobe Inc.
    Inventors: I-Ming Pao, Zhe Lin
  • Publication number: 20200134834
    Abstract: Systems and techniques for automatic object replacement in an image include receiving an original image and a preferred image. The original image is automatically segmented into an original image foreground region and an original image object region. The preferred image is automatically segmented into a preferred image foreground region and a preferred image object region. A composite image is automatically composed by replacing the original image object region with the preferred image object region such that the composite image includes the original image foreground region and the preferred image object region. An attribute of the composite image is automatically adjusted.
    Type: Application
    Filed: October 31, 2018
    Publication date: April 30, 2020
    Inventors: I-Ming Pao, Sarah Aye Kong, Alan Lee Erickson, Kalyan Sunkavalli, Hyunghwan Byun