Patents by Inventor Jianming Zhang

Jianming Zhang has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20230155172
    Abstract: The present disclosure belongs to the field of energy materials, and relates to a preparation method of a solid electrolyte, in particular to a method for forming a membrane by using an electrolyte to activate a porous powder material prepared by in-situ polymerization of a polymer on the surfaces of cellulose nanocrystals, and then hot-pressing. According to the technical solution of the present disclosure, cellulose nanocrystals are used as templates, the powder material with a porous structure is prepared by in-situ polymerization growth of the polymer on the surfaces of the cellulose nanocrystals, a small amount of electrolyte is used to activate the powder, and the solid electrolyte is prepared by hot-pressing membrane formation. The solid electrolyte prepared by the present disclosure has excellent electrochemical performance and mechanical performance, and a broad application prospect.
    Type: Application
    Filed: January 4, 2023
    Publication date: May 18, 2023
    Applicant: Qingdao University of Science and Technology
    Inventors: Yuwei CHEN, Jiying YANG, Junbo CHE, Quan WANG, Jinjin HU, Jianwen WANG, Yu LI, Jianming ZHANG
  • Patent number: 11651477
    Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing a plurality of neural networks in a multi-branch pipeline to generate image masks for digital images. Specifically, the disclosed system can classify a digital image as a portrait or a non-portrait image. Based on classifying a portrait image, the disclosed system can utilize separate neural networks to generate a first mask portion for a portion of the digital image including a defined boundary region and a second mask portion for a portion of the digital image including a blended boundary region. The disclosed system can generate the mask portion for the blended boundary region by utilizing a trimap generation neural network to automatically generate a trimap segmentation including the blended boundary region. The disclosed system can then merge the first mask portion and the second mask portion to generate an image mask for the digital image.
    Type: Grant
    Filed: August 7, 2020
    Date of Patent: May 16, 2023
    Assignee: Adobe Inc.
    Inventors: He Zhang, Seyed Morteza Safdarnejad, Yilin Wang, Zijun Wei, Jianming Zhang, Salil Tambe, Brian Price
  • Publication number: 20230128792
    Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates object masks for digital objects portrayed in digital images utilizing a detection-masking neural network pipeline. In particular, in one or more embodiments, the disclosed systems utilize detection heads of a neural network to detect digital objects portrayed within a digital image. In some cases, each detection head is associated with one or more digital object classes that are not associated with the other detection heads. Further, in some cases, the detection heads implement multi-scale synchronized batch normalization to normalize feature maps across various feature levels. The disclosed systems further utilize a masking head of the neural network to generate one or more object masks for the detected digital objects. In some cases, the disclosed systems utilize post-processing techniques to filter out low-quality masks.
    Type: Application
    Filed: January 31, 2022
    Publication date: April 27, 2023
    Inventors: Jason Wen Yong Kuen, Su Chen, Scott Cohen, Zhe Lin, Zijun Wei, Jianming Zhang
  • Patent number: 11636570
    Abstract: This disclosure describes one or more implementations of a digital image semantic layout manipulation system that generates refined digital images resembling the style of one or more input images while following the structure of an edited semantic layout. For example, in various implementations, the digital image semantic layout manipulation system builds and utilizes a sparse attention warped image neural network to generate high-resolution warped images and a digital image layout neural network to enhance and refine the high-resolution warped digital image into a realistic and accurate refined digital image.
    Type: Grant
    Filed: April 1, 2021
    Date of Patent: April 25, 2023
    Assignee: Adobe Inc.
    Inventors: Haitian Zheng, Zhe Lin, Jingwan Lu, Scott Cohen, Jianming Zhang, Ning Xu
  • Publication number: 20230122623
    Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and flexibly generating harmonized digital images utilizing an object-to-object harmonization neural network. For example, the disclosed systems implement, and learn parameters for, an object-to-object harmonization neural network to combine a style code from a reference object with features extracted from a target object. Indeed, the disclosed systems extract a style code from a reference object utilizing a style encoder neural network. In addition, the disclosed systems generate a harmonized target object by applying the style code of the reference object to a target object utilizing an object-to-object harmonization neural network.
    Type: Application
    Filed: October 18, 2021
    Publication date: April 20, 2023
    Inventors: He Zhang, Jeya Maria Jose Valanarasu, Jianming Zhang, Jose Ignacio Echevarria Vallespi, Kalyan Sunkavalli, Yilin Wang, Yinglan Ma, Zhe Lin, Zijun Wei
  • Publication number: 20230123658
    Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generate a height map for a digital object portrayed in a digital image and further utilizes the height map to generate a shadow for the digital object. Indeed, in one or more embodiments, the disclosed systems generate (e.g., utilizing a neural network) a height map that indicates the pixels heights for pixels of a digital object portrayed in a digital image. The disclosed systems utilize the pixel heights, along with lighting information for the digital image, to determine how the pixels of the digital image project to create a shadow for the digital object. Further, in some implementations, the disclosed systems utilize the determined shadow projections to generate (e.g., utilizing another neural network) a soft shadow for the digital object. Accordingly, in some cases, the disclosed systems modify the digital image to include the shadow.
    Type: Application
    Filed: October 15, 2021
    Publication date: April 20, 2023
    Inventors: Yifan Liu, Jianming Zhang, He Zhang, Elya Shechtman, Zhe Lin
  • Publication number: 20230105994
    Abstract: In implementations of resource-aware training for neural network, one or more computing devices of a system implement an architecture optimization module for monitoring parameter utilization while training a neural network. Dead neurons of the neural network are identified as having activation scales less than a threshold. Neurons with activation scales greater than or equal to the threshold are identified as survived neurons. The dead neurons are converted to reborn neurons by adding the dead neurons to layers of the neural network having the survived neurons. The reborn neurons are prevented from connecting to the survived neurons for training the reborn neurons.
    Type: Application
    Filed: December 9, 2022
    Publication date: April 6, 2023
    Applicant: Adobe Inc.
    Inventors: Zhe Lin, Siyuan Qiao, Jianming Zhang
  • Patent number: 11605156
    Abstract: Methods and systems are provided for accurately filling holes, regions, and/or portions of images using iterative image inpainting. In particular, iterative inpainting utilize a confidence analysis of predicted pixels determined during the iterations of inpainting. For instance, a confidence analysis can provide information that can be used as feedback to progressively fill undefined pixels that comprise the holes, regions, and/or portions of an image where information for those respective pixels is not known. To allow for accurate image inpainting, one or more neural networks can be used. For instance, a coarse result neural network (e.g., a GAN comprised of a generator and a discriminator) and a fine result neural network (e.g., a GAN comprised of a generator and two discriminators).
    Type: Grant
    Filed: July 14, 2022
    Date of Patent: March 14, 2023
    Assignee: ADOBE INC.
    Inventors: Zhe Lin, Yu Zeng, Jimei Yang, Jianming Zhang, Elya Shechtman
  • Patent number: 11605168
    Abstract: Techniques are disclosed for characterizing and defining the location of a copy space in an image. A methodology implementing the techniques according to an embodiment includes applying a regression convolutional neural network (CNN) to an image. The regression CNN is configured to predict properties of the copy space such as size and type (natural or manufactured). The prediction is conditioned on a determination of the presence of the copy space in the image. The method further includes applying a segmentation CNN to the image. The segmentation CNN is configured to generate one or more pixel-level masks to define the location of copy spaces in the image, whether natural or manufactured, or to define the location of a background region of the image. The segmentation CNN may include a first stage comprising convolutional layers and a second stage comprising pairs of boundary refinement layers and bilinear up-sampling layers.
    Type: Grant
    Filed: March 29, 2021
    Date of Patent: March 14, 2023
    Assignee: Adobe Inc.
    Inventors: Mingyang Ling, Alex Filipkowski, Zhe Lin, Jianming Zhang, Samarth Gulati
  • Publication number: 20230076982
    Abstract: A preparation method includes 1) dispersing cellulose nanocrystal in water and adjusting pH to 7; 2) adding carboxylate to the aqueous dispersion of the step 1), and stirring until uniform; and 3) adding a monomer and a ceric ammonium nitrate initiator to the system of the step 2), reacting for 0.5-3 h to obtain a precipitate, and subjecting the precipitate to suction filtration, washing, and drying to obtain the cellulose nanocrystal powder. By adding a small amount of carboxylate into a cellulose nanocrystal graft polymer modification system initiated by ceric ammonium nitrate, hydrolysis of cerium ions can be inhibited through complexation of the carboxylate to the cerium ions which play an initiating role in ceric ammonium nitrate, so that ceric ammonium nitrate can initiate the polymerization reaction under acid-free conditions, thereby achieving polymerization of polyvinyl acetate monomer on the surface of cellulose nanocrystals.
    Type: Application
    Filed: October 18, 2022
    Publication date: March 9, 2023
    Inventors: Jianming ZHANG, Yunxiao LIU, Yongxin DUAN, Lijuan ZHOU, Yunjie LU, Xinran LIU
  • Patent number: 11593948
    Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods that utilize a progressive refinement network to refine alpha mattes generated utilizing a mask-guided matting neural network. In particular, the disclosed systems can use the matting neural network to process a digital image and a coarse guidance mask to generate alpha mattes at discrete neural network layers. In turn, the disclosed systems can use the progressive refinement network to combine alpha mattes and refine areas of uncertainty. For example, the progressive refinement network can combine a core alpha matte corresponding to more certain core regions of a first alpha matte and a boundary alpha matte corresponding to uncertain boundary regions of a second, higher resolution alpha matte. Based on the combination of the core alpha matte and the boundary alpha matte, the disclosed systems can generate a final alpha matte for use in image matting processes.
    Type: Grant
    Filed: February 17, 2021
    Date of Patent: February 28, 2023
    Assignee: Adobe Inc.
    Inventors: Qihang Yu, Jianming Zhang, He Zhang, Yilin Wang, Zhe Lin, Ning Xu
  • Patent number: 11574142
    Abstract: The technology described herein is directed to a reinforcement learning based framework for training a natural media agent to learn a rendering policy without human supervision or labeled datasets. The reinforcement learning based framework feeds the natural media agent a training dataset to implicitly learn the rendering policy by exploring a canvas and minimizing a loss function. Once trained, the natural media agent can be applied to any reference image to generate a series (or sequence) of continuous-valued primitive graphic actions, e.g., sequence of painting strokes, that when rendered by a synthetic rendering environment on a canvas, reproduce an identical or transformed version of the reference image subject to limitations of an action space and the learned rendering policy.
    Type: Grant
    Filed: July 30, 2020
    Date of Patent: February 7, 2023
    Assignee: Adobe Inc.
    Inventors: Zhe Lin, Xihui Liu, Quan Hung Tran, Jianming Zhang, Handong Zhao
  • Publication number: 20230037282
    Abstract: Systems and methods for image editing are described. Embodiments of the present disclosure provide an image editing system for performing image object replacement or image region replacement (e.g., an image editing system for replacing an object or region of an image with an object or region from another image). For example, the image editing system may replace a sky portion of an image with a more desirable sky portion from a different replacement image. According to some embodiments described herein, real-time color harmonization based on the visible sky region may be used to produce more natural colorization. In some examples, horizon-aware sky alignment and placement with advanced padding may also be used. For example, the horizons of the original image and the replacement image may be automatically detected and aligned, and color harmonization may be performed based on the aligned images.
    Type: Application
    Filed: April 16, 2021
    Publication date: February 2, 2023
    Inventors: Alan Erickson, Kalyan Sunkavalli, I-Ming Pao, Guotong Feng, Jianming Zhang, Frederick Mandia
  • Patent number: 11568544
    Abstract: The present disclosure relates to utilizing a neural network having a two-stream encoder architecture to accurately generate composite digital images that realistically portray a foreground object from one digital image against a scene from another digital image. For example, the disclosed systems can utilize a foreground encoder of the neural network to identify features from a foreground image and further utilize a background encoder to identify features from a background image. The disclosed systems can then utilize a decoder to fuse the features together and generate a composite digital image. The disclosed systems can train the neural network utilizing an easy-to-hard data augmentation scheme implemented via self-teaching. The disclosed systems can further incorporate the neural network within an end-to-end framework for automation of the image composition process.
    Type: Grant
    Filed: September 23, 2021
    Date of Patent: January 31, 2023
    Assignee: Adobe Inc.
    Inventors: Zhe Lin, Jianming Zhang, He Zhang, Federico Perazzi
  • Patent number: 11551093
    Abstract: In implementations of resource-aware training for neural network, one or more computing devices of a system implement an architecture optimization module for monitoring parameter utilization while training a neural network. Dead neurons of the neural network are identified as having activation scales less than a threshold. Neurons with activation scales greater than or equal to the threshold are identified as survived neurons. The dead neurons are converted to reborn neurons by adding the dead neurons to layers of the neural network having the survived neurons. The reborn neurons are prevented from connecting to the survived neurons for training the reborn neurons.
    Type: Grant
    Filed: January 22, 2019
    Date of Patent: January 10, 2023
    Assignee: Adobe Inc.
    Inventors: Zhe Lin, Siyuan Qiao, Jianming Zhang
  • Publication number: 20230005197
    Abstract: The present disclosure provides systems and methods for image editing. Embodiments of the present disclosure provide an image editing system for perform image object replacement or image region replacement (e.g., an image editing system for replacing an object or region of an image with an object or region from another image). For example, the image editing system may replace a sky portion of an image with a more desirable sky portion from a different replacement image. The original image and the replacement image (e.g., the image including a desirable object or region) include layers of masks. A sky from the replacement image may replace the sky of the image to produce an aesthetically pleasing composite image.
    Type: Application
    Filed: March 17, 2021
    Publication date: January 5, 2023
    Inventors: JIANMING ZHANG, Alan Erickson, I-Ming Pao, Guotong Feng, Kalyan Sunkavalli, Frederick Mandia, Hyunghwan Byun, Betty Leong, Meredith Payne Stotzner, Yukie Takahashi, Quynn Megan Le, Sarah Kong
  • Publication number: 20220405899
    Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that utilize color density estimation in a blended boundary region of a digital image to generate an image mask. For example, the disclosed system extracts a foreground region, a background region, and a blended boundary region from a digital image. The disclosed system determines a color histogram—within a color space selected utilizing the foreground region and the background region—for a portion of the background region along an edge of the blended boundary region. Additionally, the disclosed system generates a color density map for the blended boundary region by comparing colors in the blended boundary region to colors in the color histogram of the background band. The disclosed system then generates a final mask for the digital image based on the color density map.
    Type: Application
    Filed: August 30, 2022
    Publication date: December 22, 2022
    Inventor: Jianming Zhang
  • Patent number: 11507800
    Abstract: Semantic segmentation techniques and systems are described that overcome the challenges of limited availability of training data to describe the potentially millions of tags that may be used to describe semantic classes in digital images. In one example, the techniques are configured to train neural networks to leverage different types of training datasets using sequential neural networks and use of vector representations to represent the different semantic classes.
    Type: Grant
    Filed: March 6, 2018
    Date of Patent: November 22, 2022
    Assignee: ADOBE INC.
    Inventors: Zhe Lin, Yufei Wang, Xiaohui Shen, Scott David Cohen, Jianming Zhang
  • Publication number: 20220366546
    Abstract: Methods and systems are provided for accurately filling holes, regions, and/or portions of images using iterative image inpainting. In particular, iterative inpainting utilize a confidence analysis of predicted pixels determined during the iterations of inpainting. For instance, a confidence analysis can provide information that can be used as feedback to progressively fill undefined pixels that comprise the holes, regions, and/or portions of an image where information for those respective pixels is not known. To allow for accurate image inpainting, one or more neural networks can be used. For instance, a coarse result neural network (e.g., a GAN comprised of a generator and a discriminator) and a fine result neural network (e.g., a GAN comprised of a generator and two discriminators).
    Type: Application
    Filed: July 14, 2022
    Publication date: November 17, 2022
    Inventors: Zhe LIN, Yu ZENG, Jimei YANG, Jianming ZHANG, Elya SHECHTMAN
  • Publication number: 20220335671
    Abstract: Systems and methods for image editing are described. Embodiments of the present disclosure provide an image editing system for performing image object replacement or image region replacement (e.g., an image editing system for replacing an object or region of an image with an object or region from another image). For example, the image editing system may replace a sky portion of an image with a more desirable sky portion from a different replacement image. According to some embodiments described herein, real-time color harmonization based on the visible sky region may be used to produce more natural colorization. In some examples, horizon-aware sky alignment and placement with advanced padding may also be used. For example, the horizons of the original image and the replacement image may be automatically detected and aligned, and color harmonization may be performed based on the aligned images.
    Type: Application
    Filed: April 16, 2021
    Publication date: October 20, 2022
    Inventors: Alan Erickson, Kalyan Sunkavalli, I-Ming Pao, Guotong Feng, Jianming Zhang, Frederick Mandia