Patents by Inventor Jianming Zhang
Jianming Zhang has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 12367561Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for panoptically guiding digital image inpainting utilizing a panoptic inpainting neural network. In some embodiments, the disclosed systems utilize a panoptic inpainting neural network to generate an inpainted digital image according to panoptic segmentation map that defines pixel regions corresponding to different panoptic labels. In some cases, the disclosed systems train a neural network utilizing a semantic discriminator that facilitates generation of digital images that are realistic while also conforming to a semantic segmentation. The disclosed systems generate and provide a panoptic inpainting interface to facilitate user interaction for inpainting digital images. In certain embodiments, the disclosed systems iteratively update an inpainted digital image based on changes to a panoptic segmentation map.Type: GrantFiled: October 3, 2022Date of Patent: July 22, 2025Assignee: Adobe Inc.Inventors: Zhe Lin, Haitian Zheng, Elya Shechtman, Jianming Zhang, Jingwan Lu, Ning Xu, Qing Liu, Scott Cohen, Sohrab Amirghodsi
-
Patent number: 12367585Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for utilizing machine learning models to generate refined depth maps of digital images utilizing digital segmentation masks. In particular, in one or more embodiments, the disclosed systems generate a depth map for a digital image utilizing a depth estimation machine learning model, determine a digital segmentation mask for the digital image, and generate a refined depth map from the depth map and the digital segmentation mask utilizing a depth refinement machine learning model. In some embodiments, the disclosed systems generate first and second intermediate depth maps using the digital segmentation mask and an inverse digital segmentation mask and merger the first and second intermediate depth maps to generate the refined depth map.Type: GrantFiled: April 12, 2022Date of Patent: July 22, 2025Assignee: Adobe Inc.Inventors: Jianming Zhang, Soo Ye Kim, Simon Niklaus, Yifei Fan, Su Chen, Zhe Lin
-
Patent number: 12367586Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for panoptically guiding digital image inpainting utilizing a panoptic inpainting neural network. In some embodiments, the disclosed systems utilize a panoptic inpainting neural network to generate an inpainted digital image according to panoptic segmentation map that defines pixel regions corresponding to different panoptic labels. In some cases, the disclosed systems train a neural network utilizing a semantic discriminator that facilitates generation of digital images that are realistic while also conforming to a semantic segmentation. The disclosed systems generate and provide a panoptic inpainting interface to facilitate user interaction for inpainting digital images. In certain embodiments, the disclosed systems iteratively update an inpainted digital image based on changes to a panoptic segmentation map.Type: GrantFiled: October 3, 2022Date of Patent: July 22, 2025Assignee: Adobe Inc.Inventors: Zhe Lin, Haitian Zheng, Elya Shechtman, Jianming Zhang, Jingwan Lu, Ning Xu, Qing Liu, Scott Cohen, Sohrab Amirghodsi
-
Patent number: 12367562Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for panoptically guiding digital image inpainting utilizing a panoptic inpainting neural network. In some embodiments, the disclosed systems utilize a panoptic inpainting neural network to generate an inpainted digital image according to panoptic segmentation map that defines pixel regions corresponding to different panoptic labels. In some cases, the disclosed systems train a neural network utilizing a semantic discriminator that facilitates generation of digital images that are realistic while also conforming to a semantic segmentation. The disclosed systems generate and provide a panoptic inpainting interface to facilitate user interaction for inpainting digital images. In certain embodiments, the disclosed systems iteratively update an inpainted digital image based on changes to a panoptic segmentation map.Type: GrantFiled: October 3, 2022Date of Patent: July 22, 2025Assignee: Adobe Inc.Inventors: Zhe Lin, Haitian Zheng, Elya Shechtman, Jianming Zhang, Jingwan Lu, Ning Xu, Qing Liu, Scott Cohen, Sohrab Amirghodsi
-
Publication number: 20250232575Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates object masks for digital objects portrayed in digital images utilizing a detection-masking neural network pipeline. In particular, in one or more embodiments, the disclosed systems utilize detection heads of a neural network to detect digital objects portrayed within a digital image. In some cases, each detection head is associated with one or more digital object classes that are not associated with the other detection heads. Further, in some cases, the detection heads implement multi-scale synchronized batch normalization to normalize feature maps across various feature levels. The disclosed systems further utilize a masking head of the neural network to generate one or more object masks for the detected digital objects. In some cases, the disclosed systems utilize post-processing techniques to filter out low-quality masks.Type: ApplicationFiled: April 7, 2025Publication date: July 17, 2025Inventors: Jason Wen Yong Kuen, Su Chen, Scott Cohen, Zhe Lin, Zijun Wei, Jianming Zhang
-
Publication number: 20250232526Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for estimating the three-dimensional geometry of an object in a digital image by modeling the ground, object, and camera simultaneously. In particular, in one or more embodiments, the disclosed systems receive a two-dimensional digital image portraying an object. Further, the systems generate, utilizing a dense representation neural network, an estimate of an object-ground relationship of the object portrayed in the two-dimensional digital image and an estimate of camera parameters for the two-dimensional digital image. Additionally, the systems generate, utilizing a perspective field guided pixel height reprojection model, one or more of a three-dimensional point cloud or a depth map of the object from the estimated object-ground relationship and the estimated camera parameters.Type: ApplicationFiled: January 11, 2024Publication date: July 17, 2025Inventors: Yunze Man, Jianming Zhang
-
Patent number: 12361512Abstract: This disclosure describes one or more implementations of a digital image semantic layout manipulation system that generates refined digital images resembling the style of one or more input images while following the structure of an edited semantic layout. For example, in various implementations, the digital image semantic layout manipulation system builds and utilizes a sparse attention warped image neural network to generate high-resolution warped images and a digital image layout neural network to enhance and refine the high-resolution warped digital image into a realistic and accurate refined digital image.Type: GrantFiled: April 11, 2023Date of Patent: July 15, 2025Assignee: Adobe Inc.Inventors: Haitian Zheng, Zhe Lin, Jingwan Lu, Scott Cohen, Jianming Zhang, Ning Xu
-
Patent number: 12347080Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.Type: GrantFiled: March 27, 2023Date of Patent: July 1, 2025Assignee: Adobe Inc.Inventors: Krishna Kumar Singh, Yijun Li, Jingwan Lu, Duygu Ceylan Aksit, Yangtuanfeng Wang, Jimei Yang, Tobias Hinz, Qing Liu, Jianming Zhang, Zhe Lin
-
Patent number: 12299844Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately, efficiently, and flexibly generating harmonized digital images utilizing a self-supervised image harmonization neural network. In particular, the disclosed systems can implement, and learn parameters for, a self-supervised image harmonization neural network to extract content from one digital image (disentangled from its appearance) and appearance from another from another digital image (disentangled from its content). For example, the disclosed systems can utilize a dual data augmentation method to generate diverse triplets for parameter learning (including input digital images, reference digital images, and pseudo ground truth digital images), via cropping a digital image with perturbations using three-dimensional color lookup tables (“LUTs”).Type: GrantFiled: February 13, 2024Date of Patent: May 13, 2025Assignee: Adobe Inc.Inventors: He Zhang, Yifan Jiang, Yilin Wang, Jianming Zhang, Kalyan Sunkavalli, Sarah Kong, Su Chen, Sohrab Amirghodsi, Zhe Lin
-
Patent number: 12288279Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For instance, in one or more embodiments, the disclosed systems generate utilizing a segmentation neural network, an object mask for each object of a plurality of objects of a digital image. The disclosed systems detect a first user interaction with an object in the digital image displayed via a graphical user interface. The disclosed systems surface, via the graphical user interface, the object mask for the object in response to the first user interaction. The disclosed systems perform an object-aware modification of the digital image in response to a second user interaction with the object mask for the object.Type: GrantFiled: November 23, 2022Date of Patent: April 29, 2025Assignee: Adobe Inc.Inventors: Jonathan Brandt, Scott Cohen, Zhe Lin, Zhihong Ding, Darshan Prasad, Matthew Joss, Celso Gomes, Jianming Zhang, Olena Soroka, Klaas Stoeckmann, Michael Zimmermann, Thomas Muehrke
-
Patent number: 12272127Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates object masks for digital objects portrayed in digital images utilizing a detection-masking neural network pipeline. In particular, in one or more embodiments, the disclosed systems utilize detection heads of a neural network to detect digital objects portrayed within a digital image. In some cases, each detection head is associated with one or more digital object classes that are not associated with the other detection heads. Further, in some cases, the detection heads implement multi-scale synchronized batch normalization to normalize feature maps across various feature levels. The disclosed systems further utilize a masking head of the neural network to generate one or more object masks for the detected digital objects. In some cases, the disclosed systems utilize post-processing techniques to filter out low-quality masks.Type: GrantFiled: January 31, 2022Date of Patent: April 8, 2025Assignee: Adobe Inc.Inventors: Jason Wen Yong Kuen, Su Chen, Scott Cohen, Zhe Lin, Zijun Wei, Jianming Zhang
-
Publication number: 20250103604Abstract: A computing device can receive a query that identifies a set of data to be processed and determine that a portion of the set of data resides in an external data system. The query system can request data identifiers associated with data objects of the set of data from the external data system and communicate the data identifiers to a data queue. The computing device can instruct one or more search nodes to retrieve the identifiers from the data queue. The search nodes can use the data identifiers to retrieve data objects from the external data system and process the data objects according to instructions received from the computing device. The search nodes can provide results of the processing to the computing device.Type: ApplicationFiled: June 20, 2024Publication date: March 27, 2025Inventors: Alexandros Batsakis, Nitilaksha Satyaveera Halakatti, Ningxuan He, Prem Kumar Jayaraj, Manuel Gregorio Martinez, Balaji Rao, Jianming Zhang, Steve Yu Zhang
-
Patent number: 12260557Abstract: An image processing system generates an image mask from an image. The image is processed by an object detector to identify a region having an object, and the region is classified based on an object type of the object. A masking pipeline is selected from a number of masking pipelines based on the classification of the region. The region is processed using the masking pipeline to generate a region mask. An image mask for the image is generated using the region mask.Type: GrantFiled: June 13, 2022Date of Patent: March 25, 2025Assignee: adobe inc.Inventors: Zijun Wei, Yilin Wang, Jianming Zhang, He Zhang
-
Patent number: 12260530Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.Type: GrantFiled: March 27, 2023Date of Patent: March 25, 2025Assignee: Adobe Inc.Inventors: Krishna Kumar Singh, Yijun Li, Jingwan Lu, Duygu Ceylan Aksit, Yangtuanfeng Wang, Jimei Yang, Tobias Hinz, Qing Liu, Jianming Zhang, Zhe Lin
-
Publication number: 20250086849Abstract: Embodiments of the present disclosure include obtaining a text prompt describing an element, layout information indicating a target region for the element, and a precision level corresponding to the element. Some embodiments generate a text feature pyramid based on the text prompt, the layout information, and the precision level, wherein the text feature pyramid comprises a plurality of text feature maps at a plurality of scales, respectively. Then, an image is generated based on the text feature pyramid. In some cases, the image includes an object corresponding to the element of the text prompt at the target region. Additionally, a shape of the object corresponds to a shape of the target region based on the precision level.Type: ApplicationFiled: September 8, 2023Publication date: March 13, 2025Inventors: Yu Zeng, Zhe Lin, Jianming Zhang, Qing Liu, Jason Wen Yong Kuen, John Philip Collomosse
-
Patent number: 12236640Abstract: Systems and methods for image dense field based view calibration are provided. In one embodiment, an input image is applied to a dense field machine learning model that generates a vertical vector dense field (VVF) and a latitude dense field (LDF) from the input image. The VVF comprises a vertical vector of a projected vanishing point direction for each of the pixels of the input image. The latitude dense field (LDF) comprises a projected latitude value for the pixels of the input image. A dense field map for the input image comprising the VVF and the LDF can be directly or indirectly used for a variety of image processing manipulations. The VVF and LDF can be optionally used to derive traditional camera calibration parameters from uncontrolled images that have undergone undocumented or unknown manipulations.Type: GrantFiled: March 28, 2022Date of Patent: February 25, 2025Assignee: Adobe Inc.Inventors: Jianming Zhang, Linyi Jin, Kevin Matzen, Oliver Wang, Yannick Hold-Geoffroy
-
Publication number: 20250054116Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that generate inpainted digital images utilizing a cascaded modulation inpainting neural network. For example, the disclosed systems utilize a cascaded modulation inpainting neural network that includes cascaded modulation decoder layers. For example, in one or more decoder layers, the disclosed systems start with global code modulation that captures the global-range image structures followed by an additional modulation that refines the global predictions. Accordingly, in one or more implementations, the image inpainting system provides a mechanism to correct distorted local details. Furthermore, in one or more implementations, the image inpainting system leverages fast Fourier convolutions block within different resolution layers of the encoder architecture to expand the receptive field of the encoder and to allow the network encoder to better capture global structure.Type: ApplicationFiled: October 28, 2024Publication date: February 13, 2025Inventors: Haitian Zheng, Zhe Lin, Jingwan Lu, Scott Cohen, Elya Shechtman, Connelly Barnes, Jianming Zhang, Ning Xu, Sohrab Amirghodsi
-
Patent number: 12223661Abstract: Systems and methods provide editing operations in a smart editing system that may generate a focal point within a mask of an object for each frame of a video segment and perform editing effects on the frames of the video segment to quickly provide users with natural video editing effects. An eye-gaze network may produce a hotspot map of predicted focal points in a video frame. These predicted focal points may then be used by a gaze-to-mask network to determine objects in the image and generate an object mask for each of the detected objects. This process may then be repeated to effectively track the trajectory of objects and object focal points in videos. Based on the determined trajectory of an object in a video clip and editing parameters, the editing engine may produce editing effects relative to an object for the video clip.Type: GrantFiled: May 3, 2022Date of Patent: February 11, 2025Assignee: ADOBE INC.Inventors: Lu Zhang, Jianming Zhang, Zhe Lin, Radomir Mech
-
Patent number: 12223439Abstract: Systems and methods for multi-modal representation learning are described. One or more embodiments provide a visual representation learning system trained using machine learning techniques. For example, some embodiments of the visual representation learning system are trained using cross-modal training tasks including a combination of intra-modal and inter-modal similarity preservation objectives. In some examples, the training tasks are based on contrastive learning techniques.Type: GrantFiled: March 3, 2021Date of Patent: February 11, 2025Assignee: ADOBE INC.Inventors: Xin Yuan, Zhe Lin, Jason Wen Yong Kuen, Jianming Zhang, Yilin Wang, Ajinkya Kale, Baldo Faieta
-
Patent number: D1070778Type: GrantFiled: January 13, 2023Date of Patent: April 15, 2025Assignee: DONGGUAN LUXSHARE TECHNOLOGIES CO., LTDInventors: Jianming Zhang, Xiaogang Liu, Rongzhe Guo