Patents by Inventor Richard Zhang
Richard Zhang has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20230360376Abstract: Semantic fill techniques are described that support generating fill and editing images from semantic inputs. A user input, for example, is received by a semantic fill system that indicates a selection of a first region of a digital image and a corresponding semantic label. The user input is utilized by the semantic fill system to generate a guidance attention map of the digital image. The semantic fill system leverages the guidance attention map to generate a sparse attention map of a second region of the digital image. A semantic fill of pixels is generated for the first region based on the semantic label and the sparse attention map. The edited digital image is displayed in a user interface.Type: ApplicationFiled: May 16, 2022Publication date: November 9, 2023Applicant: Adobe Inc.Inventors: Tobias Hinz, Taesung Park, Richard Zhang, Matthew David Fisher, Difan Liu, Evangelos Kalogerakis
-
Publication number: 20230342893Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for combining digital images. In particular, in one or more embodiments, the disclosed systems combine latent codes of a source digital image and a target digital image utilizing a blending network to determine a combined latent encoding and generate a combined digital image from the combined latent encoding utilizing a generative neural network. In some embodiments, the disclosed systems determine an intersection face mask between the source digital image and the combined digital image utilizing a face segmentation network and combine the source digital image and the combined digital image utilizing the intersection face mask to generate a blended digital image.Type: ApplicationFiled: April 21, 2022Publication date: October 26, 2023Inventors: Tobias Hinz, Shabnam Ghadar, Richard Zhang, Ratheesh Kalarot, Jingwan Lu, Elya Shechtman
-
Publication number: 20230316591Abstract: Techniques for identity preserved controllable facial image manipulation are described that support generation of a manipulated digital image based on a facial image and a render image. For instance, a facial image depicting a facial representation of an individual is received as input. A feature space including an identity parameter and at least one other visual parameter is extracted from the facial image. An editing module edits one or more of the visual parameters and preserves the identity parameter. A renderer generates a render image depicting a morphable model reconstruction of the facial image based on the edit. The render image and facial image are encoded, and a generator of a neural network is implemented to generate a manipulated digital image based on the encoded facial image and the encoded render image.Type: ApplicationFiled: March 31, 2022Publication date: October 5, 2023Applicant: Adobe Inc.Inventors: Zhixin Shu, Zhe Lin, Yuchen Liu, Yijun Li, Richard Zhang
-
Publication number: 20230316606Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for latent-based editing of digital images using a generative neural network. In particular, in one or more embodiments, the disclosed systems perform latent-based editing of a digital image by mapping a feature tensor and a set of style vectors for the digital image into a joint feature style space. In one or more implementations, the disclosed systems apply a joint feature style perturbation and/or modification vectors within the joint feature style space to determine modified style vectors and a modified feature tensor. Moreover, in one or more embodiments the disclosed systems generate a modified digital image utilizing a generative neural network from the modified style vectors and the modified feature tensor.Type: ApplicationFiled: March 21, 2022Publication date: October 5, 2023Inventors: Hui Qu, Baldo Faieta, Cameron Smith, Elya Shechtman, Jingwan Lu, Ratheesh Kalarot, Richard Zhang, Saeid Motiian, Shabnam Ghadar, Wei-An Lin
-
Patent number: 11763495Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and efficiently modifying a generative adversarial neural network using few-shot adaptation to generate digital images corresponding to a target domain while maintaining diversity of a source domain and realism of the target domain. In particular, the disclosed systems utilize a generative adversarial neural network with parameters learned from a large source domain. The disclosed systems preserve relative similarities and differences between digital images in the source domain using a cross-domain distance consistency loss. In addition, the disclosed systems utilize an anchor-based strategy to encourage different levels or measures of realism over digital images generated from latent vectors in different regions of a latent space.Type: GrantFiled: January 29, 2021Date of Patent: September 19, 2023Assignee: Adobe Inc.Inventors: Utkarsh Ojha, Yijun Li, Richard Zhang, Jingwan Lu, Elya Shechtman, Alexei A. Efros
-
Patent number: 11762951Abstract: Embodiments are disclosed for generative image congealing which provides an unsupervised learning technique that learns transformations of real data to improve the image quality of GANs trained using that image data. In particular, in one or more embodiments, the disclosed systems and methods comprise generating, by a spatial transformer network, an aligned real image for a real image from an unaligned real dataset, providing, by the spatial transformer network, the aligned real image to an adversarial discrimination network to determine if the aligned real image resembles aligned synthetic images generated by a generator network, and training, by a training manager, the spatial transformer network to learn updated transformations based on the determination of the adversarial discrimination network.Type: GrantFiled: November 18, 2020Date of Patent: September 19, 2023Assignee: Adobe Inc.Inventors: Elya Shechtman, William Peebles, Richard Zhang, Jun-Yan Zhu, Alyosha Efros
-
Publication number: 20230289970Abstract: In implementations of systems for image inversion using multiple latent spaces, a computing device implements an inversion system to generate a segment map that segments an input digital image into a first image region and a second image region and assigns the first image region to a first latent space and the second image region to a second latent space that corresponds to a layer of a convolutional neural network. An inverted latent representation of the input digital image is computed using a binary mask for the second image region. The inversion system modifies the inverted latent representation of the input digital image using an edit direction vector that corresponds to a visual feature. An output digital image is generated that depicts a reconstruction of the input digital image having the visual feature based on the modified inverted latent representation of the input digital image.Type: ApplicationFiled: March 14, 2022Publication date: September 14, 2023Applicant: Adobe Inc.Inventors: Gaurav Parmar, Krishna Kumar Singh, Yijun Li, Richard Zhang, Jingwan Lu
-
Publication number: 20230274535Abstract: An image generation system enables user input during the process of training a generative model to influence the model's ability to generate new images with desired visual features. A source generative model for a source domain is fine-tuned using training images in a target domain to provide an adapted generative model for the target domain. Interpretable factors are determined for the source generative model and the adapted generative model. A user interface is provided that enables a user to select one or more interpretable factors. The user-selected interpretable factor(s) are used to generate a user-adapted generative model, for instance, by using a loss function based on the user-selected interpretable factor(s). The user-adapted generative model can be used to create new images in the target domain.Type: ApplicationFiled: February 25, 2022Publication date: August 31, 2023Inventors: Yijun Li, Utkarsh Ojha, Richard Zhang, Jingwan Lu, Elya Shechtman, Alexei A. Efros
-
Publication number: 20230260175Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating digital images depicting photorealistic scenes utilizing a digital image collaging neural network. For example, the disclosed systems utilize a digital image collaging neural network having a particular architecture for disentangling generation of scene layouts and pixel colors for different regions of a digital image. In some cases, the disclosed systems break down the process of generating a collage digital into generating images representing different regions such as a background and a foreground to be collaged into a final result. For example, utilizing the digital image collaging neural network, the disclosed systems determine scene layouts and pixel colors for both foreground digital images and background digital images to ultimately collage the foreground and background together into a collage digital image depicting a real-world scene.Type: ApplicationFiled: February 14, 2022Publication date: August 17, 2023Inventors: Nadav Epstein, Alexei A. Efros, Taesung Park, Richard Zhang, Elya Shechtman
-
Publication number: 20230245363Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and flexibly generating modified digital images utilizing a novel swapping autoencoder that incorporates scene layout. In particular, the disclosed systems can receive a scene layout map that indicates or defines locations for displaying specific digital content within a digital image. In addition, the disclosed systems can utilize the scene layout map to guide combining portions of digital image latent code to generate a modified digital image with a particular textural appearance and a particular geometric structure defined by the scene layout map. Additionally, the disclosed systems can utilize a scene layout map that defines a portion of a digital image to modify by, for instance, adding new digital content to the digital image, and can generate a modified digital image depicting the new digital content.Type: ApplicationFiled: April 10, 2023Publication date: August 3, 2023Inventors: Taesung Park, Alexei A. Efros, Elya Shechtman, Richard Zhang, Junyan Zhu
-
Patent number: 11625875Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and flexibly generating modified digital images utilizing a novel swapping autoencoder that incorporates scene layout. In particular, the disclosed systems can receive a scene layout map that indicates or defines locations for displaying specific digital content within a digital image. In addition, the disclosed systems can utilize the scene layout map to guide combining portions of digital image latent code to generate a modified digital image with a particular textural appearance and a particular geometric structure defined by the scene layout map. Additionally, the disclosed systems can utilize a scene layout map that defines a portion of a digital image to modify by, for instance, adding new digital content to the digital image, and can generate a modified digital image depicting the new digital content.Type: GrantFiled: November 6, 2020Date of Patent: April 11, 2023Assignee: Adobe Inc.Inventors: Taesung Park, Alexei A. Efros, Elya Shechtman, Richard Zhang, Junyan Zhu
-
Publication number: 20230102055Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating a modified digital image from extracted spatial and global codes. For example, the disclosed systems can utilize a global and spatial autoencoder to extract spatial codes and global codes from digital images. The disclosed systems can further utilize the global and spatial autoencoder to generate a modified digital image by combining extracted spatial and global codes in various ways for various applications such as style swapping, style blending, and attribute editing.Type: ApplicationFiled: November 22, 2022Publication date: March 30, 2023Inventors: Taesung Park, Richard Zhang, Oliver Wang, Junyan Zhu, Jingwan Lu, Elya Shechtman, Alexei A. Efros
-
Patent number: 11615292Abstract: A target image is projected into a latent space of generative model by determining a latent vector by applying a gradient-free technique and a class vector by applying a gradient-based technique. An image is generated from the latent and class vectors, and a loss function is used to determine a loss between the target image and the generated image. This determining of the latent vector and the class vector, generating an image, and using the loss function is repeated until a loss condition is satisfied. In response to the loss condition being satisfied, the latent and class vectors that resulted in the loss condition being satisfied are identified as the final latent and class vectors, respectively. The final latent and class vectors are provided to the generative model and multiple weights of the generative model are adjusted to fine-tune the generative model.Type: GrantFiled: August 31, 2022Date of Patent: March 28, 2023Assignee: Adobe Inc.Inventors: Richard Zhang, Sylvain Philippe Paris, Junyan Zhu, Aaron Phillip Hertzmann, Jacob Minyoung Huh
-
Publication number: 20230055204Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that utilize one or more stages of a two-stage image colorization neural network to colorize or re-colorize digital images. In one or more embodiments, the disclosed system generates a color digital image from a grayscale digital image by utilizing a colorization neural network. Additionally, the disclosed system receives one or more inputs indicating local hints comprising one or more color selections to apply to one or more objects of the color digital image. The disclosed system then utilizes a re-colorization neural network to generate a modified digital image from the color digital image by modifying one or more colors of the object(s) based on the luminance channel, color channels, and selected color(s).Type: ApplicationFiled: August 18, 2021Publication date: February 23, 2023Inventors: Adrian-Stefan Ungureanu, Ionut Mironica, Richard Zhang
-
Patent number: 11562166Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating shift-resilient neural network outputs based on utilizing a dense pooling layer, a low-pass filter layer, and a downsampling layer of a neural network. For example, the disclosed systems can generate a pooled feature map utilizing a dense pooling layer to densely pool feature values extracted from an input. The disclosed systems can further apply a low-pass filter to the pooled feature map to generate a shift-adaptive feature map. In addition, the disclosed systems can downsample the shift-adaptive feature map utilizing a downsampling layer. Based on the downsampled, shift-adaptive feature map, the disclosed systems can generate shift-resilient neural network outputs such as digital image classifications.Type: GrantFiled: May 21, 2021Date of Patent: January 24, 2023Assignee: Adobe Inc.Inventor: Richard Zhang
-
Patent number: 11544880Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating a modified digital image from extracted spatial and global codes. For example, the disclosed systems can utilize a global and spatial autoencoder to extract spatial codes and global codes from digital images. The disclosed systems can further utilize the global and spatial autoencoder to generate a modified digital image by combining extracted spatial and global codes in various ways for various applications such as style swapping, style blending, and attribute editing.Type: GrantFiled: May 14, 2020Date of Patent: January 3, 2023Assignee: Adobe Inc.Inventors: Taesung Park, Richard Zhang, Oliver Wang, Junyan Zhu, Jingwan Lu, Elya Shechtman, Alexei A Efros
-
Publication number: 20220414431Abstract: A target image is projected into a latent space of generative model by determining a latent vector by applying a gradient-free technique and a class vector by applying a gradient-based technique. An image is generated from the latent and class vectors, and a loss function is used to determine a loss between the target image and the generated image. This determining of the latent vector and the class vector, generating an image, and using the loss function is repeated until a loss condition is satisfied. In response to the loss condition being satisfied, the latent and class vectors that resulted in the loss condition being satisfied are identified as the final latent and class vectors, respectively. The final latent and class vectors are provided to the generative model and multiple weights of the generative model are adjusted to fine-tune the generative model.Type: ApplicationFiled: August 31, 2022Publication date: December 29, 2022Applicant: Adobe Inc.Inventors: Richard Zhang, Sylvain Philippe Paris, Junyan Zhu, Aaron Phillip Hertzmann, Jacob Minyoung Huh
-
Patent number: 11514632Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that utilize a contrastive perceptual loss to modify neural networks for generating synthetic digital content items. For example, the disclosed systems generate a synthetic digital content item based on a guide input to a generative neural network. The disclosed systems utilize an encoder neural network to generate encoded representations of the synthetic digital content item and a corresponding ground-truth digital content item. Additionally, the disclosed systems sample patches from the encoded representations of the encoded digital content items and then determine a contrastive loss based on the perceptual distances between the patches in the encoded representations. Furthermore, the disclosed systems jointly update the parameters of the generative neural network and the encoder neural network utilizing the contrastive loss.Type: GrantFiled: November 6, 2020Date of Patent: November 29, 2022Assignee: Adobe Inc.Inventors: Bryan Russell, Taesung Park, Richard Zhang, Junyan Zhu, Alexander Andonian
-
Patent number: 11508148Abstract: The present disclosure relates to systems, computer-implemented methods, and non-transitory computer readable medium for automatically transferring makeup from a reference face image to a target face image using a neural network trained using semi-supervised learning. For example, the disclosed systems can receive, at a neural network, a target face image and a reference face image, where the target face image is selected by a user via a graphical user interface (GUI) and the reference face image has makeup. The systems transfer, by the neural network, the makeup from the reference face image to the target face image, where the neural network is trained to transfer the makeup from the reference face image to the target face image using semi-supervised learning. The systems output for display the makeup on the target face image.Type: GrantFiled: March 18, 2020Date of Patent: November 22, 2022Assignee: ADOBE INC.Inventors: Yijun Li, Zhifei Zhang, Richard Zhang, Jingwan Lu
-
Patent number: 11478804Abstract: A sprinkler assembly including a pressure regulator and a flow stop element combined in a single assembly. In one embodiment, the sprinkler includes a sprinkler body in fluid communication with a water supply to provide water to the sprinkler, a riser movably mounted in the sprinkler body and in fluid communication with the sprinkler body such that the riser rises up and out of the sprinkler body when water is provided to the sprinkler, a nozzle assembly mounted on a top end of the riser and in fluid communication with the riser; and a regulation element mounted in the riser and in contact with a portion of the nozzle assembly. The nozzle assembly holds the regulation member in an open position to allow flow of water to the nozzle assembly and to maintain a substantially constant water pressure in the nozzle assembly.Type: GrantFiled: January 9, 2019Date of Patent: October 25, 2022Assignee: K-Rain Manufacturing Corp.Inventors: Carl L. C. Kah, Jr., Carl L. C. Kah, III, Weiming Feng, Richard Zhang