Abstract: A method, apparatus, non-transitory computer readable medium, and system for image processing include obtaining an input image and a modification prompt, where the input image depicts an object with a first attribute and the modification prompt describes a modification from the first attribute to a second attribute different from the first attribute, encoding the modification prompt to obtain a text embedding, where the text embedding represents the modification in an embedding space, and generating a modified image based on the input image and the text embedding, where the modified image depicts the object with the second attribute.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for latent-based editing of digital images using a generative neural network. In particular, in one or more embodiments, the disclosed systems perform latent-based editing of a digital image by mapping a feature tensor and a set of style vectors for the digital image into a joint feature style space. In one or more implementations, the disclosed systems apply a joint feature style perturbation and/or modification vectors within the joint feature style space to determine modified style vectors and a modified feature tensor. Moreover, in one or more embodiments the disclosed systems generate a modified digital image utilizing a generative neural network from the modified style vectors and the modified feature tensor.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates spatial-temporal positional encodings. For example, the disclosed systems generate a noised token from adding noise to an embedding of a frame of a video. Moreover, the disclosed systems generate a spatial embedding for a token using a centered two-dimensional coordinate map. Further, the disclosed systems generate temporal embeddings for the token from a timestamp of the token in the video. Further, the disclosed systems generate a denoised token by removing noise from the noised token according to spatial-temporal positional encodings that include the spatial embedding and the temporal embedding via a diffusion model. Additionally, the disclosed systems modify parameters of the diffusion model based on a comparison of the denoised token and the token.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates an image or a video from a text prompt. For example, the disclosed systems receive a text prompt and generates text tokens from the text prompt. Moreover, the disclosed systems generate combined tokens by combining the text tokens with noised tokens. Further, the disclosed systems generate denoised tokens by removing noise from noised tokens in a manner that incorporates a context indicated by the text tokens and further generates an image or video from the denoised tokens.

Abstract: A method, apparatus, non-transitory computer readable medium, and system for image processing include obtaining a text prompt and a guidance parameter, where the text prompt describes an image element and the guidance parameter indicates a level of guidance intensity for the text prompt, computing guidance features based on the text prompt and the guidance parameter, and generating a synthetic image that depicts the image element based on the text prompt and the guidance features.

Abstract: Systems and methods use a non-linear latent filter neural network for editing an image. An image editing system trains a first neural network by minimizing a loss based upon a predicted attribute value for a target attribute in a training image. The image editing system obtains a latent space representation of an input image to be edited and a target attribute value for the target attribute in the input image. The image editing system provides the latent space representation and the target attribute value as input to the trained first neural network for modifying the target attribute in the input image to generate a modified latent space representation of the input image. The image editing system provides the modified latent space representation as input to a second neural network to generate an output image with a modification to the target attribute corresponding to the target attribute value.

Abstract: Systems and methods train an encoder neural network for fast and accurate projection into the latent space of a Generative Adversarial Network (GAN). The encoder is trained by providing an input training image to the encoder and producing, by the encoder, a latent space representation of the input training image. The latent space representation is provided as input to the GAN to generate a generated training image. A latent code is sampled from a latent space associated with the GAN and the sampled latent code is provided as input to the GAN. The GAN generates a synthetic training image based on the sampled latent code. The sampled latent code is provided as input to the encoder to produce a synthetic training code. The encoder is updated by minimizing a loss between the generated training image and the input training image, and the synthetic training code and the sampled latent code.

Abstract: An improved system architecture uses a Generative Adversarial Network (GAN) including a specialized generator neural network to generate multiple resolution output images. The system produces a latent space representation of an input image. The system generates a first output image at a first resolution by providing the latent space representation of the input image as input to a generator neural network comprising an input layer, an output layer, and a plurality of intermediate layers and taking the first output image from an intermediate layer, of the plurality of intermediate layers of the generator neural network. The system generates a second output image at a second resolution different from the first resolution by providing the latent space representation of the input image as input to the generator neural network and taking the second output image from the output layer of the generator neural network.

Abstract: Systems and methods train and apply a specialized encoder neural network for fast and accurate projection into the latent space of a Generative Adversarial Network (GAN). The specialized encoder neural network includes an input layer, a feature extraction layer, and a bottleneck layer positioned after the feature extraction layer. The projection process includes providing an input image to the encoder and producing, by the encoder, a latent space representation of the input image. Producing the latent space representation includes extracting a feature vector from the feature extraction layer, providing the feature vector lo the bottleneck layer as input, and producing the latent space representation as output. The latent space representation produced by the encoder is provided as input to the GAN, which generates an output image based upon the latent space representation.

Abstract: Methods, systems, and non-transitory computer readable media are disclosed for intelligently enhancing details in edited images. The disclosed system iteratively updates residual detail latent code for segments in edited images where detail has been lost through the editing process. More particularly, the disclosed system enhances an edited segment in an edited image based on details in a detailed segment of an image. Additionally, the disclosed system may utilize a detail neural network encoder to project the detailed segment and a corresponding segment of the edited image into a residual detail latent code. In some embodiments, the disclosed system generates a refined edited image based on the residual detail latent code and a latent vector of the edited image.

Abstract: An item recommendation system receives a set of recommendable items and a request to select, from the set of recommendable items, a contrast group. The item recommendation system selects a contrast group from the set of recommendable items by applying a image modification model to the set of recommendable items. The image modification model includes an item selection model configured to determine an unbiased conversion rate for each item of the set of recommendable items and select a recommended item from the set of recommendable items having a greatest unbiased conversion rate. The image modification model includes a contrast group selection model configured to select, for the recommended item, a contrast group comprising the recommended item and one or more contrast items. The item recommendation system transmits the contrast group responsive to the request.

Abstract: One or more aspects of a method, apparatus, and non-transitory computer readable medium include obtaining an input latent vector for an image generation network and a target lighting representation. A modified latent vector is generated based on the input latent vector and the target lighting representation, and an image generation network generates an image based on the modified latent vector using.

Abstract: Systems and methods for editing multiple attributes of an image are described. Embodiments are configured to receive input comprising an image of a face and a target value of an attribute of the face to be modified; encode the image using an encoder of an image generation neural network to obtain an image embedding; and generate a modified image of the face having the target value of the attribute based on the image embedding using a decoder of the image generation neural network. The image generation neural network is trained using a plurality of training images generated by a separate training image generation neural network, and the plurality of training images include a first synthetic image having a first value of the attribute and a second synthetic image depicting a same face as the first synthetic image with a second value of the attribute.

Abstract: Systems and methods generate paired image data comprising synthesized eyeglass reflections and use the paired image data to train a machine learning model for reflection removal. A training dataset is generated that includes image pairs. Each image pair comprises a first version of a face image with eyeglasses not having a reflection and a second version of the face image with eyeglasses having a reflection. A first image pair in the training dataset is generated by: obtaining a first face image with eyeglasses not having a reflection, obtaining a reflection image, and generating a composite image using the first face image and the reflection image. Once generated, the training dataset is used to train a machine learning model to provide a trained machine learning model that performs reflection removal on input face images with eyeglass reflections.

Abstract: The present disclosure describes systems, methods, and non-transitory computer readable media for detecting user interactions to edit a digital image from a client device and modify the digital image for the client device by using a web-based intermediary that modifies a latent vector of the digital image and an image modification neural network to generate a modified digital image from the modified latent vector. In response to user interaction to modify a digital image, for instance, the disclosed systems modify a latent vector extracted from the digital image to reflect the requested modification. The disclosed systems further use a latent vector stream renderer (as an intermediary device) to generate an image delta that indicates a difference between the digital image and the modified digital image. The disclosed systems then provide the image delta as part of a digital stream to a client device to quickly render the modified digital image.

Abstract: Systems and methods dynamically adjust an available range for editing an attribute in an image. An image editing system computes a metric for an attribute in an input image as a function of a latent space representation of the input image and a filtering vector for editing the input image. The image editing system compares the metric to a threshold. If the metric exceeds the threshold, then the image editing system selects a first range for editing the attribute in the input image. If the metric does not exceed the threshold, a second range is selected. The image editing system causes display of a user interface for editing the input image comprising an interface element for editing the attribute within the selected range.

Abstract: Systems and methods for facial image generation are described. One aspect of the systems and methods includes receiving an image depicting a face, wherein the face has an identity non-related attribute and a first identity-related attribute; encoding the image to obtain an identity non-related attribute vector in an identity non-related attribute vector space, wherein the identity non-related attribute vector represents the identity non-related attribute; selecting an identity-related vector from an identity-related vector space, wherein the identity-related vector represents a second identity-related attribute different from the first identity-related attribute; generating a modified latent vector in a latent vector space based on the identity non-related attribute vector and the identity-related vector; and generating a modified image based on the modified latent vector, wherein the modified image depicts a face that has the identity non-related attribute and the second identity-related attribute.

Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure encode features of a source image to obtain a source appearance encoding that represents inherent attributes of a face in the source image; encode features of a target image to obtain a target non-appearance encoding that represents contextual attributes of the target image; combine the source appearance encoding and the target non-appearance encoding to obtain combined image features; and generate a modified target image based on the combined image features, wherein the modified target image includes the inherent attributes of the face in the source image together with the contextual attributes of the target image.

Abstract: Systems and methods for facial image generation are described. One aspect of the systems and methods includes receiving an image depicting a face, wherein the face has an identity non-related attribute and a first identity-related attribute; encoding the image to obtain an identity non-related attribute vector in an identity non-related attribute vector space, wherein the identity non-related attribute vector represents the identity non-related attribute; selecting an identity-related vector from an identity-related vector space, wherein the identity-related vector represents a second identity-related attribute different from the first identity-related attribute; generating a modified latent vector in a latent vector space based on the identity non-related attribute vector and the identity-related vector; and generating a modified image based on the modified latent vector, wherein the modified image depicts a face that has the identity non-related attribute and the second identity-related attribute.

Abstract: An improved system architecture uses a pipeline including a Generative Adversarial Network (GAN) including a generator neural network and a discriminator neural network to generate an image. An input image in a first domain and information about a target domain are obtained. The domains correspond to image styles. An initial latent space representation of the input image is produced by encoding the input image. An initial output image is generated by processing the initial latent space representation with the generator neural network. Using the discriminator neural network, a score is computed indicating whether the initial output image is in the target domain. A loss is computed based on the computed score. The loss is minimized to compute an updated latent space representation. The updated latent space representation is processed with the generator neural network to generate an output image in the target domain.