Abstract: A method of generating a stylized 3D avatar is provided. The method includes receiving an input image of a user, generating, using a generative adversarial network (GAN) generator, a stylized image, based on the input image, and providing the stylized image to a first model to generate a first plurality of parameters. The first plurality of parameters include a discrete parameter and a continuous parameter. The method further includes providing the stylized image and the first plurality of parameters to a second model that is trained to generate an avatar image, receiving, from the second model, the avatar image, comparing the stylized image to the avatar image, based on a loss function, to determine an error, updating the first model to generate a second plurality of parameters that correspond to the first plurality of parameters, based on the error, and providing the second plurality of parameters as an output.

Abstract: A method of generating a style image is described. The method includes receiving an input image of a subject. The method further includes encoding the input image using a first encoder of a generative adversarial network (GAN) to obtain a first latent code. The method further includes decoding the first latent code using a first decoder of the GAN to obtain a normalized style image of the subject, wherein the GAN is trained using a loss function according to semantic regions of the input image and the normalized style image.

Abstract: Systems and method directed to generating a stylized image are disclosed. In particular, the method includes, in a first data path, (a) applying first stylization to an input image and (b) applying enlargement to the stylized image from (a). The method also includes, in a second data path, (c) applying segmentation to the input image to identify a face region of the input image and generate a mask image, and (d) applying second stylization to an entirety of the input image and inpainting to the identified face region of the stylized image. Machine-assisted blending is performed based on (1) the stylized image after the enlargement from the first data path, (2) the inpainted image from the second data path, and (3) the mask image, in order to obtain a final stylized image.

Abstract: Systems and methods for rendering a translucent object are provided. In one aspect, the system includes a processor coupled to a storage medium that stores instructions, which, upon execution by the processor, cause the processor to receive at least one mesh representing at least one translucent object. For each pixel to be rendered, the processor performs a rasterization-based differentiable rendering of the pixel to be rendered using the at least one mesh and determines a plurality of values for the pixel to be rendered based on the rasterization-based differentiable rendering. The rasterization-based differentiable rendering can include performing a probabilistic rasterization process along with aggregation techniques to compute the plurality of values for the pixel to be rendered. The plurality of values includes a set of color channel values and an opacity channel value. Once values are determined for all pixels, an image can be rendered.

Abstract: Methods and systems for enlarging a stylized region of an image are disclosed that include receiving an input image, generating, using a first generative adversarial network (GAN) generator, a first stylized image, based on the input image, normalizing the input image, generating, using a second generative adversarial network (GAN) generator, a second stylized image, based on the normalized input image, blending the first stylized image and the second stylized image to obtain a third stylized image, and providing the third stylized image as an output.

Abstract: Methods and systems for generating synthetic images based on an input image are described. The method may include receiving an input image; generating, using an encoder, a first latent code vector representation based on the input image; receiving a latent code corresponding to a feature to be added to the input image; modifying the first latent code vector representation based on the latent code corresponding to the feature to be added; generating, by an image decoder, a synthesized image based on the modified first latent code vector representation; identifying, using a landmark detector, one or more landmarks in the base image; identifying, using a landmark detector, one or more landmarks in the synthesized image; determining a measure of similarity between the landmark identified on the base image and the landmark identified in the synthesized image; and discarding the synthesized image based on the comparison.

Abstract: Systems and method directed to an inversion-consistent transfer learning framework for generating portrait stylization using only limited exemplars. In examples, an input image is received and encoded using a variational autoencoder to generate a latent vector. The latent vector may be provided to a generative adversarial network (GAN) generator to generate a stylized image. In examples, the variational autoencoder is trained using a plurality of images while keeping the weights of a pre-trained GAN generator fixed, where the pre-trained GAN generator acts as a decoder for the encoder. In other examples, a multi-path attribute aware generator is trained using a plurality of exemplar images and learning transfer using the pre-trained GAN generator.

Abstract: Systems and methods directed to controlling the similarity between stylized portraits and an original photo are described. In examples, an input image is received and encoded using a variational autoencoder to generate a latent vector. The latent vector may be blended with latent vectors that best represent a face in the original user portrait image. The resulting blended latent vector may be provided to a generative adversarial network (GAN) generator to generate a controlled stylized image. In examples, one or more layers of the stylized GAN generator may be swapped with one or more layers of the original GAN generator. Accordingly, a user can interactively determine how much stylization vs. personalization should be included in a resulting stylized portrait.

Abstract: Systems and method directed to generating a stylized image are disclosed. In particular, the method includes, in a first data path, (a) applying first stylization to an input image and (b) applying enlargement to the stylized image from (a). The method also includes, in a second data path, (c) applying segmentation to the input image to identify a face region of the input image and generate a mask image, and (d) applying second stylization to an entirety of the input image and inpainting to the identified face region of the stylized image. Machine-assisted blending is performed based on (1) the stylized image after the enlargement from the first data path, (2) the inpainted image from the second data path, and (3) the mask image, in order to obtain a final stylized image.

Abstract: Systems and method directed to an inversion-consistent transfer learning framework for generating portrait stylization using only limited exemplars. In examples, an input image is received and encoded using a variational autoencoder to generate a latent vector. The latent vector may be provided to a generative adversarial network (GAN) generator to generate a stylized image. In examples, the variational autoencoder is trained using a plurality of images while keeping the weights of a pre-trained GAN generator fixed, where the pre-trained GAN generator acts as a decoder for the encoder. In other examples, a multi-path attribute aware generator is trained using a plurality of exemplar images and learning transfer using the pre-trained GAN generator.