Abstract: A method including capturing a first plurality of images that include a foreground object and a background, capturing a second plurality of images that include the background, generating an alpha matte based on the first plurality of images and the second plurality of images using a trained machine learned model trained using a loss function configured to cause the trained machine learned model to learn high-frequency details of the foreground object, generating a foreground object image based on the first plurality of images and the second plurality of images using the trained machine learned model, and synthesizing an image including the foreground object image and a second background scene using the alpha matte.

Abstract: A method including determining a viewpoint, generating a first image using an image generator, the first image including an object in a first orientation based on the viewpoint, modifying the image generator based on a second orientation of the object, and generating a second image based on the first image using the modified image generator.

Abstract: Techniques are disclosed for generating photorealistic images of objects, such as heads, from multiple viewpoints. In some embodiments, a morphable radiance field (MoRF) model that generates images of heads includes an identity model that maps an identifier (ID) code associated with a head into two codes: a deformation ID code encoding a geometric deformation from a canonical head geometry, and a canonical ID code encoding a canonical appearance within a shape-normalized space. The MoRF model also includes a deformation field model that maps a world space position to a shape-normalized space position based on the deformation ID code. Further, the MoRF model includes a canonical neural radiance field (NeRF) model that includes a density multi-layer perceptron (MLP) branch, a diffuse MLP branch, and a specular MLP branch that output densities, diffuse colors, and specular colors, respectively. The MoRF model can be used to render images of heads from various viewpoints.

Abstract: Techniques are disclosed for generating photorealistic images of objects, such as heads, from multiple viewpoints. In some embodiments, a morphable radiance field (MoRF) model that generates images of heads includes an identity model that maps an identifier (ID) code associated with a head into two codes: a deformation ID code encoding a geometric deformation from a canonical head geometry, and a canonical ID code encoding a canonical appearance within a shape-normalized space. The MoRF model also includes a deformation field model that maps a world space position to a shape-normalized space position based on the deformation ID code. Further, the MoRF model includes a canonical neural radiance field (NeRF) model that includes a density multi-layer perceptron (MLP) branch, a diffuse MLP branch, and a specular MLP branch that output densities, diffuse colors, and specular colors, respectively. The MoRF model can be used to render images of heads from various viewpoints.