Abstract: Certain aspects and features of this disclosure relate to rendering images using target-augmented material maps. In one example, a graphics imaging application is loaded with a scene and an input material map, as well as a file for a target image. A stored, material generation prior is accessed by the graphics imaging application. This prior, as an example, is based on a pre-trained, generative adversarial network (GAN). An input material appearance from the input material map is encoded to produce a projected latent vector. The value for the projected latent vector is optimized to produce the material map that is used to render the scene, producing a material map augmented by a realistic target material appearance.

Abstract: Embodiments are disclosed for optimizing a material graph for replicating a material of the target image. Embodiments include receiving a target image and a material graph to be optimized for replicating a material of the target image. Embodiments include identifying a non-differentiable node of the material graph, the non-differentiable node including a set of input parameters. Embodiments include selecting a differentiable proxy from a library of the selected differentiable proxy is trained to replicate an output of the identified non-differentiable node. Embodiments include generating an optimized input parameters for the identified non-differentiable node using the corresponding trained neural network and the target image. Embodiments include replacing the set of input parameters of the non-differentiable node of the material graph with the optimized input parameters.

Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure include an image processing apparatus configured to efficiently perform texture synthesis (e.g., increase the size of, or extend, texture in an input image while preserving a natural appearance of the synthesized texture pattern in the modified output image). In some aspects, the image processing apparatus implements an attention mechanism with a multi-stage attention model where different stages (e.g., different transformer blocks) progressively refine image feature patch mapping at different scales, while utilizing repetitive patterns in texture images to enable network generalization. One or more embodiments of the disclosure include skip connections and convolutional layers (e.g., between transformer block stages) that combine high-frequency and low-frequency features from different transformer stages and unify attention to micro-structures, meso-structures and macro-structures.

Abstract: A method of image processing is described. The method comprises receiving a set of at least three images of an object including at least two linearly-polarized images and at least one color image, wherein the three images have the same view of the object and are acquired under the same illumination condition in which either diffuse polarization or specular polarization dominates in surface reflectance, and wherein a set of Stokes parameters s0, s1 and s2 is determinable from the at least three images.

Abstract: A method of image processing is described. The method comprises receiving a set of at least three images of an object including at least two linearly-polarized images and at least one color image, wherein the three images have the same view of the object and are acquired under the same illumination condition in which either diffuse polarization or specular polarization dominates in surface reflectance, and wherein a set of Stokes parameters s0, s1 and s2 is determinable from the at least three images.