Abstract: An appearance-responsive material map generation system generates a set of material maps based on the appearance of a material depicted in the source material data. A neural network included in the appearance-responsive material map generation system is trained to identify features of particular source material data, such as features that contribute to a highly realistic appearance of a graphical object rendered with the material depicted in the source material data. In some cases, the trained neural network receives source material data that includes at least one source material map. Based on the features that are identified for the particular source material data, the appearance-responsive material map generation system creates a respective set of appearance-responsive material maps for the particular source material data. In some cases, the appearance-responsive rendering map set is arranged as an inconsistent pyramid of material maps.

Abstract: A modeling system displays a three-dimensional (3D) space including a 3D object including a plurality of points and a cage model of the 3D object including a first configuration of vertices and quad faces. Each of the plurality of points is located at a respective initial location. The modeling system generates cage coordinates for the cage model including a vertex coordinate for each vertex of the cage model and four quad coordinates for each quad face of the cage model corresponding to each corner vertex of the quad. The modeling system deforms, responsive to receiving a request, the cage model to change the first configuration of vertices to a second configuration. The modeling system generates, based on the cage coordinates, the first configuration of vertices, and the second configuration of vertices, an updated 3D object by determining a subsequent location for each of the plurality of points.

Abstract: Aspects and features of the present disclosure provide a direct ray tracing operator with a low memory footprint for surfaces enriched with displacement maps. A graphics editing application can be used to manipulate displayed representations of a 3D object that include surfaces with displacement textures. The application creates an independent map of a displaced surface. The application ray-traces bounding volumes on the fly and uses the intersection of a query ray with a bounding volume to produce rendering information for a displaced surface. The rendering information can be used to generate displaced surfaces for various base surfaces without significant re-computation so that updated images can be rendered quickly, in real time or near real time.

Abstract: Aspects of a system and method for procedural media generation include generating a sequence of operator types using a node generation network; generating a sequence of operator parameters for each operator type of the sequence of operator types using a parameter generation network; generating a sequence of directed edges based on the sequence of operator types using an edge generation network; combining the sequence of operator types, the sequence of operator parameters, and the sequence of directed edges to obtain a procedural media generator, wherein each node of the procedural media generator comprises an operator that includes an operator type from the sequence of operator types, a corresponding sequence of operator parameters, and an input connection or an output connection from the sequence of directed edges that connects the node to another node of the procedural media generator; and generating a media asset using the procedural media generator.

Abstract: An appearance-responsive material map generation system generates a set of material maps based on the appearance of a material depicted in the source material data. A neural network included in the appearance-responsive material map generation system is trained to identify features of particular source material data, such as features that contribute to a highly realistic appearance of a graphical object rendered with the material depicted in the source material data. In some cases, the trained neural network receives source material data that includes at least one source material map. Based on the features that are identified for the particular source material data, the appearance-responsive material map generation system creates a respective set of appearance-responsive material maps for the particular source material data. In some cases, the appearance-responsive rendering map set is arranged as an inconsistent pyramid of material maps.

Abstract: The disclosure describes one or more embodiments of systems, methods, and non-transitory computer-readable media that modify vertex positions of a 3D-object mesh to align the vertices with discontinuities indicated by a sharpness map for a displacement map. For example, the disclosed systems access a sharpness map that encodes discontinuities of a displacement map to preserve sharp features. During sharp tessellation, for a given vertex of the 3D-object mesh, the disclosed systems iterate over neighboring vertices to search for a nearest intersection between one of the incident edges of the vertex and neighboring vertices and a discontinuity line from the sharpness map. Then, the vertex is moved within a threshold distance of the nearest discontinuity line intersection along the incident edge. The disclosed systems similarly reposition other vertices of the 3D-object mesh to generate a modified 3D-object mesh that includes vertices that align with discontinuities present in a displacement map.

Abstract: Aspects of a system and method for procedural media generation include generating a sequence of operator types using a node generation network; generating a sequence of operator parameters for each operator type of the sequence of operator types using a parameter generation network; generating a sequence of directed edges based on the sequence of operator types using an edge generation network; combining the sequence of operator types, the sequence of operator parameters, and the sequence of directed edges to obtain a procedural media generator, wherein each node of the procedural media generator comprises an operator that includes an operator type from the sequence of operator types, a corresponding sequence of operator parameters, and an input connection or an output connection from the sequence of directed edges that connects the node to another node of the procedural media generator; and generating a media asset using the procedural media generator.

Abstract: The present disclosure relates to using end-to-end differentiable pipeline for optimizing parameters of a base procedural material to generate a procedural material corresponding to a target physical material. For example, the disclosed systems can receive a digital image of a target physical material. In response, the disclosed systems can retrieve a differentiable procedural material for use as a base procedural material in response. The disclosed systems can compare a digital image of the base procedural material with the digital image of the target physical material using a loss function, such as a style loss function that compares visual appearance. Based on the determined loss, the disclosed systems can modify the parameters of the base procedural material to determine procedural material parameters for the target physical material. The disclosed systems can generate a procedural material corresponding to the base procedural material using the determined procedural material parameters.

Abstract: Generative shape creation and editing is leveraged in a digital medium environment. An object editor system represents a set of training shapes as sets of visual elements known as “handles,” and converts sets of handles into signed distance field (SDF) representations. A handle processor model is then trained using the SDF representations to enable the handle processor model to generate new shapes that reflect salient visual features of the training shapes. The trained handle processor model, for instance, generates new sets of handles based on salient visual features learned from the training handle set. Thus, utilizing the described techniques, accurate characterizations of a set of shapes can be learned and used to generate new shapes. Further, generated shapes can be edited and transformed in different ways.

Abstract: Certain aspects and features of this disclosure relate to modeling shapes using differentiable, signed distance functions. 3D modeling software can edit a 3D model represented using the differentiable, signed distance functions while displaying the model in a manner that is computing resource efficient and fast. Further, such 3D modeling software can automatically create such an editable 3D model from a reference representation that can be obtained in various ways and stored in a variety of formats. For example, a real-world object can be scanned using LiDAR and a reference representation can be produced from the LiDAR data. Candidate procedural models from a library of curated procedural models are optimized to obtain the best procedural model for editing. A selected procedural model provides an editable, reconstructed shape based on the reference representation of the object.

Abstract: The present disclosure relates to using end-to-end differentiable pipeline for optimizing parameters of a base procedural material to generate a procedural material corresponding to a target physical material. For example, the disclosed systems can receive a digital image of a target physical material. In response, the disclosed systems can retrieve a differentiable procedural material for use as a base procedural material in response. The disclosed systems can compare a digital image of the base procedural material with the digital image of the target physical material using a loss function, such as a style loss function that compares visual appearance. Based on the determined loss, the disclosed systems can modify the parameters of the base procedural material to determine procedural material parameters for the target physical material. The disclosed systems can generate a procedural material corresponding to the base procedural material using the determined procedural material parameters.

Abstract: The disclosure describes one or more embodiments of systems, methods, and non-transitory computer-readable media that modify vertex positions of a 3D-object mesh to align the vertices with discontinuities indicated by a sharpness map for a displacement map. For example, the disclosed systems access a sharpness map that encodes discontinuities of a displacement map to preserve sharp features. During sharp tessellation, for a given vertex of the 3D-object mesh, the disclosed systems iterate over neighboring vertices to search for a nearest intersection between one of the incident edges of the vertex and neighboring vertices and a discontinuity line from the sharpness map. Then, the vertex is moved within a threshold distance of the nearest discontinuity line intersection along the incident edge. The disclosed systems similarly reposition other vertices of the 3D-object mesh to generate a modified 3D-object mesh that includes vertices that align with discontinuities present in a displacement map.

Abstract: Embodiments provide systems, methods, and computer storage media for fitting 3D primitives to a 3D point cloud. In an example embodiment, 3D primitives are fit to a 3D point cloud using a global primitive fitting network that evaluates the entire 3D point cloud and a local primitive fitting network that evaluates local patches of the 3D point cloud. The global primitive fitting network regresses a representation of larger (global) primitives that fit the global structure. To identify smaller 3D primitives for regions with fine detail, local patches are constructed by sampling from a pool of points likely to contain fine detail, and the local primitive fitting network regresses a representation of smaller (local) primitives that fit the local structure of each of the local patches. The global and local primitives are merged into a combined, multi-scale set of fitted primitives, and representative primitive parameters are computed for each fitted primitive.

Abstract: Various disclosed embodiments are directed to image-to-material translation based on delighting an input image, thereby allowing proper capturing of the color and geometry properties of the input image for generating a visual rendering. This, among other functionality described herein, improves the inaccuracies, user experience, and computing resource consumption of existing technologies.

Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure include an image processing apparatus configured to efficiently perform texture synthesis (e.g., increase the size of, or extend, texture in an input image while preserving a natural appearance of the synthesized texture pattern in the modified output image). In some aspects, the image processing apparatus implements an attention mechanism with a multi-stage attention model where different stages (e.g., different transformer blocks) progressively refine image feature patch mapping at different scales, while utilizing repetitive patterns in texture images to enable network generalization. One or more embodiments of the disclosure include skip connections and convolutional layers (e.g., between transformer block stages) that combine high-frequency and low-frequency features from different transformer stages and unify attention to micro-structures, meso-structures and macro-structures.

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating enriched light sources by utilizing surface-centric representations of three-dimensional surfaces. Specifically, the disclosed system utilizes a surface-centric re-parameterization that combines geometric and algebraic components of a sphere to model different light source types in a continuous range of lighting configurations. The disclosed systems utilize a set of intuitive parameters to determine a shape and emission parameters for generating an enriched light source. Additionally, the disclosed system provides a set of interactive light source controls to modify a position, orientation, shape, emittance, and lighting attenuation over distance of a light source within a three-dimensional environment. The disclosed system determines the light source controls based on sets of three-dimensional interaction primitives to control one or more parameters of the light source.

Abstract: The disclosure describes one or more embodiments of systems, methods, and non-transitory computer-readable media that utilize a sharpness map that includes information on how to filter a displacement map on a per-texel basis to preserve sharp features while sampling a displacement map. For instance, the disclosed systems utilize a sharpness map that encodes combinable patterns to represent discontinuities of features within a displacement map. In some embodiments, the disclosed systems generate a sharpness map having texels encoded with discontinuity configurations that are referenced to control filtering (e.g., via interpolation) of a displacement map such that sharp features within the displacement map are not lost (due to smoothing during interpolation).

Abstract: The disclosure describes one or more embodiments of systems, methods, and non-transitory computer-readable media that modify vertex positions of a 3D-object mesh to align the vertices with discontinuities indicated by a sharpness map for a displacement map. For example, the disclosed systems access a sharpness map that encodes discontinuities of a displacement map to preserve sharp features. During sharp tessellation, for a given vertex of the 3D-object mesh, the disclosed systems iterate over neighboring vertices to search for a nearest intersection between one of the incident edges of the vertex and neighboring vertices and a discontinuity line from the sharpness map. Then, the vertex is moved within a threshold distance of the nearest discontinuity line intersection along the incident edge. The disclosed systems similarly reposition other vertices of the 3D-object mesh to generate a modified 3D-object mesh that includes vertices that align with discontinuities present in a displacement map.

Abstract: A multi-characteristic remeshing system that generates remeshed 3D graphical surfaces can include a compact geometric descriptive language (“CGDL”) conversion module, one or more geometric characteristic parsing modules, and a geometric computation module. The CGDL conversion module receives an input mesh for a 3D graphical object and CGDL source text that describes target characteristics of an output mesh of the 3D graphical object. Each geometric characteristic parsing module identifies inherent geometric characteristics of the input mesh, and generates a geometric characteristic map. The geometric characteristic map includes instructions to generate the output mesh with respective target characteristics. The instruction describes a relationship of the one or more inherent geometric characteristics with the respective target characteristic.

Abstract: Aspects and features of the present disclosure provide a direct ray tracing operator with a low memory footprint for surfaces enriched with displacement maps. A graphics editing application can be used to manipulate displayed representations of a 3D object that include surfaces with displacement textures. The application creates an independent map of a displaced surface. The application ray-traces bounding volumes on the fly and uses the intersection of a query ray with a bounding volume to produce rendering information for a displaced surface. The rendering information can be used to generate displaced surfaces for various base surfaces without significant re-computation so that updated images can be rendered quickly, in real time or near real time.