Abstract: A plurality of pixel-based sampling points are identified within an image, wherein sampling points of a pixel are distributed within the pixel. For individual sampling points of individual pixels, a corresponding radiance vector is estimated. A radiance vector includes one or more radiance values characterizing light received at a sampling point. A first machine learning module generates, for each pixel, a corresponding intermediate radiance feature vector, based on the radiance vectors associated with the sampling points within that pixel. A second machine learning module generates, for each pixel, a corresponding final radiance feature vector, based on an intermediate radiance feature vector for that pixel, and one or more other intermediate radiance feature vectors for one or more other pixels neighboring that pixel. One or more kernels are generated, based on the final radiance feature vectors, and applied to corresponding pixels of the image, to generate a lower noise image.

Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure include receiving a raster image depicting a radial color gradient; compute a radial disk model for the radial color gradient, wherein the radial disk model defines a plurality of disks with centers aligned in a same direction; construct a vector graphics representation of the radial color gradient based on the radial disk model; and generate a vector graphics image depicting the radial color gradient based on the vector graphics representation.

Abstract: Techniques for vector object blending are described to generate a transformed vector object based on a first vector object and a second vector object. A transformation module, for instance, receives a first vector object that includes a plurality of first paths and a second vector object that includes a plurality of second paths. The transformation module computes morphing costs based on a correspondence within candidate path pairs that include one of the first paths and one of the second paths. Based on the morphing costs, the transformation module generates a low-cost mapping of paths between the first paths and the second paths. To generate the transformed vector object, the transformation module adjusts one or more properties of at least one of the first paths based on the mapping, such as geometry, appearance, and z-order.

Abstract: A visual lens system is described that identifies, automatically and without user intervention, digital tool parameters for achieving a visual appearance of an image region in raster image data. To do so, the visual lens system processes raster image data using a tool region detection network trained to output a mask indicating whether the digital tool is useable to achieve a visual appearance of each pixel in the raster image data. The mask is then processed by a tool parameter estimation network trained to generate a probability distribution indicating an estimation of discrete parameter configurations applicable to the digital tool to achieve the visual appearance. The visual lens system generates an image tool description for the parameter configuration and incorporates the image tool description into an interactive image for the raster image data. The image tool description enables transfer of the digital tool parameter configuration to different image data.

Abstract: Embodiments are disclosed for reconstructing linear gradients from an input image that can be applied to another image. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a raster image, the raster image including a representation of a linear color gradient. The disclosed systems and methods further comprise determining a vector representing a direction of the linear color gradient. The disclosed systems and methods further comprise analyzing pixel points along the direction of the linear color gradient to compute color stops of the linear color gradient. The disclosed systems and methods further comprise generating an output color gradient vector with the computed color stops of the linear color gradient, the output color gradient vector to be applied to a vector graphic.

Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure receive a raster image depicting a radial color gradient; compute an origin point of the radial color gradient based on an orthogonality measure between a color gradient vector at a point in the raster image and a relative position vector between the point and the origin point; construct a vector graphics representation of the radial color gradient based on the origin point; and generate a vector graphics image depicting the radial color gradient based on the vector graphics representation.

Abstract: Embodiments are disclosed for articulated part extraction using images of animated characters from sprite sheets by a digital design system. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving an input including a plurality of images depicting an animated character in different poses. The disclosed systems and methods further comprise, for each pair of images in the plurality of images, determining, by a first machine learning model, pixel correspondences between pixels of the pair of images, and determining, by a second machine learning model, pixel clusters representing the animated character, each pixel cluster corresponding to a different structural segment of the animated character. The disclosed systems and methods further comprise selecting a subset of clusters that reconstructs the different poses of the animated character. The disclosed systems and methods further comprise creating a rigged animated character based on the selected subset of clusters.

Abstract: Embodiments are disclosed for articulated part extraction using images of animated characters from sprite sheets by a digital design system. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving an input including a plurality of images depicting an animated character in different poses. The disclosed systems and methods further comprise, for each pair of images in the plurality of images, determining, by a first machine learning model, pixel correspondences between pixels of the pair of images, and determining, by a second machine learning model, pixel clusters representing the animated character, each pixel cluster corresponding to a different structural segment of the animated character. The disclosed systems and methods further comprise selecting a subset of clusters that reconstructs the different poses of the animated character. The disclosed systems and methods further comprise creating a rigged animated character based on the selected subset of clusters.

Abstract: A visual lens system is described that identifies, automatically and without user intervention, digital tool parameters for achieving a visual appearance of an image region in raster image data. To do so, the visual lens system processes raster image data using a tool region detection network trained to output a mask indicating whether the digital tool is useable to achieve a visual appearance of each pixel in the raster image data. The mask is then processed by a tool parameter estimation network trained to generate a probability distribution indicating an estimation of discrete parameter configurations applicable to the digital tool to achieve the visual appearance. The visual lens system generates an image tool description for the parameter configuration and incorporates the image tool description into an interactive image for the raster image data. The image tool description enables transfer of the digital tool parameter configuration to different image data.

Abstract: Semantic fill techniques are described that support generating fill and editing images from semantic inputs. A user input, for example, is received by a semantic fill system that indicates a selection of a first region of a digital image and a corresponding semantic label. The user input is utilized by the semantic fill system to generate a guidance attention map of the digital image. The semantic fill system leverages the guidance attention map to generate a sparse attention map of a second region of the digital image. A semantic fill of pixels is generated for the first region based on the semantic label and the sparse attention map. The edited digital image is displayed in a user interface.

Abstract: Vector object transformation techniques are described that support generation of a transformed vector object based on a first vector object and a second vector object. A plurality of paths for a first and second vector object, for instance, are generated. Corresponding paths are determined by detecting which of the plurality of paths from the first vector object correspond to which of the plurality of paths from the second vector object. A mapping of control points between the first and second vector objects is generated. Using the mapping, a transformation of the first vector object is generated by adjusting one or more control points of the first vector object. As a result, the transformed vector object includes visual characteristics based on both the first vector object and the second vector object.

Abstract: Vector object transformation techniques are described that support generation of a transformed vector object based on a first vector object and a second vector object. A plurality of paths for a first and second vector object, for instance, are generated. Corresponding paths are determined by detecting which of the plurality of paths from the first vector object correspond to which of the plurality of paths from the second vector object. A mapping of control points between the first and second vector objects is generated. Using the mapping, a transformation of the first vector object is generated by adjusting one or more control points of the first vector object. As a result, the transformed vector object includes visual characteristics based on both the first vector object and the second vector object.

Abstract: Embodiments are disclosed for reconstructing freeform gradients from an input image. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving an input image, computing an outline of the input image, identifying a set of candidate color handles for the input image, each candidate color handle of the set of candidate color handles representing an extremum point for a color in the input image, generating a reconstructed image using a subset of the set of candidate color handles, determining a reconstruction error by computing a difference between the input image and the reconstructed image, and providing the reconstructed image when the reconstruction error is below a threshold value.

Abstract: A computer-implemented method including receiving an input image at a first image stage and receiving a request to generate a plurality of variations of the input image at a second image stage. The method including generating, using an auto-regressive generative deep learning model, the plurality of variations of the input image at the second image stage and outputting the plurality of variations of the input image at the second image stage.

Abstract: Generating a vector representation of a hand-drawn sketch is described. To do so, the sketch is segmented into different superpixel regions. Superpixels are grown by distributing superpixel seeds throughout an image of the sketch and assigning unassigned pixels to a neighboring superpixel based on pixel value differences. The border between each pair of adjacent superpixels is then classified as either an active or an inactive boundary, with active boundaries indicating that the border corresponds to a salient sketch stroke. Vector paths are generated by traversing edges between pixel vertices along the active boundaries. To minimize vector paths included in the vector representation, vector paths are greedily generated first for longer curves along active boundaries until each edge is assigned to a vector path. Regions encompassed by vector paths corresponding to a foreground superpixel are filled to produce a high-fidelity vector representation of the sketch.

Abstract: Embodiments are disclosed for reconstructing freeform gradients from an input image. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving an input image, computing an outline of the input image, identifying a set of candidate color handles for the input image, each candidate color handle of the set of candidate color handles representing an extremum point for a color in the input image, generating a reconstructed image using a subset of the set of candidate color handles, determining a reconstruction error by computing a difference between the input image and the reconstructed image, and providing the reconstructed image when the reconstruction error is below a threshold value.

Abstract: A computer-implemented method including receiving an input image at a first image stage and receiving a request to generate a plurality of variations of the input image at a second image stage. The method including generating, using an auto-regressive generative deep learning model, the plurality of variations of the input image at the second image stage and outputting the plurality of variations of the input image at the second image stage.

Abstract: Vector object stylization techniques from raster objects are described that support editing of vector objects in a manner that maintains an underlying mathematical representation of object. A raster object, for instance, is generated from an edited version of an output of a vector object. This raster object, along with the vector object are received as inputs by a vector conversion system. These inputs are utilized by the vector conversion system to generate a stylized vector object having a visual appearance that mimics and simulates a visual appearance of the raster object. As a result, the stylized vector object provides a mathematical representation of the raster object.

Abstract: Vector object stylization techniques from raster objects are described that support editing of vector objects in a manner that maintains an underlying mathematical representation of object. A raster object, for instance, is generated from an edited version of an output of a vector object. This raster object, along with the vector object are received as inputs by a vector conversion system. These inputs are utilized by the vector conversion system to generate a stylized vector object having a visual appearance that mimics and simulates a visual appearance of the raster object. As a result, the stylized vector object provides a mathematical representation of the raster object.

Abstract: Techniques are disclosed for generating a vector image from a raster image, where the raster image is, for instance, a photographed or scanned version of a hand-drawn sketch. While drawing a sketch, an artist may perform multiple strokes to draw a line, and the resultant raster image may have adjacent or partially overlapping salient and non-salient lines, where the salient lines are representative of the artist's intent, and the non-salient (or auxiliary) lines are formed due to the redundant strokes or otherwise as artefacts of the creation process. The raster image may also include other auxiliary features, such as blemishes, non-white background (e.g., reflecting the canvas on which the hand-sketch was made), and/or uneven lighting. In an example, the vector image is generated to include the salient lines, but not the non-salient lines or other auxiliary features. Thus, the generated vector image is a cleaner version of the raster image.