Abstract: One or more aspects of the method, apparatus, and non-transitory computer readable medium include obtaining an original image, a scene graph describing elements of the original image, and a description of a modification to the original image. The one or more aspects further include updating the scene graph based on the description of the modification. The one or more aspects further include generating a modified image using an image generation neural network based on the updated scene graph, wherein the modified image incorporates content based on the original image and the description of the modification.

Abstract: In implementations of systems for generating templates using structure-based matching, a computing device implements a template system to receive input data describing a set of digital design elements. The template system represents the input data as a sentence in a design structure language that describes structural relationships between design elements included in the set of digital design elements. An input template embedding is generated based on the sentence in the design structure language. The template system generates a digital template that includes the set of digital design elements for display in a user interface based on the input template embedding.

Abstract: A method includes receiving a natural language description of an image to be generated using a machine learning model. The method further includes extracting, from the natural language description of the image to be generated, a control element and a sub-prompt. The method further includes identifying a relationship between the control element and the sub-prompt based on the natural language description of the image to be generated. The method further includes generating, by the machine learning model, an image based on the control element, the sub-prompt, and the relationship. The image includes visual elements corresponding to the control element and the sub-prompt.

Abstract: A method includes receiving an input including a target style and a glyph. The method further includes masking the glyph. The method further includes generating a stylized glyph by a glyph generative model using the masked glyph. The method further includes rendering the stylized glyph as a unicode stylized glyph.

Abstract: A method includes receiving a description of content to be generated using a generative model. The received description of content is associated with a user profile. The method further includes determining a semantic term based on the description of content. The method further includes generating a user-specific template including the semantic term and a user preference associated with the user profile. The method further includes generating the content using the generative model based on the user-specific template. The method further includes outputting the content for display on a target user device.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that provides to a user a subset of digital design templates as recommendations based on a creative segment classification and template classifications. For instance, in one or more embodiments, the disclosed systems generate the creative segment classification for the user and determines geo-seasonal intent data. Furthermore, the disclosed system generates template classifications using a machine learning model based on geo-seasonality and creative intent. In doing so, the disclosed system identifies a subset of digital design templates based on the template classifications, geo-seasonal intent data, and the creative segment classification of the user.

Abstract: Systems and methods seamlessly blend edited and unedited regions of an image. A computing system crops an input image around a region to be edited. The system applies an affine transformation to rotate the cropped input image. The system provides the rotated cropped input image as input to a machine learning model to generate a latent space representation of the rotated cropped input image. The system edits the latent space representation and provides the edited latent space representation to a generator neural network to generate a generated edited image. The system applies an inverse affine transformation to rotate the generated edited image and aligns an identified segment of the rotated generated edited image with an identified corresponding segment of the input image to produce an aligned rotated generated edited image. The system blends the aligned rotated generated edited image with the input image to generate an edited output image.

Abstract: Systems and methods combine an input image with an edited image generated using a generator neural network to preserve detail from the original image. A computing system provides an input image to a machine learning model to generate a latent space representation of the input image. The system provides the latent space representation to a generator neural network to generate a generated image. The system generates multiple scale representations of the input image, as well as multiple scale representations of the generated image. The system generates a first combined image based on first scale representations of the images and a first value. The system generates a second combined image based on second scale representations of the images and a second value. The system blends the first combined image with the second combined image to generate an output image.

Abstract: An illustrator system accesses a multi-element document, the multi-element document including a plurality of elements. The illustrator system determines, for each of the plurality of elements, an element-specific topic distribution comprising a ranked list of topics. The illustrator system creates a first aggregated topic distribution from the determined element-specific topic distributions. The illustrator system determines a global intent for the multi-element document, the global intent including one or more terms from the first aggregated topic distribution. The illustrator system queries a database using the global intent to retrieve a substitute element. The illustrator system generates a replacement multi-element document that includes a substitute element in place of an element in the multi-element document The at least one substitute element is different from the element in the displayed multi-element document.

Abstract: An illustrator system accesses a multi-element document including a plurality of elements. The illustrator system selects, from the plurality of elements, a selected element. The illustrator system generates a replacement multi-element document that includes a substitute element in place of the selected element in the multi-element document, wherein the substitute element is different from the selected element. The illustrator system displays, via a user interface with the multi-element document, a preview of the replacement multi-element document providing a view of the replacement multi-element document, wherein the view of the replacement multi-element document is focused to depict the substitute element.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for utilizing a design language model and a generative language model to generate digital design documents with design variations. In particular embodiments, the disclosed systems implement the design language model to tokenize the design of a document into a sequence of language tokens. For example, the disclosed systems tokenize visual elements and a layout of the document—in addition to optional user-added content. The generative language model utilizes the sequence of language tokens to predict a next language token representing a suggested design variation. Based on the predicted language token, the disclosed systems generate a modified digital design document visually portraying the suggested design variation. Further, in one or more embodiments, the disclosed systems perform iterative refinements to the modified digital design document.

Abstract: An illustrator system accesses a multi-element document, the multi-element document including a plurality of elements. The illustrator system determines, for each of the plurality of elements, an element-specific topic distribution comprising a ranked list of topics. The illustrator system creates a first aggregated topic distribution from the determined element-specific topic distributions. The illustrator system determines a global intent for the multi-element document, the global intent including one or more terms from the first aggregated topic distribution. The illustrator system queries a database using the global intent to retrieve a substitute element. The illustrator system generates a replacement multi-element document that includes a substitute element in place of an element in the multi-element document The at least one substitute element is different from the element in the displayed multi-element document.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for utilizing a design language model and a generative language model to generate digital design documents with design variations. In particular embodiments, the disclosed systems implement the design language model to tokenize the design of a document into a sequence of language tokens. For example, the disclosed systems tokenize visual elements and a layout of the document—in addition to optional user-added content. The generative language model utilizes the sequence of language tokens to predict a next language token representing a suggested design variation. Based on the predicted language token, the disclosed systems generate a modified digital design document visually portraying the suggested design variation. Further, in one or more embodiments, the disclosed systems perform iterative refinements to the modified digital design document.

Abstract: Systems and methods combine an input image with an edited image generated using a generator neural network to preserve detail from the original image. A computing system provides an input image to a machine learning model to generate a latent space representation of the input image. The system provides the latent space representation to a generator neural network to generate a generated image. The system generates multiple scale representations of the input image, as well as multiple scale representations of the generated image. The system generates a first combined image based on first scale representations of the images and a first value. The system generates a second combined image based on second scale representations of the images and a second value. The system blends the first combined image with the second combined image to generate an output image.

Abstract: Systems and methods seamlessly blend edited and unedited regions of an image. A computing system crops an input image around a region to be edited. The system applies an affine transformation to rotate the cropped input image. The system provides the rotated cropped input image as input to a machine learning model to generate a latent space representation of the rotated cropped input image. The system edits the latent space representation and provides the edited latent space representation to a generator neural network to generate a generated edited image. The system applies an inverse affine transformation to rotate the generated edited image and aligns an identified segment of the rotated generated edited image with an identified corresponding segment of the input image to produce an aligned rotated generated edited image. The system blends the aligned rotated generated edited image with the input image to generate an edited output image.