Abstract: In implementation of techniques for progressively generating fine polygon meshes, a computing device implements a mesh progression system to receive a coarse polygon mesh. The mesh progression system generates a fine polygon mesh that has a higher level of resolution than the coarse polygon mesh by decoding the coarse polygon mesh using a machine learning model. The mesh progression system then receives additional data describing a residual feature of a polygon mesh. Based on the additional data, the mesh progression system generates an adjusted fine polygon mesh that has a higher level of resolution than the fine polygon mesh.

Abstract: Database query translation and schema synthesis techniques are described. These techniques support training data generation usable to train a machine-learning model to translate a natural language query into a database query. A database system is configurable to address a variety of evolution types to a database schema in support of machine-learning model training through use of a training data generation module. The training data generation module is configurable to generate database query variations that address different types of database schema evolutions. An evaluation module is also configurable as part of the database system to evaluate operability of a trained machine-learning model to perform query translation using an evaluation module.

Abstract: In implementation of techniques for removing image overlays, a computing device implements a reflection removal system to receive an input RAW digital image, the input RAW digital image including both a base image and an overlay image. Using a machine learning model, the reflection removal system segments the base image from the overlay image. The reflection removal system generates an output RAW digital image that includes the base image and displays the output RAW digital image in a user interface.

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating a lens blur effect in a digital image with interactive light source adjustment. The disclosed system determines a gradient mask by detecting edges of a luminance map comprising luminance values of pixels in a digital image. The disclosed system determines a highlight mask by thresholding the luminance map to determine a subset of pixels with luminance values meeting a threshold luminance. The disclosed system also generates a gradient-highlight mask including pixel values from a combination of the gradient mask and the highlight mask. The disclosed system further generates a highlight guide image comprising indications of one or more light sources in the digital image based on the gradient-highlight mask and the highlight mask.

Abstract: A system of present disclosure, in one or more embodiments, receives selections of first and second points for a path. The first point is at a first position and the second point is at a second position in a digital design document. The system identifies a glyph of text nearest a location of the first position and determines a geometry of the glyph. The system determines a first parametric value of the geometry of the glyph nearest to the first position and determines a second parametric value of the geometry of the glyph nearest to the second position. The system generates the path between the first position and the second position that follows the geometry of the glyph at a consistent offset relative to the glyph by utilizing the first parametric value and the second parametric value to generate path geometry that follows the geometry of the glyph.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generate preliminary object masks for objects in an image, surface the preliminary object masks as object mask previews, and on-demand converts preliminary object masks into refined object masks. Indeed, in one or more implementations, an object mask preview and on-demand generation system automatically detects objects in an image. For the detected objects, the object mask preview and on-demand generation system generates preliminary object masks for the detected objects of a first lower resolution. The object mask preview and on-demand generation system surfaces a given preliminary object mask in response to detecting a first input. The object mask preview and on-demand generation system also generates a refined object mask of a second higher resolution in response to detecting a second input.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify two-dimensional images via scene-based editing using three-dimensional representations of the two-dimensional images. For instance, in one or more embodiments, the disclosed systems utilize three-dimensional representations of two-dimensional images to generate and modify shadows in the two-dimensional images according to various shadow maps. Additionally, the disclosed systems utilize three-dimensional representations of two-dimensional images to modify humans in the two-dimensional images. The disclosed systems also utilize three-dimensional representations of two-dimensional images to provide scene scale estimation via scale fields of the two-dimensional images. In some embodiments, the disclosed systems utilizes three-dimensional representations of two-dimensional images to generate and visualize 3D planar surfaces for modifying objects in two-dimensional images.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for extracting and tracing ridges at junctions, including confluences, corners, and terminals. For example, the disclosed systems generate, from a digital image, a pixel ridge using a distance transform and a depth-first traversal of pixels in the digital image. In some embodiments, the disclosed systems determine a junction within the pixel ridge by segmenting the pixel ridge into logical branches. In certain embodiments, the disclosed systems classify the junction as a confluence based on determining that the junction is made up of at least a threshold number of incident branches within the pixel ridge. In some embodiments, the disclosed systems reconstruct the confluence by detecting a region of deformity for the confluence and replacing pixels in the region of deformity with extrapolated pixels connecting incident branches.

Abstract: Three-dimensional object edit and visualization techniques and systems are described. In a first example, a content navigation control is implemented by a content editing system to aid navigation through a history of how a three-dimensional environment and a three-dimensional object included in the environment is created. In a second example, the content editing system is configured to streamline placement of a three-dimensional object within a three-dimensional environment. The content editing system, for instance, generates a manipulation visualization in support of corresponding editing operations to act as a guide, e.g., as an alignment guide or an option guide. In a third example, the content editing system implements a shadow control that is usable as part of an editing and as a visualization to control rendering of illumination within a three-dimensional environment.

Abstract: This disclosure describes one or more implementations of systems, non-transitory computer-readable media, and methods that recognize speech from a digital video utilizing an unsupervised machine learning model, such as a generative adversarial neural network (GAN) model. In one or more implementations, the disclosed systems utilize an image encoder to generate self-supervised deep visual speech representations from frames of an unlabeled (or unannotated) digital video. Subsequently, in one or more embodiments, the disclosed systems generate viseme sequences from the deep visual speech representations (e.g., via segmented visemic speech representations from clusters of the deep visual speech representations) utilizing the adversarially trained GAN model. Indeed, in some instances, the disclosed systems decode the viseme sequences belonging to the digital video to generate an electronic transcription and/or digital audio for the digital video.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for determining predicted digital fonts for textual characters within digital images utilizing one or more machine learning models or neural networks. In particular, in one or more embodiments, the disclosed systems determine textual characters within a target digital image and determine one or more predicted fonts for the textual characters utilizing a font recognition machine learning model to extract features of the textual characters from the target digital image, the font recognition machine learning model comprising parameters learned from synthetic text data comprising sample textual images generated with a multi-attribute probabilistic model across a distribution of text attributes.

Abstract: Systems and methods for image dewarping are described. The method includes obtaining an image depicting a warped object and generating a parametric curve corresponding to an edge of the warped object. Then, a mesh overlay is generated for the warped object based on the parametric curve. A dewarped image is generated based on the mesh overlay.

Abstract: A method, apparatus, and non-transitory computer readable medium for image processing include obtaining a document element and an input prompt, wherein the input prompt describes a decoration element for the document element; generating a decoration mask based on the document element, wherein the decoration mask indicates a location for the decoration element; generating a decoration image based on the input prompt and the decoration mask, wherein the decoration image includes the decoration element; and generating a decorated document by combining the document element and the decoration image.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that utilize deep learning to map query videos to known videos so as to identify a provenance of the query video or identify editorial manipulations of the query video relative to a known video. For example, the video comparison system includes a deep video comparator model that generates and compares visual and audio descriptors utilizing codewords and an inverse index. The deep video comparator model is robust and ignores discrepancies due to benign transformations that commonly occur during electronic video distribution.

Abstract: Embodiments of the present disclosure include generating a summary of a source document. Some embodiments generate a set of topics based on the summary and a predetermined number of topics. An expanded text is generated for each of the plurality of topics. An image is selected from the source document for each of the set of topics by computing a similarity score between the image and the expanded text. Then, a summary document is generated based on the plurality of topics and the expanded text.

Abstract: An example vector path trajectory imitation system is configured to create a new vector path or to extend an existing vector path based on a reference. In this manner, a user (e.g., artist, illustrator, or designer) does not need to tweak individual anchor points to align a trajectory of the new vector path with the trajectory of the reference. Instead, the user moves a position indicator (e.g., a mouse cursor) on a digital canvas in a freehand fashion while the vector path trajectory imitation system provides visual feedback to show the user how a resultant curve will look. When the user reaches a position on the digital canvas where a new vector path is to be drawn, the user can perform an action (e.g., releasing a mouse button) and the new vector path, which follows the trajectory of the reference, is created.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for predicting summary quality scores and determining summary generation costs of large language models to generate a digital document summary. In particular, in one or more embodiments, the disclosed systems extract one or more text segments from a digital document. Further, the disclosed systems generate, utilizing a quality prediction neural network, a predicted summary quality score for each of a plurality of large language models for the one or more text segments. Furthermore, the disclosed systems select a large language model from the plurality of large language models based on the predicted summary quality scores. Moreover, the disclosed systems generate, utilizing the selected large language model, a summary of the digital document.

Abstract: A method, apparatus, non-transitory computer readable medium, and system for media processing include obtaining a text prompt describing content, generating, using a multi-modal encoder, a text embedding based on the text prompt, and obtaining an image depicting the content based on the text embedding. The multi-modal encoder is trained to encode image descriptions based on a similarity between a caption of a training image and a paraphrase of the caption.

Abstract: Systems and methods for image processing are provided. One aspect of the systems and methods includes obtaining a text prompt in a first language. Another aspect of the systems and methods includes encoding the text prompt using a multilingual encoder to obtain a multilingual text embedding. Yet another aspect of the systems and methods includes processing the multilingual text embedding using a diffusion prior model to obtain an image embedding, wherein the diffusion prior model is trained to process multilingual text embeddings from the first language and a second language based on training data from the first language and the second language. Yet another aspect of the systems and methods includes generating an image using a diffusion model based on the image embedding, wherein the image includes an element corresponding to the text prompt.

Abstract: Digital video editing techniques are described that are based on a target digital image. In one or more implementations, inputs are received. The inputs include a target text prompt, a target digital image depicting a target object, and a source digital video having a plurality of frames depicting a source object. Regions-of-interest are identified in the plurality of frames of the source digital video, respectively, based on the target text prompt and the target digital image using a machine-learning model, e.g., a diffusion model. A plurality of frames of a target digital video are generated as having the target object using a generative machine-learning model. The generating is based on the regions-of-interest, the target digital image, the source digital video, and a source text prompt describing the source digital video.