Abstract: Three-dimensional object edit and visualization techniques and systems are described. In a first example, a content navigation control is implemented by a content editing system to aid navigation through a history of how a three-dimensional environment and a three-dimensional object included in the environment is created. In a second example, the content editing system is configured to streamline placement of a three-dimensional object within a three-dimensional environment. The content editing system, for instance, generates a manipulation visualization in support of corresponding editing operations to act as a guide, e.g., as an alignment guide or an option guide. In a third example, the content editing system implements a shadow control that is usable as part of an editing and as a visualization to control rendering of illumination within a three-dimensional environment.

Abstract: Three-dimensional object edit and visualization techniques and systems are described. In a first example, a content navigation control is implemented by a content editing system to aid navigation through a history of how a three-dimensional environment and a three-dimensional object included in the environment is created. In a second example, the content editing system is configured to streamline placement of a three-dimensional object within a three-dimensional environment. The content editing system, for instance, generates a manipulation visualization in support of corresponding editing operations to act as a guide, e.g., as an alignment guide or an option guide. In a third example, the content editing system implements a shadow control that is usable as part of an editing and as a visualization to control rendering of illumination within a three-dimensional environment.

Abstract: In implementation of techniques for removing image overlays, a computing device implements a reflection removal system to receive an input RAW digital image, the input RAW digital image including both a base image and an overlay image. Using a machine learning model, the reflection removal system segments the base image from the overlay image. The reflection removal system generates an output RAW digital image that includes the base image and displays the output RAW digital image in a user interface.

Abstract: An example vector path trajectory imitation system is configured to create a new vector path or to extend an existing vector path based on a reference. In this manner, a user (e.g., artist, illustrator, or designer) does not need to tweak individual anchor points to align a trajectory of the new vector path with the trajectory of the reference. Instead, the user moves a position indicator (e.g., a mouse cursor) on a digital canvas in a freehand fashion while the vector path trajectory imitation system provides visual feedback to show the user how a resultant curve will look. When the user reaches a position on the digital canvas where a new vector path is to be drawn, the user can perform an action (e.g., releasing a mouse button) and the new vector path, which follows the trajectory of the reference, is created.

Abstract: A high dynamic range editing system is configured to generate visualizations to aide digital image editing in both high dynamic ranges and standard dynamic ranges. In a first example, the visualization is generated as a histogram. In a second example, the visualization is generated to indicate high dynamic range capabilities. In a third example, the visualization is generated to indicate ranges of luminance values within a digital image. In a fourth example, the visualization is generated as a point curve that defines a mapping between detected luminance values from a digital image and output luminance values over both a standard dynamic range and a high dynamic range. In a fifth example, the visualization is generated as a preview to convert pixels from the digital image in a high dynamic range into a standard dynamic range.

Abstract: A high dynamic range editing system is configured to generate visualizations to aide digital image editing in both high dynamic ranges and standard dynamic ranges. In a first example, the visualization is generated as a histogram. In a second example, the visualization is generated to indicate high dynamic range capabilities. In a third example, the visualization is generated to indicate ranges of luminance values within a digital image. In a fourth example, the visualization is generated as a point curve that defines a mapping between detected luminance values from a digital image and output luminance values over both a standard dynamic range and a high dynamic range. In a fifth example, the visualization is generated as a preview to convert pixels from the digital image in a high dynamic range into a standard dynamic range.

Abstract: Digital representation techniques of intertwined vector objects are described. These techniques support a non-destructive representation of intertwined digital objects. Additionally, these techniques support editing of overlaps to change a visual ordering in an intuitive and efficient manner. Optimization operations are also implemented that remove redundancy, combine overlaps into a single representation, address visual artifacts at borders between the intertwined objected, and so forth.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that implements related image search and image modification processes using various search engines and a consolidated graphical user interface. For instance, in one or more embodiments, the disclosed systems receive an input digital image and search input and further modify the input digital image using the image search results retrieved in response to the search input. In some cases, the search input includes a multi-modal search input having multiple queries (e.g., an image query and a text query), and the disclosed systems retrieve the image search results utilizing a weighted combination of the queries. In some implementations, the disclosed systems generate an input embedding for the search input (e.g., the multi-modal search input) and retrieve the image search results using the input embedding.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates object masks for digital objects portrayed in digital images utilizing a detection-masking neural network pipeline. In particular, in one or more embodiments, the disclosed systems utilize detection heads of a neural network to detect digital objects portrayed within a digital image. In some cases, each detection head is associated with one or more digital object classes that are not associated with the other detection heads. Further, in some cases, the detection heads implement multi-scale synchronized batch normalization to normalize feature maps across various feature levels. The disclosed systems further utilize a masking head of the neural network to generate one or more object masks for the detected digital objects. In some cases, the disclosed systems utilize post-processing techniques to filter out low-quality masks.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for synchronizing timestamps associated with media files across a variety of capture devices. For example, the disclosed systems receive a media file captured with a user device and determine, from metadata, initial timestamp information for the media file that includes inaccurate or incomplete time zone information. The disclosed systems determine a predicted time zone for the media file by extracting GPS information from the metadata associated with the media file or identifying a peer media file and utilizing a time zone or GPS information associated with the peer media file. The disclosed systems generate a synchronized timestamp for the media file with updated timestamp information based on the predicted time zone.

Abstract: An image inpainting system is described that receives an input image that includes a masked region. From the input image, the image inpainting system generates a synthesized image that depicts an object in the masked region by selecting a first code that represents a known factor characterizing a visual appearance of the object and a second code that represents an unknown factor characterizing the visual appearance of the object apart from the known factor in latent space. The input image, the first code, and the second code are provided as input to a generative adversarial network that is trained to generate the synthesized image using contrastive losses. Different synthesized images are generated from the same input image using different combinations of first and second codes, and the synthesized images are output for display.

Abstract: A method for training a GAN to transfer lighting from a reference image to a source image includes: receiving the source image and the reference image; generating a lighting vector from the reference image; applying features of the source image and the lighting vector to a generative network of the GAN to create a generated image; applying features of the reference image and the lighting vector to a discriminative network of the GAN to update weights of the discriminative network; and applying features of the generated image and the lighting vector to the discriminative network to update weights of the generative network.

Abstract: Enhanced methods for improving the performance of classifiers are described. A ground-truth labeled dataset is accessed. A classifier predicts a predicted label for datapoints of the dataset. A confusion matrix for the dataset and classifier is generated. A credibility interval is determined for a performance metric for each label. A first labels with a sufficiently large credibility interval is identified. A second label is identified, where the classifier is likely to confuse, in its predictions, the first label with the second label. The identification of the second label is based on instances of incorrect label predictions of the classifier for the first and/or the second labels. The classifier is updated based on a new third label that includes an aggregation of the first label and the second label. The updated classifier model predicts the third label for any datapoint that the classifier previously predicted the first or second labels.

Abstract: Embodiments are disclosed for performing wire segmentation of images using machine learning. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving an input image, generating, by a first trained neural network model, a global probability map representation of the input image indicating a probability value of each pixel including a representation of wires, and identifying regions of the input image indicated as including the representation of wires. The disclosed systems and methods further comprise, for each region from the identified regions, concatenating the region and information from the global probability map to create a concatenated input, and generating, by a second trained neural network model, a local probability map representation of the region based on the concatenated input, indicating pixels of the region including representations of wires. The disclosed systems and methods further comprise aggregating local probability maps for each region.

Abstract: A job scheduling system determines a rate at which a user is providing user inputs to a user interface of a computing device. A set of jobs that is to be performed to display or otherwise present a current view of the user interface is identified in response to a user input. This set of jobs is modified by excluding from the set of jobs at least one job that is not estimated to run prior to the next user input. The user interface is displayed or otherwise presented as the modified set of jobs is performed.

Abstract: In implementations of systems and procedures for generating surrogate curvatures for assisted vector drawings, a computing device implements acquisition of a target vector curve and compares a curvature of the target vector curve to a curvature of a reference vector curve. The computing device determines whether the curvature of the target vector curve is within a threshold tolerance of the curvature of the reference vector curve. An edited curvature of the targeted vector curve is generated based on the curvature of the reference vector curve.

Abstract: In implementation of techniques for template-based behaviors in machine learning, a computing device implements a template system to receive a digital video and data executable to generate animated content. The template system determines a location within a frame of the digital video to place the animated content using a machine learning model. The template system then renders the animated content within the frame of the digital video at the location determined by the machine learning model. The template system then displays the rendered animated content within the frame of the digital video in a user interface.

Abstract: Embodiments are disclosed for editing video using image diffusion. The method may include receiving an input video depicting a target and a prompt including an edit to be made to the target. A keyframe associated with the input video is then identified. The keyframe is edited, using a generative neural network, based on the prompt to generate an edited keyframe. A subsequent frame of the input video is edited using the generative neural network, based on the prompt, features of the edited keyframe, and features of an intervening frame to generate an edited output video.

Abstract: A computing system receives a query for a three-dimensional representation of a target object. The query comprises input in the form of text describing the target object, a two-dimensional image of the target object, or a three-dimensional model of the target object. The computing system encodes the input using a machine learning model to generate an encoded representation of the input. The computing system searches a search space using nearest neighbors to identify a three-dimensional representation of the target object. The search space comprises encoded representations of multiple views of a plurality of sample three-dimensional object representations. The computing system outputs the identified three-dimensional representation of the target object.

Abstract: Methods and systems are provided for facilitating generation and utilization of a commonsense contextualizing machine learning (ML) model, in accordance with embodiments described herein. In embodiments, a commonsense contextual ML model is trained by fine-tuning a pre-trained language model using a set of training path-sentence pairs. Each training path-sentence pair includes a commonsense path, identified via a commonsense knowledge graph, and a natural language sentence identified as contextually related to the commonsense path. The trained commonsense contextualizing ML model can then be used to generate a commonsense inference path for a text input. Such a commonsense inference path can include a sequence of entities and relations that provide commonsense context to the text input. Thereafter, the commonsense inference path can be provided to a natural language processing system for use in performing a natural language processing task.