Abstract: Implementations of systems and methods for determining viewpoints suitable for performing one or more digital operations on a three-dimensional object are disclosed. Accordingly, a set of candidate viewpoints is established. The subset of candidate viewpoints provides views of an outer surface of a three-dimensional object and those views provide overlapping surface data. A subset of activated viewpoints is determined from the set of candidate viewpoints, the subset of activated viewpoints providing less of the overlapping surface data. The subset of activated viewpoints is used to perform one or more digital operation on the three-dimensional object.

Abstract: Constrained stroke editing techniques for digital content are described. In these examples, a stroke constraint system is employed as part of a digital content creation system to manage input, editing, and erasure (i.e., removal) of strokes via a user interface as part of editing digital content. To do so, locations and attributes of a displayed stroke are used to constrain location and/or attributes of an input stroke.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generate and implement semantic scene graph for digital image editing. For instance, in some embodiments, the disclosed systems receive a digital image from a client device. The disclosed systems determine, utilizing one or more neural networks, characteristics of the digital image by determining a plurality of objects portrayed in the digital image and a plurality of relationships associated with the plurality of objects. Further, the disclosed systems generate a semantic scene graph for the digital image based on its characteristics. For example, in some cases, the disclosed systems generate a structure of nodes and edges representing the characteristics of the digital image utilizing an image analysis graph and assign behaviors to the plurality of objects based on the plurality of relationships using a behavioral policy graph. The disclosed systems modify the digital image using the semantic scene graph.

Abstract: Methods and systems are provided for conformal cage-based deformation with polynomial curves. In embodiments described herein, a cage model in a first configuration is displayed along with an image. The cage model in the first configuration defines a boundary of at least a portion of points of the image. Curve control points are generated along the boundary of the cage model. The cage model is deformed from the first configuration to a second configuration by changing the location of curve control points thereby changing curvature of a corresponding portion of the boundary of the cage model. Based on the first and second configuration of the cage model, an updated image is generated by performing a conformal transformation of the points of the image within the boundary of the cage model in the first configuration. The updated image is then displayed.

Abstract: One or more implementations include systems, non-transitory computer-readable media, and methods that utilize a segmentation approach that distinguishes between smooth-shaded regions from high-frequency regions in an image within a vectorization pipeline to generate a vector image. For instance, the disclosed systems utilize a smoothing function to identify non-overlapping sets of pixels that include locally smooth pixels and pixels with high frequency details for an image. Furthermore, in some instances, the disclosed systems generate separate sets of fill functions (representing color-based regions) using color-based pixel clustering for the non-overlapping sets of pixels. Moreover, in one or more instances, the disclosed systems merge neighboring color-based regions in the sets of fill functions (using color similarity) to generate a set of segmented regions for an image.

Abstract: Systems and methods for training a Generative Adversarial Network (GAN) using feature regularization are described herein. Embodiments are configured to generate a candidate image using a generator network of a GAN, classify the candidate image as real or generated using a discriminator network of the GAN, and train the GAN to generate realistic images based on the classifying of the candidate image. The training process includes regularizing a gradient with respect to features extracted using a discriminator network of the GAN.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates a vectorized digital image of a raster image according to at least one gradient fill function. In particular, the disclosed systems generate a segmentation map of a raster digital image by using a segmentation model to cluster pixel values while minimizing a number of boundaries. Moreover, the disclosed systems generate one or more closed connected partitions that include locally smooth pixel values within the raster digital image according to the boundaries of the segmentation map. Further, the disclosed systems fit one or more fill function to the closed connected partition(s), where the one or more fill function includes at least one gradient fill function for a closed connected partition. The disclosed systems generate the vectorized digital image according to the at least one gradient fill function for the closed connected partition.

Abstract: An analytics system jointly predicts values for multiple unobserved individual-level features using aggregate data for those features. Given a dataset, a transformation is applied to individual-level information for the dataset to generate transformed data in a higher dimensional space. Bag-wise mean embeddings are generated using the transformed data. The bag-wise mean embeddings and aggregate data for unobserved individual-level features for the dataset are used to train a model to jointly predict values for the unobserved individual-features for data instances. In particular, a given data instance can be transformed to a representation in a higher dimensional space. Given this representation, the trained model predicts values for the unobserved individual-level features for the data instance, and the data instance can be augmented with the predicted values.

Abstract: In implementation of techniques for offsetting camera filter shift, a computing device implements an offset system to capture a first digital image using a filter at a first position relative to an image capture device and to capture a second digital image using the filter at a second position relative to the image capture device resulting from movement of the filter between the first position and the second position. The offset system determines a filter shift resulting from the movement by comparing the first and second digital images. The offset system then controls an offset of a portion of the image capture device based on the filter shift.

Abstract: A method, non-transitory computer readable medium, apparatus, and system for image generation include obtaining an input image having a first resolution, where the input image includes random noise, and generating a low-resolution image based on the input image, where the low-resolution image has the first resolution. The method, non-transitory computer readable medium, apparatus, and system further include generating a high-resolution image based on the low-resolution image, where the high-resolution image has a second resolution that is greater than the first resolution.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for segmenting digital videos into topic chapters. In particular, in some embodiments, the disclosed systems generate, utilizing a text encoder, a text representation for a transcript sentence of a video transcript. In addition, in some embodiments, the disclosed systems generate, utilizing a frame encoder, a set of frame representations for a set of video frames associated with the transcript sentence. Moreover, in some embodiments, the disclosed systems generate, utilizing a cross-modal attention model, a text-aware visual representation from the text representation and the set of frame representations. Furthermore, in some embodiments, the disclosed systems determine a topic-boundary label for the transcript sentence from the text representation and the text-aware visual representation.

Abstract: A computing system captures image data using a camera and captures spatial information using one or more sensors. The computing system receives voice data using a microphone. The computing system analyzes the voice data to identify a keyword. The computing system analyzes the image data and the spatial information to identify an object corresponding to the keyword. The computing system generates text based on the voice data and the keyword. The computing system stores the text in association with the object. The computing system generates and provides output comprising the text linked to the object or a derivative thereof.

Abstract: Curvilinear perspective digital image editing techniques are described. In one or more examples, a digital image and a curvilinear grid are presented for display in a user interface. The curvilinear grid depicts a plurality of axis connecting vanishing points as indicating a curvilinear perspective. An input is received defining an edit operation to the digital image. The input is conformed based on the curvilinear grid and the user interface is output having the edit operation applied to the digital image as conformed based on the curvilinear grid. The user interface is output having the edit operation applied to the digital image as conformed based on the curvilinear grid.

Abstract: A surface normal model is trained to predict normal maps from single images using pair-wise angular losses. A training dataset comprising a training image and a ground truth normal map for the training image is received. To train the surface normal model using the training dataset, a predicted normal map is generated for the training image using the surface normal model. A loss is determined as a function of angular values between pairs of normal vectors for the predicted normal map and corresponding angular values between pairs of normal vectors for the ground truth normal map. The surface normal model is updated based on the loss.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that implement depth-aware object move operations for digital image editing. For instance, in some embodiments, the disclosed systems determine a first object depth for a first object portrayed within a digital image and a second object depth for a second object portrayed within the digital image. Additionally, the disclosed systems move the first object to create an overlap area between the first object and the second object within the digital image. Based on the first object depth and the second object depth, the disclosed systems modify the digital image to occlude the first object or the second object within the overlap area.

Abstract: A material search computing system generates a joint feature comparison space by combining joint image-text features of surface material data objects. The joint feature comparison space is a consistent comparison space. The material search computing system extracts a query joint feature set from a query data object that includes text data or image data. In addition, the material search computing system compares the query joint feature set to the joint image-text features included in the joint feature comparison space. Based on the comparison, the material search computing system identifies a result joint feature set and associated result surface material data objects. The material search computing system generates material query result data describing the result surface material data objects, and provides the material query result data to an additional computing system.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for generating and visualization font variation instances of a variable font using deep-aware focal points in a font variation space. In particular, in one or more embodiments, the disclosed systems determine, for a variable font, a set of predefined font variation instances defining named typeface variations of the variable font. In some embodiments, the disclosed systems generate, from the set of predefined font variation instances, a plurality of focal points representing additional typeface variations of the variable font in a font variation space. In one or more embodiments, the disclosed systems generate, from the plurality of focal points in the font variation space, a modified set of font variation instances defining intermediate typeface variations between the named typeface variations of the variable font.

Abstract: In implementation of techniques for generating text descriptive of digital images, a computing device implements a captioning system to generate a text description of an image using a machine learning model. The captioning system identifies a negative word in the text description of the image based on a quantitative risk level for the negative word. Based on the text description of the image and the quantitative risk level, the captioning system determines an amount of suppression for use of the negative word. The captioning system then generates an updated text description of the image by applying the amount of suppression to the machine learning model to change a likelihood of using the negative word.

Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that provide a platform for on-demand selection of machine-learning models and on-demand learning of parameters for the selected machine-learning models via cloud-based systems. For instance, the disclosed system receives a request indicating a selection of a machine-learning model to perform a machine-learning task (e.g., a natural language task) utilizing a specific dataset (e.g., a user-defined dataset). The disclosed system utilizes a scheduler to monitor available computing devices on cloud-based storage systems for instantiating the selected machine-learning model. Using the indicated dataset at a determined cloud-based computing device, the disclosed system automatically trains the machine-learning model.

Abstract: Certain aspects and features of this disclosure relate to modeling shapes using SDF approximation. For example, a method involves partitioning a surface-bounding volume configured to contain the surface of a shape using a volumetric grid. The method also involves approximating, using a basis function, a value of a signed distance function (SDF) for samples taken inside, and on a boundary of, each surface-containing cell of the grid, and storing, based on the sign changes, coordinates for each of the cells. The method further involves constructing a surface-bounding volume hierarchy for the shape by assigning bounding volumes to each of the cells based on the coordinates. The method additionally involves determining an intersection of a ray with each surface-bounding volume in the surface-bounding volume hierarchy and rendering or storing a representation of the shape based on each intersection.