Abstract: Systems and methods for managing rights for creative work are provided. One aspect of the systems and methods includes receiving, at a rights contract on a distributed virtual machine operated based on a public blockchain, input data including an ownership token identifier for an ownership token, where the ownership token indicates ownership of a creative work. Another aspect of the systems and methods includes obtaining, at the rights contract, an indication of usage rights for the creative work corresponding to the ownership token. Yet another aspect of the systems and methods includes minting, via the rights contract, a rights token corresponding to the ownership token, where the rights token includes a reference to the indication of the usage rights for the creative work.

Abstract: A method, apparatus, non-transitory computer readable medium, and system for generating synthetic image includes obtaining a prompt indicating an image element. In some cases, a base generation model generates a first score function based on the prompt and an auxiliary image generation model generates a second score function based on the prompt. Additionally, the first score function and the second score function are combined to obtain a combined score function. In some cases, the combined score function includes positive guidance from the first score function and negative guidance from the second score function. A synthetic image that depicts the image element is generated based on the combined score function.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for augmenting the functionality of large language models using a hybrid causal-bidirectional attention method. In particular, the disclosed systems generate, from a plurality of tokens interpretable by a large language model, a set of context tokens comprising tokens with bidirectional attention and a set of span tokens comprising tokens with causal attention and bidirectional attention. Additionally, the disclosed systems modify parameters of the large language model at a first training stage by utilizing a first loss function that incorporates the set of context tokens and a second loss function that incorporates the set of span tokens. Further, the disclosed systems modify the parameters of the large language model at a second training stage by utilizing the first loss function, the second loss function, and a third loss function that incorporates the set of context tokens.

Abstract: Embodiments are disclosed for training a system to generate audio and video representations using self-supervised learning. The method may include receiving a video signal including an audio component and a video component. A first machine learning model is trained to determine a representation of the audio component using a contrastive learning task and a temporal learning task. A second machine learning model to determine a representation of the video component using the contrastive learning task and the temporal learning task. By training the machine learning models using both contrastive learning tasks and temporal learning tasks, the machine learning models learn short term features, long term features, and semantic features of input data.

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating difference captions indicating detected differences in digital image pairs. The disclosed system generates a first feature map of a first digital image and a second feature map of a second digital image. The disclosed system converts, utilizing a linear projection neural network, the first feature map to a first modified feature map in a feature space corresponding to a large language machine-learning model. The disclosed system also converts, utilizing the linear projection neural network layer, the second feature map to a second modified feature map in the feature space corresponding to the large language machine-learning model.

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating segmentations of a raster image via a half-edge mesh structure with scanline operations. The disclosed system determines, during scanline operations on a raster image, a plurality of sets of adjacent pixels having a common color value in the raster image. The disclosed system determines, during the scanline operations on the raster image, a plurality of half-edges at edges of pixels along a boundary of a set of adjacent pixels of the plurality of sets of adjacent pixels with next half-edge directions indicating directions of subsequent half-edges along the boundary of the set of adjacent pixels. The disclosed system generates one or more oriented polyline boundary loops representing the boundary of the set of adjacent pixels from the plurality of half-edges and the next half-edge directions of the set of adjacent pixels.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that leverages a series of versions of a digital image to generate a caption prediction. Furthermore, the disclosed systems receive an image difference captioning request that includes a series of versions of a digital image with a series of manipulations applied to the series of versions. Moreover, the disclosed systems access one or more edit descriptions for one or more of the series of manipulations. Further, the disclosed systems generate text inputs from the series of versions of the digital image and the one or more edit description. From the text inputs and using a large language model, the disclosed systems generate a caption prediction that indicates a difference between a first version and a last version of the series of versions of the digital image.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that utilize deep learning to identify regions of an image that have been editorially modified. For example, the image comparison system includes a deep image comparator model that compares a pair of images and localizes regions that have been editorially manipulated relative to an original or trusted image. More specifically, the deep image comparator model generates and surfaces visual indications of the location of such editorial changes on the modified image. The deep image comparator model is robust and ignores discrepancies due to benign image transformations that commonly occur during electronic image distribution. The image comparison system optionally includes an image retrieval model utilizes a visual search embedding that is robust to minor manipulations or benign modifications of images. The image retrieval model utilizes a visual search embedding for an image to robustly identify near duplicate images.

Abstract: A method, apparatus, non-transitory computer readable medium, apparatus, and system for image processing include obtaining an input prompt describing an image element, generating, using an image generation model, an output image depicting the image element and including a watermark, and identifying the training image as a source of the output image based on the watermark. The image generation model is trained using a training image including the image element and the watermark.

Abstract: Embodiments are disclosed for performing content authentication. A method of content authentication may include dividing a query video into a plurality of chunks. A feature vector may be generated, using a fingerprinting model, for each chunk from the plurality of chunks. Similar video chunks are identified from a trusted chunk database based on the feature vectors using a multi-chunk search policy. One or more original videos corresponding to the query video are then returned.

Abstract: Embodiments of the present invention provide systems, methods, and computer storage media for guided visual search. A visual search query can be represented as a sketch sequence that includes ordering information of the constituent strokes in the sketch. The visual search query can be encoded into a structural search encoding in a common search space by a structural neural network. Indexed visual search results can be identified in the common search space and clustered in an auxiliary semantic space. Sketch suggestions can be identified from a plurality of indexed sketches in the common search space. A sketch suggestion can be identified for each semantic cluster of visual search results and presented with the cluster to guide a user towards relevant content through an iterative search process. Selecting a sketch suggestion as a target sketch can automatically transform the visual search query to the target sketch via adversarial images.

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating difference captions indicating detected differences in digital image pairs. The disclosed system generates a first feature map of a first digital image and a second feature map of a second digital image. The disclosed system converts, utilizing a linear projection neural network, the first feature map to a first modified feature map in a feature space corresponding to a large language machine-learning model. The disclosed system also converts, utilizing the linear projection neural network layer, the second feature map to a second modified feature map in the feature space corresponding to the large language machine-learning model.

Abstract: Embodiments are disclosed for performing content authentication. A method of content authentication may include dividing a query video into a plurality of chunks. A feature vector may be generated, using a fingerprinting model, for each chunk from the plurality of chunks. Similar video chunks are identified from a trusted chunk database based on the feature vectors using a multi-chunk search policy. One or more original videos corresponding to the query video are then returned.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that utilize a deep visual fingerprinting model with parameters learned from robust contrastive learning to identify matching digital images and image provenance information. For example, the disclosed systems utilize an efficient learning procedure that leverages training on bounded adversarial examples to more accurately identify digital images (including adversarial images) with a small computational overhead. To illustrate, the disclosed systems utilize a first objective function that iteratively identifies augmentations to increase contrastive loss. Moreover, the disclosed systems utilize a second objective function that iteratively learns parameters of a deep visual fingerprinting model to reduce the contrastive loss.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that utilize deep learning to map query videos to known videos so as to identify a provenance of the query video or identify editorial manipulations of the query video relative to a known video. For example, the video comparison system includes a deep video comparator model that generates and compares visual and audio descriptors utilizing codewords and an inverse index. The deep video comparator model is robust and ignores discrepancies due to benign transformations that commonly occur during electronic video distribution.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that utilize deep learning to map query videos to known videos so as to identify a provenance of the query video or identify editorial manipulations of the query video relative to a known video. For example, the video comparison system includes a deep video comparator model that generates and compares visual and audio descriptors utilizing codewords and an inverse index. The deep video comparator model is robust and ignores discrepancies due to benign transformations that commonly occur during electronic video distribution.

Abstract: Systems and methods for managing rights for creative work are provided. One aspect of the systems and methods includes receiving, at a rights contract on a distributed virtual machine operated based on a public blockchain, input data including an ownership token identifier for an ownership token, where the ownership token indicates ownership of a creative work. Another aspect of the systems and methods includes obtaining, at the rights contract, an indication of usage rights for the creative work corresponding to the ownership token. Yet another aspect of the systems and methods includes minting, via the rights contract, a rights token corresponding to the ownership token, where the rights token includes a reference to the indication of the usage rights for the creative work.

Abstract: Embodiments are disclosed for identifying matching content using neural content fingerprints. The method may include receiving a request to identify content matching a query content item, wherein the query content item is a time varying content item, generating, by an embedding network, a neural fingerprint for the query content item, identifying one or more candidate content items based on the neural fingerprint of the query content item, determining, by a ranking network, one or more similarity scores corresponding to the one or more candidate content items, and identifying one or more matching content items based on the one or more similarity scores.

Abstract: Embodiments are disclosed for training a system to generate audio and video representations using self-supervised learning. The method may include receiving a video signal including an audio component and a video component. A first machine learning model is trained to determine a representation of the audio component using a contrastive learning task and a temporal learning task. A second machine learning model to determine a representation of the video component using the contrastive learning task and the temporal learning task. By training the machine learning models using both contrastive learning tasks and temporal learning tasks, the machine learning models learn short term features, long term features, and semantic features of input data.

Abstract: Techniques and systems are provided for configuring neural networks to perform certain image manipulation operations. For instance, in response to obtaining an image for manipulation, an image manipulation system determines the fitness scores for a set of neural networks resulting from the processing of a noise map. Based on these fitness scores, the image manipulation system selects a subset of the set of neural networks for cross-breeding into a new generation of neural networks. The image manipulation system evaluates the performance of this new generation of neural networks and continues cross-breeding this neural networks until a fitness threshold is satisfied. From the final generation of neural networks, the image manipulation system selects a neural network that provides a desired output and uses the neural network to generate the manipulated image.