Abstract: In accordance with the described techniques, a code conversion system receives a digital image of a webpage. Using an object detection model, the code conversion system detects a webpage block in the digital image, as well as a block class assigned to the webpage block. In addition, the code conversion system extracts webpage content of the webpage block from source code of the webpage. Using a generative artificial intelligence (AI) model, the code conversion system generates custom code formatted in accordance with a webpage publication system based on the webpage block, the block class, and the webpage content.

Abstract: In implementations of systems for generating captions, a processing device implements a caption generation service to receive an input for caption generation that includes a text input indicating example language or content for the caption and an action input indicating a desired action. The processing device receives the text input via a user interface. The caption generation service generates a textual prompt for a machine-learning model based on the action input and text input. The machine-learning model uses the textual prompt to generate the caption in a specified structural format. The processing device then causes the generated caption to be presented to a user via the user interface.

Abstract: In implementations of systems for digital content analysis, a computing device implements an analysis system to extract a first content component and a second content component from digital content to be analyzed based on content metrics. The analysis system generates first embeddings using a first machine learning model and second embedding using a second machine learning model. The first embeddings and the second embeddings are combined as concatenated embeddings. The analysis system generates an indication of a content metric for display in a user interface using a third machine learning model based on the concatenated embeddings.

Abstract: Content relevance based table query answering is described. In one or more examples, a query and a table are received. The table includes a plurality of cells. A plurality of scores for calculated that correspond to the plurality of cells based on the query. One or more machine-learning models are then leveraged to generate a search result from the query, table, and scores, which is presented in a user interface for display.

Abstract: Content relevance based table query answering is described. In one or more examples, a query and a table are received. The table includes a plurality of cells. A plurality of scores for calculated that correspond to the plurality of cells based on the query. One or more machine-learning models are then leveraged to generate a search result from the query, table, and scores, which is presented in a user interface for display.

Abstract: Content relevance based table query answering is described. In one or more examples, a query and a table are received. The table includes a plurality of cells. A plurality of scores for calculated that correspond to the plurality of cells based on the query. One or more machine-learning models are then leveraged to generate a search result from the query, table, and scores, which is presented in a user interface for display.

Abstract: In implementations of systems for digital content analysis, a computing device implements an analysis system to extract a first content component and a second content component from digital content to be analyzed based on content metrics. The analysis system generates first embeddings using a first machine learning model and second embedding using a second machine learning model. The first embeddings and the second embeddings are combined as concatenated embeddings. The analysis system generates an indication of a content metric for display in a user interface using a third machine learning model based on the concatenated embeddings.

Abstract: Methods, computer systems, computer-storage media, and graphical user interfaces are provided for determining user affinities by tracking historical user interactions with tagged digital content and using the user affinities in content generation applications. Accordingly, the system may track user interactions with published digital content in order to generate user interaction reports whenever a user engages with the digital content. The system may aggregate the interaction reports to generate an affinity profile for a user or audience of users. A marketer may then generate digital content for a target user or audience of users and the system may process the digital content to generate a set of tags for the digital content. Based on the set of tags, the system may then evaluate the digital content in view of the affinity profile for the target user/audience to determine similarities or differences between the digital content and the affinity profile.

Abstract: Embodiments are generally directed to extending artificial intelligence (AI) and machine learning (ML) techniques to determine the memorability of visual content, including the long-term memorability of the visual content. One method of determining the memorability of visual content includes generating language tokens from the visual content. The language tokens represent the visual content in a language space and include visual encoding tokens computed by a visual encoding model and verbalization tokens computed by a verbalization model. A natural language processing (NLP) model, such as a pre-trained large language model (LLM), is trained using at least one memorability dataset to process the language tokens and determine a memorability prediction for the visual content. The memorability prediction includes a probability of the digital visual content being remembered by a viewer, for instance, over a long-term duration.

Abstract: Methods, computer systems, computer-storage media, and graphical user interfaces are provided for efficiently generating video insights based on text representations of videos. In embodiments, text data associated with a video is obtained. Thereafter, a model prompt to be input into a large language model is generated. The model prompt includes the text data associated with the video. As output from the large language model, a text representation that represents the video in natural language based on the text data is obtained. The text representation is provided as input into a machine learning model to generate a video insight that indicates context of the video.

Abstract: In implementations of systems for generating digital content, a computing device implements a generation system to receive a user input specifying a characteristic for digital content. The generation system generates input text based on the characteristic for processing by a first machine learning model. Output text generated by the first machine learning model based on processing the input text is received. The output text describes a digital content component. The generation system generates the digital content component by processing the output text using a second machine learning model. The generation system generates the digital content including the digital content component for display in a user interface based on the characteristic.

Abstract: In implementations of systems for digital content analysis, a computing device implements an analysis system to extract a first content component and a second content component from digital content to be analyzed based on content metrics. The analysis system generates first embeddings using a first machine learning model and second embedding using a second machine learning model. The first embeddings and the second embeddings are combined as concatenated embeddings. The analysis system generates an indication of a content metric for display in a user interface using a third machine learning model based on the concatenated embeddings.

Abstract: Content creation techniques are described that leverage content analytics to provide insight and guidance as part of content creation. To do so, content features are extracted by a content analytics system from a plurality of content and used by the content analytics system as a basis to generate a content dataset. Event data is also collected by the content analytics system from an event data source. Event data describes user interaction with respective items of content, including subsequent activities in both online and physical environments. The event data is then used to generate an event dataset. An analytics user interface is then generated by the content analytics system using the content dataset and the event dataset and is usable to guide subsequent content creation and editing.

Abstract: Content creation techniques are described that leverage content analytics to provide insight and guidance as part of content creation. To do so, content features are extracted by a content analytics system from a plurality of content and used by the content analytics system as a basis to generate a content dataset. Event data is also collected by the content analytics system from an event data source. Event data describes user interaction with respective items of content, including subsequent activities in both online and physical environments. The event data is then used to generate an event dataset. An analytics user interface is then generated by the content analytics system using the content dataset and the event dataset and is usable to guide subsequent content creation and editing.

Abstract: Methods, computer systems, computer-storage media, and graphical user interfaces are provided for determining user affinities by tracking historical user interactions with tagged digital content and using the user affinities in content generation applications. Accordingly, the system may track user interactions with published digital content in order to generate user interaction reports whenever a user engages with the digital content. The system may aggregate the interaction reports to generate an affinity profile for a user or audience of users. A marketer may then generate digital content for a target user or audience of users and the system may process the digital content to generate a set of tags for the digital content. Based on the set of tags, the system may then evaluate the digital content in view of the affinity profile for the target user/audience to determine similarities or differences between the digital content and the affinity profile.

Abstract: Some embodiments described herein relate to a training module comprising a scanpath generation model training system. The training module may be used to generate a scanpath generation model. The training module may comprise an adversarial training neural network. Using training data, which includes a text input and a recorded scanpath corresponding to the text input, the adversarial training neural network is trained to generate a scanpath generation model. A scanpath may comprise a sequence of words and a corresponding sequence of fixation durations, wherein the sequence of words comprises one or more words comprising the text input. The training module may then output the trained scanpath generation model.

Abstract: Methods, computer systems, computer-storage media, and graphical user interfaces are provided for determining user affinities by tracking historical user interactions with tagged digital content and using the user affinities in content generation applications. Accordingly, the system may track user interactions with published digital content in order to generate user interaction reports whenever a user engages with the digital content. The system may aggregate the interaction reports to generate an affinity profile for a user or audience of users. A marketer may then generate digital content for a target user or audience of users and the system may process the digital content to generate a set of tags for the digital content. Based on the set of tags, the system may then evaluate the digital content in view of the affinity profile for the target user/audience to determine similarities or differences between the digital content and the affinity profile.

Abstract: A content analysis system provides content understanding for a content item using an attention aware multi-modal model. Given a content item, feature extractors extract features from content components of the content item in which the content components comprise multiple modalities. A cross-modal attention encoder of the attention aware multi-modal model generates an embedding of the content item using features extracted from the content components. A decoder of the attention aware multi-modal model generates an action-reason statement using the embedding of the content item from the cross-modal attention encoder.

Abstract: Content creation techniques are described that leverage content analytics to provide insight and guidance as part of content creation. To do so, content features are extracted by a content analytics system from a plurality of content and used by the content analytics system as a basis to generate a content dataset. Event data is also collected by the content analytics system from an event data source. Event data describes user interaction with respective items of content, including subsequent activities in both online and physical environments. The event data is then used to generate an event dataset. An analytics user interface is then generated by the content analytics system using the content dataset and the event dataset and is usable to guide subsequent content creation and editing.

Abstract: This disclosure describes one or more implementations of systems, non-transitory computer-readable media, and methods that recognize speech from a digital video utilizing an unsupervised machine learning model, such as a generative adversarial neural network (GAN) model. In one or more implementations, the disclosed systems utilize an image encoder to generate self-supervised deep visual speech representations from frames of an unlabeled (or unannotated) digital video. Subsequently, in one or more embodiments, the disclosed systems generate viseme sequences from the deep visual speech representations (e.g., via segmented visemic speech representations from clusters of the deep visual speech representations) utilizing the adversarially trained GAN model. Indeed, in some instances, the disclosed systems decode the viseme sequences belonging to the digital video to generate an electronic transcription and/or digital audio for the digital video.