Abstract: Video content creation is described. A video generation system employs a machine learning system to automate video generation, from textual input to final video output. The video generation system receives a textual description and optional parameters as user input and uses this input to generate a prompt for input to at least one machine learning model. The prompt causes the at least one machine learning model to output a summary understanding of the user-provided inputs along with a scene-by-scene script for a video. Based on the script, the video generation system automatically imports or generates visuals, synthesizes narration, and selects audio for each video scene. The video generation system is then configured to adjust individual scene elements for optimal cohesion and timing during video playback. In some implementations, the video generation system produces multiple video variations, enabling users to ideate by visualizing and comparing different options.

Abstract: The present disclosure is directed toward systems, methods, and non-transitory computer readable media that finetune large language models for code generation based on code execution results using preference-guided learning. For example, the disclosed systems utilize a large language model to generate a set of code responses from a natural language query, wherein the set of code responses correspond to code. Furthermore, the disclosed systems compare the set of code responses to a ground truth code response to generate a set of correct code responses and a set of incorrect code responses. The disclosed systems generate a preference dataset from preference pairs selected from the set of correct code responses and the set of incorrect code responses. In addition, the disclosed systems finetune the large language model by modifying parameters of the large language model to reduce incorrect code responses according to a loss function based on the preference dataset.

Abstract: Techniques for learning based assessment of clip assemblies for multi-shot videos are described. In an example, a processing device is operable to receive a plurality of single-shot video clips and generate embeddings of each of the single-shot video clips using a machine learning model. The processing device is further operable to transform the embeddings into a video representation using the machine learning model and determine a learned quality score of a multi-shot video sequence assembled from the single-shot video clips by processing the video representation through a regression layer of the machine learning model. The processing device is configurable to output the learned quality score as a measure of assembly quality for the multi-shot video sequence.

Abstract: Techniques for shape aware descriptive movement synthesis are described. In an example, a processing device is operable to receive a description of a body motion and a body, used as input to a trained machine learning system that generates motion data from the description. A shape parameter and a plurality of motion tokens are jointly predicted by the machine learning system based on the description. A plurality of discrete motion features is obtained from applying a finite scalar de-quantization to the motion tokens. Using the machine learning system, the processing device generates a plurality of shape conditioned motion features by integrating a shape feature projected from the shape parameter with the discrete motion features. The shape conditioned motion features are decoded by the machine learning system into the motion data, which models the motion with specific variations caused by the shape.

Abstract: A topic mapping system generates customized mapping schemas for multiple topic sets. The topic mapping system generates document clusters that represent groups of digital documents. The topic mapping system also generates, for each topic set, a document-topic mapping data object (“DTM data object”) that describes a customized mapping schema of the document clusters to labels in the topic set. The topic mapping system identifies customized groups of documents for responding to multiple requests that have a particular keyword. For each request, the topic mapping system identifies a particular topic set and DTM data object associated with a computing system that provided the request. Based on the keyword, the topic mapping system identifies documents that are categorized according to the customized mapping schema in the DTM data object. The topic mapping system can provide customized groups of documents to respective computing systems that provided the multiple requests.

Abstract: In accordance with the described techniques, a query is received from a user for processing by an artificial intelligence (AI) chat agent. An ambiguity classifier model is leveraged to determine whether the query is ambiguous. Responsive to determining that the query is ambiguous, a query rewrite model is leveraged to update the query to resolve one or more ambiguities of the query, and the updated query is provided to the AI chat agent for processing. Responsive to determining that the query is unambiguous, the query is provided to the AI chat agent for processing.

Abstract: An improved electronic communication system schedules transmission of electronic communications based on a predicted open time and click time. The open and click times are predicted from a machine learning model that is trained to optimize for both tasks. Additionally, when training the machine learning model, the loss used for adjusting the system to achieve a desired accuracy may be a biased loss determined from a function that penalizes overpredicting the open time. As such, the loss value may be determined by different set of rules depending on whether the predicted time is greater than the actual time or not.

Abstract: A search system facilitates efficient and fast near neighbor search given item vector representations of items, regardless of item type or corpus size. To index an item, the search system expands an item vector for the item to generate an expanded item vector and selects elements of the expanded item vector. The item is index by storing an identifier of the item in posting lists of an index corresponding to the position of each selected element in the expanded item vector. When a query is received, a query vector for the item is expanded to generate an expanded query vector, and elements of the expanded query vector are selected. Candidate items are identified based on posting lists corresponding to the position of each selected element in the expand query vector. The candidate items may be ranked, and a result set is returned as a response to the query.

Abstract: learned from digital content items and their corresponding performance metrics. In accordance with some aspects, a training dataset is accessed that comprises training samples that each include a digital content item and a performance metric, and the multimodal generative model is trained using the training data. The training can include, for a training sample, using encoders of the multimodal generative model to generate a latent representation of a digital content item from the training sample and a latent representation of a performance metric from the training sample. The latent representations are merged to provide a combined latent representation, and decoders of the multimodal generative model decode the combined latent representation to provide an output digital content item and output performance metric. Losses are determined from the outputs and used to update parameters of the multimodal generative model.

Abstract: Some aspects relate to technologies for dynamically generating digital content for events using event data and content intent descriptors. In some aspects, when a content server identifies an event for digital content creation, the content server provides data to a user device that is based on event data for the event and a content intent descriptor. The user device generates a prompt using the received data and provides the prompt to a generative model on the user device, causing the generative model to generate a digital content item using the prompt as input. The user device then presents the digital content item.

Abstract: Some aspects relate to technologies for generating custom digital content using a content intent descriptor from a content server and on-device contextual data maintained on a user device. In some aspects, a user device receives a content intent descriptor communicated over a network from a content server. The user device generates a prompt using the content intent descriptor and on-device contextual data maintained on the user device. A generative model is caused to generate a digital content item using the prompt, and the digital content item is presented on the user device.

Abstract: Some aspects relate to technologies for generating and/or presenting digital content using on-device subscription data maintained on a user device. In some aspects, a user device receives a content intent descriptor communicated over a network from a content server. The user device performs a comparison of on-device subscription data with the content intent descriptor. Based on the comparison, the user device generates a prompt using the content intent descriptor. A generative model is caused to generate a digital content item using the prompt, and the digital content item is presented on the user device.

Abstract: Some aspects relate to technologies for performance-guided content generation and exploration using a multimodal generative model with a joint latent space learned from digital content items and their corresponding performance metrics. In accordance with some aspects, input is received for content generation. The input includes a digital content item and is encoded by one or more encoders of the multimodal generative model into a latent representation in the joint latent space. A latent space transformation from the latent representation of the input is performed to provide a transformed latent representation, which is decoded by one or more decoders of the multimodal generative model to generate an output digital content item. In some aspects, the one or more decoders also decode the transformed latent representation to generate a predicted performance metric for the output digital content item.

Abstract: A method, non-transitory computer readable medium, system, and apparatus for data processing includes obtaining a user query and a plurality of context examples and generating a first input and a second input. The first input comprises the user query appended to a first portion of the plurality of context examples, and the second input comprises the user query appended to a second portion of the plurality of context examples. The method, non-transitory computer readable medium, system, and apparatus for data processing further includes generating a response to the user query based on the first input and the second input.

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for detecting specific ambiguity types in queries to a large language model. The disclosed system determines a query in a prompt by a client device to a large language model. The disclosed system generates, utilizing a small language model, a label indicating that the query comprises an ambiguity of an identified ambiguity type of a plurality of ambiguity types according to a plurality of quantitative features of the query. Additionally, the disclosed system generates, for display to the client device, a response to the query based on the ambiguity of the identified ambiguity type.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for generating modifications to an LLM based artificial intelligence assistant based on classifying the severity of errors and focusing the modifications on resolving high-severity errors. In particular, the disclosed systems receive prompts via an artificial intelligence assistant graphical user interface and generate responses to the prompts using the LLM based artificial intelligence assistant. Further, the disclosed systems determine errors in the responses using an annotation tool to generate annotated errors and an error analysis mechanism to generate indications of the errors based on the annotated errors. Additionally, the disclosed systems classify the errors as one of high-severity, mid-severity, or low-severity. Moreover, the disclosed systems generate modifications to components of the LLM based artificial intelligence assistant based on the high-severity errors.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that utilize collaborative filtering and a reinforcement learning model having an actor-critic framework to provide digital content items across client devices. In particular, in one or more embodiments, the disclosed systems monitor interactions of a client device with one or more digital content items to generate item embeddings (e.g., utilizing a collaborative filtering model). The disclosed systems further utilize a reinforcement learning model to generate a recommendation (e.g., determine one or more additional digital content items to provide to the client device) based on the user interactions. In some implementations, the disclosed systems utilize the reinforcement learning model to analyze every negative and positive interaction observed when generating the recommendation.

Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that intelligently generate and modify schedules of task sequences utilizing a graph neural network and/or reinforcement learning model. For example, the disclosed system utilizes a graph neural network to generate performance efficiency scores indicating predicted performances of the sets of tasks. Additionally, the disclosed systems utilizes the performance efficiency scores to rank sets of tasks and then determine a schedule including an ordered sequence of tasks. Furthermore, disclosed system generates modified schedules in response to detecting a modification to the schedule. For example, the disclosed system utilizes a reinforcement learning model to provide recommendations of new tasks or task sequences deviating from the schedule in the event of an interruption. The disclosed system also utilizes the reinforcement learning model to learn from user choices to inform future scheduling of tasks.

Abstract: A method, apparatus, non-transitory computer readable medium, and system for data processing include receiving a natural language query including a request for data from a database, generating a natural language query embedding representing the natural language query in a vector space, and determining a validity of the natural language query by comparing the natural language query embedding to a valid query embedding in the vector space. Some embodiments include converting the natural language query into a structured query based on the validity of the natural language query and retrieving the data from the database using the structured query.

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating action plans utilizing a large language model with a best-first search model. The disclosed system determines a request to utilize a large language model to generate an action plan via one or more software tools. The disclosed system generates the action plan by traversing a decision tree comprising an action space involving the one or more software tools by iteratively: selecting, utilizing a best-first search model, an action from a set of possible actions in the action space of the decision tree; and expanding, utilizing the best-first search model, the action space of the decision tree to include an additional set of possible actions. The disclosed system also executes the action plan via one or more interactions with the one or more software tools according to the action.