Abstract: Disclosed herein are methods, systems, and computer-readable media for integrating an application programming interface (API) with a natural language model user interface. In one embodiment, a method includes receiving a registration of an external API via a user interface connected to a natural language model, the natural language model being configured to integrate a plurality of external APIs, accessing a manifest file hosted at a first online location by a publisher of the external API, the manifest file comprising parameters for interfacing with the external API and a second online location of a specification associated with the external API, the parameters and second online location being defined by the publisher of the external API, accessing the specification at the second online location, and integrating the external API with the natural language model based on data from at least one of the manifest file or the specification.

Abstract: Disclosed embodiments may include a method of interacting with a multimodal machine learning model; the method may include providing a graphical user interface associated with a multimodal machine learning model. The method may further include displaying an image to a user in the graphical user interface. The method may also include receiving a textual prompt from the user and then generating input data using the image and the textual prompt. The method may further include generating an output at least in part by applying the input data to the multimodal machine learning model, the multimodal machine learning model configured using prompt engineering to identify a location in the image conditioned on the image and the textual prompt, wherein the output includes a first location indication. The method may also include displaying, in the graphical user interface, an emphasis indicator at the indicated first location in the image.

Abstract: The disclosed embodiments may include a method of interacting with a multimodal machine learning model; the method may include providing a graphical user interface associated with a multimodal machine learning model. The method may further include displaying an image to a user in the graphical user interface. The method may also include receiving a textual prompt from the user and then generating input data using the image and the textual prompt. The method may further include generating an output at least in part by applying the input data to the multimodal machine learning model, the multimodal machine learning model configured using prompt engineering to identify a location in the image conditioned on the image and the textual prompt, wherein the output comprises a first location indication. The method may also include displaying, in the graphical user interface, an emphasis indicator at the indicated first location in the image.

Abstract: While AI models, like large language models, are powerful tools with multiple applications, they can be complex to use and can require a lot of resources to operate. The disclosed systems and methods provide tools to generate customized models (or AI agents) that are configured with features like tailored knowledge, capabilities, and instructions that make them faster, more efficient, and use less computational resources. AI agents may offer several technical advantages of improved efficiency, resource use, and connectivity. This disclosure describes systems and methods to configure, evaluate, generate, and deploy the custom models that can more efficiently run specific tasks. Disclosed systems and methods are configured to, for example, receive a query to generate a custom model, generate the AI agent custom model with the information in the query, and then resolve user queries more efficiently using the custom model.

Abstract: While AI models, like large language models, are powerful tools with multiple applications, they can be complex to use and can require a lot of resources to operate. The disclosed systems and methods provide tools to generate customized models (or AI agents) that are configured with features like tailored knowledge, capabilities, and instructions that make them faster, more efficient, and use less computational resources. AI agents may offer several technical advantages of improved efficiency, resource use, and connectivity. This disclosure describes systems and methods to configure, evaluate, generate, and deploy the custom models that can more efficiently run specific tasks. Disclosed systems and methods are configured to, for example, receive a query to generate a custom model, generate the AI agent custom model with the information in the query, and then resolve user queries more efficiently using the custom model.

Abstract: Disclosed herein are methods, systems, and computer-readable media for regenerating a region of an image with a machine learning model based on a text input. Disclosed embodiments involve accessing a digital input image. Disclosed embodiments involve generating a masked image by removing a masked region from the input image. Disclosed embodiments involve accessing a text input corresponding to an image enhancement prompt. Disclosed embodiments include providing at least one of the input image, the masked region, or the text input to a machine learning model configured to generate an enhanced image. Disclosed embodiments involve generating, with the machine learning model, the enhanced image based on at least one of the input image, the masked region, or the text input.

Abstract: Disclosed herein are methods, systems, and computer-readable media for integrating a particular external application programming interface (API) with a natural language model user interface. In one embodiment, a method includes receiving a first input at the natural language model user interface, determining the first input includes a request to integrate the particular external application programming interface (API) with the natural language model user interface, identifying the particular external API based on the received input, integrating the particular external API with the natural language model user interface, accessing the particular external API based on the first input or a second input at the natural language model user interface, and transmitting, based on the accessing, a response message to the natural language model user interface, the response message including a result of the accessing.

Abstract: The present technology pertains to a generative response engine that can adapt a user interface provided by its front end to receive inputs in a visual format and to provide visual formats in response to prompts. In some embodiments, the generative response engine can provide a greater variety of outputs that can be rendered by the front end. Collectively, the present technology can render dynamic user interface elements in response to prompts received by the generative response engine. Generative response engines that can provide dynamic and multimodal responses that are appropriate to a task are useful for an increased range of tasks.

Abstract: Disclosed herein are methods, systems, and computer-readable media for generating computer code based on natural language input. In an embodiment, a method may comprise one or more of: receiving a docstring representing natural language text specifying a digital programming result; generating, using a trained machine learning model, and based on the docstring, a computer code sample configured to produce respective candidate results; causing the computer code sample to be executed; identifying, based on the executing, a computer code sample configured to produce a particular candidate result associated with the digital programming result; performing at least one of outputting, via a user interface, the identified computer code sample, compiling the identified computer code sample, transmitting the identified computer code sample to a recipient device, storing the identified computer code sample, and/or re-executing the identified computer code sample.

Abstract: Embodiments of the present disclosure may include systems, methods, and computer readable media for generating a vector representation, including receiving a training data set, the training data set including a plurality of paired data samples corresponding to positive example pairs, each positive example pair including a first data unit and a second data unit. Embodiments may also include converting the training data set into at least one first vector of a vector representation. Embodiments may further include accessing one or more negative example pairs to contrast against the positive example pairs. Embodiments may also include converting the one or more negative example pairs into one or more second vectors of the vector representation. Embodiments may further include training an artificial machine learning model to generate additional vectors of the vector representation.

Abstract: Disclosed herein are methods, systems, and computer-readable media for automatically classifying and moderating content. In an embodiment, a method may include receiving input data and one or more content policies, and generating a content taxonomy. The method may also include receiving multi-domain cold start data and generating training data. The method may also include accessing a language model based on the input data and the training data, and iteratively classifying the content of the input data using the language model and the content taxonomy, refining the training data based on the classified content of the input data, refining the language model based on the refined training data, probing the refined language model, and updating the threshold value based on the probing of the refined language model. The method may also include moderating the content of the input data based on the optimized language model and the content taxonomy.

Abstract: Disclosed herein are methods, systems, and computer-readable media for generating an output transcript from an input audio segment using a multi-task transformer model. In some embodiments, the transformer model can be trained to transcribe or translate audio data in multiple languages using labeled audio data. The labeled audio data can include first audio segments associated with first same-language transcripts of the first audio segments and second audio segments associated with second different-language transcripts of the second audio segments. In some embodiments, a vocabulary of the model can include special purpose and time stamp tokens. The special purpose tokens can specify tasks for the model to perform.

Abstract: Disclosed herein are methods, systems, and computer-readable media for integrating a particular external application programming interface (API) with a natural language model user interface. In one embodiment, a method includes receiving a first input at the natural language model user interface, determining the first input includes a request to integrate the particular external application programming interface (API) with the natural language model user interface, identifying the particular external API based on the received input, integrating the particular external API with the natural language model user interface, accessing the particular external API based on the first input or a second input at the natural language model user interface, and transmitting, based on the accessing, a response message to the natural language model user interface, the response message including a result of the accessing.

Abstract: Disclosed herein are methods, systems, and computer-readable media for generating an image corresponding to a text input. In an embodiment, operations may include accessing a text description and inputting the text description into a text encoder. The operations may include receiving, from the text encoder, a text embedding, and inputting at least one of the text description or the text embedding into a first sub-model configured to generate, based on at least one of the text description or the text embedding, a corresponding image embedding. The operations may include inputting at least one of the text description or the corresponding image embedding, generated by the first sub-model, into a second sub-model configured to generate, based on at least one of the text description or the corresponding image embedding, an output image. The operations may include making the output image, generated by the first second sub-model, accessible to a device.

Abstract: Disclosed herein are methods, systems, and computer-readable media for automatically generating and editing text. In an embodiment, a method may include receiving an input text prompt and receiving one or more user instructions. The method may also include accessing a language model based on the input text prompt and the one or more user instructions. The method may also include outputting, using the accessed language model, language model output text. The method may also include editing the input text prompt based on the language model and the one or more user instructions by replacing at least a portion of the input text prompt with the language model output text.

Abstract: Disclosed herein are methods, systems, and computer-readable media for generating an output transcript from an input audio segment using a multi-task transformer model. In some embodiments, the transformer model can be trained to transcribe or translate audio data in multiple languages using labeled audio data. The labeled audio data can include first audio segments associated with first same-language transcripts of the first audio segments and second audio segments associated with second different-language transcripts of the second audio segments. In some embodiments, a vocabulary of the model can include special purpose and time stamp tokens. The special purpose tokens can specify tasks for the model to perform.

Abstract: Embodiments of the present disclosure may include systems, methods, and computer readable media for generating a vector representation, including receiving a training data set, the training data set including a plurality of paired data samples corresponding to positive example pairs, each positive example pair including a first data unit and a second data unit. Embodiments may also include converting the training data set into at least one first vector of a vector representation. Embodiments may further include accessing one or more negative example pairs to contrast against the positive example pairs. Embodiments may also include converting the one or more negative example pairs into one or more second vectors of the vector representation. Embodiments may further include training an artificial machine learning model to generate additional vectors of the vector representation.

Abstract: Disclosed herein are methods, systems, and computer-readable media for generating computer code based on natural language input. In an embodiment, a method may comprise one or more of: receiving a docstring representing natural language text specifying a digital programming result; generating, using a trained machine learning model, and based on the docstring, a computer code sample configured to produce respective candidate results; causing the computer code sample to be executed; identifying, based on the executing, a computer code sample configured to produce a particular candidate result associated with the digital programming result; performing at least one of outputting, via a user interface, the identified computer code sample, compiling the identified computer code sample, transmitting the identified computer code sample to a recipient device, storing the identified computer code sample, and/or re-executing the identified computer code sample.

Abstract: Disclosed embodiments may include a method of interacting with a multimodal machine learning model; the method may include providing a graphical user interface associated with a multimodal machine learning model. The method may further include displaying an image to a user in the graphical user interface. The method may also include receiving a textual prompt from the user and then generating input data using the image and the textual prompt. The method may further include generating an output at least in part by applying the input data to the multimodal machine learning model, the multimodal machine learning model configured using prompt engineering to identify a location in the image conditioned on the image and the textual prompt, wherein the output includes a first location indication. The method may also include displaying, in the graphical user interface, an emphasis indicator at the indicated first location in the image.

Abstract: Embodiments of the present disclosure may include systems, methods, and computer readable media for generating a vector representation, including receiving a training data set, the training data set including a plurality of paired data samples corresponding to positive example pairs, each positive example pair including a first data unit and a second data unit. Embodiments may also include converting the training data set into at least one first vector of a vector representation. Embodiments may further include accessing one or more negative example pairs to contrast against the positive example pairs. Embodiments may also include converting the one or more negative example pairs into one or more second vectors of the vector representation. Embodiments may further include training an artificial machine learning model to generate additional vectors of the vector representation.