Abstract: Artificial intelligence (AI) agents may produce errors from responses to their requests which they are not properly configured to handle. To address at least this technical problem, in some examples, the first AI agent may transmit a request to a first resource. The first AI agent may receive a response bundle responsive to the request from the first AI agent. The response bundle may include a response from the first resource to the request and an instruction for the first AI agent. The first AI agent may perform an operation responsive to the instruction. In some examples, the instruction may instruct the first AI agent to handle the response from the first resource in a way specified by the instruction.

Abstract: Methods and systems for managing functions calls by a large language model are described. A generated message is received from a generative language model, based on an input message in an ongoing conversation, the generated message indicating a function call related to the input message. The function is executed using the function call. A function response is received from the executed function. An output message is provided to the ongoing conversation based on the function response, wherein the providing of the output message bypasses the generative language model.

Abstract: A system and method are provided for handling incomplete inputs to large language models (LLMs). The method includes, responsive to detecting an incomplete input in a messaging conversation, buffering the incomplete input prior to having an LLM respond to a prompt associated with the incomplete input.

Abstract: A system and method are provided for synchronizing chat histories used in prompting large language models (LLMs). The method includes receiving an indication of an interruption in a messaging conversation at a client application. The method also includes determining a last presented portion of a response. The response is generated by an LLM for the messaging conversation and provided to the client application in response to prompting the LLM with a prompt based on at least a first input provided to the client application. The method also includes modifying a chat history maintained by a server application based on the last presented portion of the response.

Abstract: A generative model, e.g. a large language model (LLM), may be accessed by users over a network. A user might experience latency in the response from the generative model. To address the technical problem of latency, in some embodiments, the latency of the response from a first generative model is measured. If the latency falls within a particular range, then a switch to a second generative model is performed. In some embodiments, if the first generative model is not yet finished providing the response, then the partially-completed response from the first generative model is not deleted. Instead, the second generative model provides the remaining portion of the response so that the switch appears transparent and seamless to the user, and does not require restarting the generation process, thereby avoiding or mitigating the loss of already generated output and hence saving computer resources.

Abstract: Systems and methods for detecting and selectively buffering markup instruction candidates in a streamed language model output are provided. In some embodiments, a computer-implemented method includes receiving a stream of symbols from a language model; and streaming the received stream of symbols as output. The output is caused to be rendered on a display. The streaming the symbols as output include detecting a markup sequence in the received stream of symbols. In response to detecting the markup sequence, the method pauses the streaming of the symbols as output and instead streams the received stream of symbols to a buffer. The method also includes detecting a further markup sequence in the received stream of symbols. In responsive to detecting the further markup sequence in the received stream of symbols, the method causes the symbols in the buffer to be rendered and resumes streaming the received stream of symbols as output.