Abstract: A generative artificial intelligence (AI) model application programming interface (API) receives a generative AI request and routes the generative AI request to a generative AI model. The generative AI model API identifies whether the generative AI request is an asynchronous or a synchronous request and identifies a likely length of generation requested by the generative AI request. The generative AI model API evaluates the available capacity of generative AI models in a shared pool of computing system resources based upon the generative AI model type requested by the generative AI request and based upon the length of the requested generation. The generative AI model API routes the generative AI request to a generative AI model based upon the available capacity, whether the generative AI request is synchronous or asynchronous, the generative AI model type, and the length of the requested generation.

Abstract: An evaluation request user interface display is generated for interaction by a user. The user interacts with the evaluation request user interface display to specify evaluation parameters. An evaluation pipeline selection system selects an evaluation pipeline based upon the selected evaluation parameters. The evaluation pipeline processes the evaluation parameters and submits evaluation data to an LLM to obtain evaluation results. The evaluation pipeline outputs the valuation results for interaction by the user who initiated the evaluation request.

Abstract: A generative artificial intelligence (AI) model application programming interface (API) receives a generative AI request and routes the generative AI request to a generative AI model. The generative AI model API identifies whether the generative AI request is an asynchronous or a synchronous request and identifies a likely length of generation requested by the generative AI request. The generative AI model API evaluates the available capacity of generative AI models in a shared pool of computing system resources based upon the generative AI model type requested by the generative AI request and based upon the length of the requested generation. The generative AI model API routes the generative AI request to a generative AI model based upon the available capacity, whether the generative AI request is synchronous or asynchronous, the generative AI model type, and the length of the requested generation.

Abstract: A generative artificial intelligence (AI) development system exposes an interface to a prompt generation processor and receives a prompt generation input through the exposed interface. The prompt generation processor populates an editable prompt template with a prompt and detects user interaction with the editable prompt template to generate chained prompts and provides the chained prompts to a generative AI model interface and receives a response from a generative AI model through the generative AI model interface.

Abstract: A prompt storage system receives a generative artificial intelligence (AI) prompt and a response generated from a generative AI model. A prompt processor generates a prompt record based on the generative AI prompt. The prompt record includes evaluation data indicative of a performance of the generative AI prompt. The prompt record is stored so that it can be accessed by an application through a generative AI model application programming interface.