Abstract: The present disclosure generally relates to a framework for recommending and evaluating courses of action (COAs) using large language models (LLMs). In accordance with some aspects of the present disclosure, a system may generate a structured data representation using a retrieval augmented generation (RAG) system connected to data sources and an LLM. The system may provide the structured data representation to a graph-based logical induction with differentiable reasoning (GLIDR) system. The GLIDR system may select one or more solving units and provide data from the structured data representation to the selected solving units. The selected solving units may generate action data that causes an actuating device to perform one or more operations. The system may provide this action data to the actuating device. In some implementations, the structured data representation includes a knowledge graph or a graph schema.

Abstract: The present disclosure generally relates to evaluating and enhancing LLM responses. In some implementations, a system includes multiple language models with different specialized roles that work together to improve response reliability and transparency. A responder model can generate initial responses to user queries, providing diverse perspectives on the same input. An evaluator model can assess and combines responses from the responder models into an accurate and reliable output. A reporter model can generate summaries and alerts about response quality and confidence levels, providing transparency to users about the decision-making process. An artificial intelligence (AI) engine can manage the flow of information between the different models, orchestrating their interactions and ensuring proper sequencing of operations. A retrieval system can provide additional context from external knowledge sources, allowing the system to generate accurate and well-informed responses.