Abstract: A computer-implemented method for cost and carbon aware large language model (LLM) cache management is disclosed. The method includes: determining whether a prompt matching an input prompt exists in a plurality of prompts stored in a cache memory; upon determining that the prompt matching the input prompt exists in the cache memory, returning a response associated with the prompt from the cache memory; otherwise, (a) invoking the LLM and returning a response generated by the LLM, (b) updating the cache memory with the input prompt and the associated response obtained from the LLM as an entry in the cache memory, (c) estimating an operational cost and a carbon cost associated with generating the response to the input prompt, (d) fetching data associated with the LLM, and (e) updating the operational cost, the carbon cost and the data associated with the LLM as metadata of the entry in the cache memory.

Abstract: System and method for assessing and enhancing quality of prompts in generative AI-driven software coding is disclosed. The method includes, receiving a user input, wherein the user input comprises at least one keyword associated with a task, predicting an intent of the user input based on the at least one keyword, the task, and a context of the at least one keyword, and predicting a plurality of target elements applicable to the user input based on the predicted intent, wherein the plurality of target elements comprise entities and actions in the user input.

Abstract: A computer-implemented method for cost and carbon aware large language model (LLM) cache management is disclosed. The method includes: determining whether a prompt matching an input prompt exists in a plurality of prompts stored in a cache memory; upon determining that the prompt matching the input prompt exists in the cache memory, returning a response associated with the prompt from the cache memory; otherwise, (a) invoking the LLM and returning a response generated by the LLM, (b) updating the cache memory with the input prompt and the associated response obtained from the LLM as an entry in the cache memory, (c) estimating an operational cost and a carbon cost associated with generating the response to the input prompt, (d) fetching data associated with the LLM, and (e) updating the operational cost, the carbon cost and the data associated with the LLM as metadata of the entry in the cache memory.

Abstract: A device may receive system data identifying computational components and software components of a distributed and heterogeneous system executing a hybrid cloud application, and may create digital twins for the computational components and the software components. The device may create a central digital twin to receive functional data, operational data, and key performance indicators (KPIs) from the digital twins, and may create complex KPIs based on the functional data, the operational data, and the KPIs. The device may modify, based on the complex KPIs, one or more of the digital twins to generate additional KPIs, and process the additional KPIs, with a principal component analysis model and a self-organizing maps model, to detect anomalies in the distributed and heterogeneous system. The device may generate, based on the anomalies, a KPI cause vector identifying a root cause associated with the anomalies, and may perform actions based on the root cause.

Abstract: System and method for query augmentation for generating responses is disclosed. The method includes, receiving an input data from a user device, determining a context of the received input data, determining a domain specific graphical knowledge schema corresponding to the received input data, and identifying a plurality of missing entities by analyzing the determined context and at least one graphical instance corresponding to the determined appropriate domain specific graphical knowledge schema. The method further includes, prioritizing the identified plurality of missing entities, generating at least one sub-query for each of the identified plurality of missing entities and retrieving a relevant content corresponding to the generated at least one sub-query using a RAG-based system.

Abstract: Method, system, and computer-readable storage media for generating a foundation model pipeline including a set of foundation models for completion of a plurality of tasks. Each task of the plurality of tasks has a set of pre-conditions and a set of post-conditions. Based on the set of pre-conditions and the set of post-conditions, a set of possible plans for processing the plurality of tasks is generated. For each plan of the set of possible plans, the set of foundation models from a plurality of foundation models is identified for performing each task of the plurality of tasks according to the respective plan. Further, an efficiency score is estimated for each plan to perform the plurality of tasks according to the plan. Based on the estimated efficiency score of each plan, the set of foundation models is selected for the plurality of tasks.

Abstract: System and method for managing virtual machines (VMs) is disclosed. The method includes, identifying, using a map of VMs, a set of candidate VMs that are predicted to consume less power over time than the first VM for a common workload, removing from consideration any of the candidate VM from the set of candidate VMs that fails to satisfy any predetermined criteria, performing tradeoff analysis on the remaining candidate VMs based on one or more parameters to identify the candidate VMs for redeploying the software operations and ranking the identified candidate VMs based on at least predicted power consumption data for the common workload. The method further includes, selecting, based on the ranking, one of the identified candidate VMs as the second VM, redeploying the software operations from the first VM to the second VM, rerouting data inflow to the first VM to the second VM, and terminating operations of the first VM.

Abstract: Methods, systems, and computer-readable storage media for selecting foundation models. For selecting the foundation models, tasks and contextual parameters are obtained. The contextual parameters include functional requirement values and user preferences. The functional requirement values describe operating characteristics of a foundation model of a plurality of foundation models. Based on the functional requirement values and the user preference values, utility values of the foundation model are estimated. Based on the estimated utility values, a set of foundation models from the plurality of foundation models is selected. The functional requirement values and the user preference values constrain the selection of the foundation models. The selected set of foundation models are outputted for performing the tasks.

Abstract: Methods, systems, and computer-readable media for ranking large language models (LLMs). Input including list of LLMs, list of hardware and artificial intelligence (AI) prompt are provided by the user for ranking the LLMs. Based on the input, first estimating minimum number of hardware units needed to process AI prompt on each LLM/hardware combination and second estimating time to process the AI prompt using each LLM/hardware combination. Based on minimum number of hardware units and time to process AI prompt, third estimating amount of energy consumed by each LLM/hardware combination. Based on energy consumed, ranking LLM/hardware combinations for AI prompt. Based on ranking, selecting LLM and hardware, submitting AI prompt to LLM on hardware, and receiving response to submitted AI prompt from LLM.

Abstract: Methods, systems, and computer-readable storage media for information retrieval using query aware extended chunks. For retrieving relevant information from a foundation model, a plurality of chunks substantially relevant to a query are identified, the plurality of chunks containing information relevant to the query. Based on the plurality of chunks, one or more semantically similar chunks are identified for one or more of the plurality of chunks substantially relevant to the query. By combining the one or more semantically similar chunks with the respective one or more of the plurality of chunks, one or more extended chunks are generated. Based on one or more objectives, a subset of the one or more extended chunks is selected. The query and the selected subset are inputted as a context to the foundation model for retrieving relevant information from the foundation model.

Abstract: Methods, systems, and computer-readable storage media for information retrieval using query aware extended chunks. For retrieving relevant information from a foundation model, a plurality of chunks substantially relevant to a query are identified, the plurality of chunks containing information relevant to the query. Based on the plurality of chunks, one or more semantically similar chunks are identified for one or more of the plurality of chunks substantially relevant to the query. By combining the one or more semantically similar chunks with the respective one or more of the plurality of chunks, one or more extended chunks are generated. Based on one or more objectives, a subset of the one or more extended chunks is selected. The query and the selected subset are inputted as a context to the foundation model for retrieving relevant information from the foundation model.

Abstract: Methods, systems, and computer-readable storage media for generating configuration templates. For generating the configuration templates, conversational queries are generated. Based on conversational responses to the conversational queries, a task context and a task intent are determined using a first foundation model to identify software packages to be configured to perform tasks. Based on the task context, the task intent, and the conversational responses, a workflow template is generated using a second foundation model. Further, based on conversational responses, configuration fields of the workflow template for subtasks of each task are refined using a third foundation model. Based on the configuration fields of the workflow template, configuration fields of the configuration template for each task are generated using a fourth foundation model.

Abstract: Aspects of the present disclosure provide systems, methods, and computer-readable storage media that support intelligent re-use of knowledge (e.g., across an organization) using a natural text-based querying framework. A knowledge representation of prior work performed for the organization may be generated based on organizational knowledge (e.g., historical work record data that identifies a plurality of work items across an organization). The knowledge representation may include individual work-record entities for each respective work item and individual knowledge graphs corresponding to the individual work-record entities. For each individual knowledge graph, operations may be performed to identity and store project name, subgraph, sentence embedding, and word embedding information.

Abstract: A retraining monitoring system maintains the sustainability of a production machine learning (ML) model system that includes a production ML model retraining platform. The retraining monitoring system collects contextual data from the production ML model system and determines if one or more of a currently-selected architectural options has to be changed for sustainability. An architectural option of the production ML model retraining platform, such as, a processing location is selected from a cloud retraining platform or an on-premises retraining platform by a selection process based on a multi-armed bandit problem. An evaluation of the retraining architecture is dealt with as a reinforcement learning problem to implement one of a periodic retraining architecture or a reactive retraining architecture.

Abstract: A method and system for assisting program code development are disclosed. The method may include obtaining multimodal data of a meeting discussing a program code development, extracting a plurality of topics and a plurality of concepts from the multimodal data, identifying a plurality of meeting segments for the plurality of concepts. The method may further include determining a coding intent from program codes for the program code development, aligning the coding intent to a set of topics, identifying the set of concepts associated with the topic aligned with the coding intent, identifying a set of meeting segments associated with the concept. The method may further include determining an alignment metric of the meeting segment based on an alignment metric between the coding intent and the topic, and outputting one or more meeting segments for the coding intent based on alignment metrics of the one or more meeting segments.

Abstract: Disclosed herein is a system and method for generating computer code for a plurality of components of a software development project. An artificial intelligence code generator can generate computer code in response to a natural language text input describing a component of the software development project. A first database can store natural language text describing components of the software development project. A second database can store computer code generated at least partially by the artificial intelligence code generator and defining components of the software development project and corresponding to the natural language text stored in the first database. Using a pre-trained language model, the system can generate a natural language summary text based on the code intent of the first component, the identified characteristics of the objects, and the natural language text retrieved from the first database.

Abstract: In some implementations, a device may receive first energy consumption information relating to a set of hardware components of a computing system. The device may receive second energy consumption information relating to a set of virtual machines associated with the computing system. The device may receive third energy consumption information relating to a set of software elements associated with the computing system. The device may determine an energy consumption of the computing system based on the first energy consumption information, the second energy consumption information, and the third energy consumption information. The device may identify, based on the energy consumption of the computing system, an energy optimization associated with a usage context of the computing system. The device may transmit a set of instructions to alter one or more parameters of the computing system to implement the energy optimization for the computing system.

Abstract: Aspects of the present disclosure provide systems, methods, and computer-readable storage media that support intelligent re-use of knowledge (e.g., across an organization) using a natural text-based querying framework. A knowledge representation of prior work performed for the organization may be generated based on organizational knowledge (e.g., historical work record data that identifies a plurality of work items across an organization). The knowledge representation may include individual work-record entities for each respective work item and individual knowledge graphs corresponding to the individual work-record entities. For each individual knowledge graph, operations may be performed to identity and store project name, subgraph, sentence embedding, and word embedding information.

Abstract: In some implementations, a device may receive information identifying a computing system for energy management, the computing system having a set of hardware components, a set of virtual machines, and a set of software entities. The device may generate a digital twin of the computing system for simulation of the set of hardware components, the set of virtual machines, and the set of software entities. The device may determine, using the digital twin of the computing system, a set of energy consumption metrics, for the computing system, associated with a set of candidate parameters. The device may generate, using a recommendation engine, one or more recommendations for the computing system based on the set of energy consumption metrics associated with the set of candidate parameters. The device may transmit information associated with identifying the one or more recommendations.

Abstract: Disclosed herein is a system and method for generating computer code for a plurality of components of a software development project. An artificial intelligence code generator can generate computer code in response to a natural language text input describing a component of the software development project. A first database can store natural language text describing components of the software development project. A second database can store computer code generated at least partially by the artificial intelligence code generator and defining components of the software development project and corresponding to the natural language text stored in the first database. Using a pre-trained language model, the system can generate a natural language summary text based on the code intent of the first component, the identified characteristics of the objects, and the natural language text retrieved from the first database.