Abstract: The current document is directed to reinforcement-learning-based management-system agents that control distributed applications and the infrastructure environments in which they run. Management-system agents are initially trained in simulated environments and specialized training environments before being deployed to live, target distributed computer systems where they operate in a controller mode in which they do not explore the control-state space or attempt to learn better policies and value functions, but instead produce traces that are collected and stored for subsequent use. Each deployed management-system agent is associated with a twin training agent that uses the collected traces produced by the deployed management-system agent for optimizing its policy and value functions.

Abstract: The current document is directed to reinforcement-learning-based controllers and managers that control distributed applications and the infrastructure environments in which they run. The reinforcement-learning-based controllers and managers are both referred to as “management-system agents” in this document. Management-system agents are initially trained in simulated environments and specialized training environments before being deployed to live, target distributed computer systems. The management-system agents deployed to live, target distributed computer systems operate in a controller mode, in which they do not explore the control-state space or attempt to learn better policies and value functions, but instead produce traces that are collected and stored for subsequent use.

Abstract: The current document is directed to a meta-level management system (“MMS”) that aggregates information and functionalities provided by multiple management systems and provides additional management functionalities and information. In one implementation, the MMS interfaces to external entities and users through an MMS application programming interface (“API”) implemented as a GraphQL™ interface. The MMS API, in turn, accesses microservices and stream/batch processing components through microservice and stream/batch-processing-component GraphQL interfaces. The MMS employs at least three different databases: (1) an inventory/configuration database; (2) a metrics database that stores metrics derived from time-series data obtained from the multiple management systems and from other information stored in the inventory/configuration database; and (3) an MMS database that stores business insights and other MMS-generated data. A central data bus is implemented by a KAFKA™ event-streaming system.

Abstract: The current document is directed to reinforcement-learning-based management-system agents that control distributed applications and the infrastructure environments in which they run. Management-system agents are initially trained in simulated environments and specialized training environments before being deployed to live, target distributed computer systems where they operate in a controller mode in which they do not explore the control-state space or attempt to learn better policies and value functions, but instead produce traces that are collected and stored for subsequent use. Each deployed management-system agent is associated with a twin training agent that uses the collected traces produced by the deployed management-system agent for optimizing its policy and value functions. When the optimized policy is determined to be more robust, stable, and effective than the policy of the corresponding deployed management-system agent, the optimized policy is transferred to the deployed management-system agent.

Abstract: The current document is directed to methods and systems that determine workload characteristics of computational entities from stored data and that evaluate deployment/configuration policies in order to facilitate deploying, launching, and controlling distributed applications, distributed-application components, and other computational entities within distributed computer systems. Deployment/configuration policies are powerful tools for assisting managers and administrators of distributed applications and distributed computer systems, but constructing deployment/configuration policies and, in particular, evaluating the relative effectiveness of deployment/configuration policies in increasingly complex distributed-computer-system environments may be difficult or practically infeasible for many administrators and managers and may be associated with undesirable or intolerable levels of risk.