Abstract: The current document is directed to reinforcement-learning-based management-system agents that control distributed applications and the infrastructure environments in which they run. Management-system agents are initially trained in simulated environments and specialized training environments before being deployed to live, target distributed computer systems where they operate in a controller mode in which they do not explore the control-state space or attempt to learn better policies and value functions, but instead produce traces that are collected and stored for subsequent use. Each deployed management-system agent is associated with a twin training agent that uses the collected traces produced by the deployed management-system agent for optimizing its policy and value functions.

Abstract: The current document is directed to reinforcement-learning-based management-system agents that control distributed applications and the infrastructure environments in which they run. Management-system agents are initially trained in simulated environments and specialized training environments before being deployed to live, target distributed computer systems where they operate in a controller mode in which they do not explore the control-state space or attempt to learn better policies and value functions, but instead produce traces that are collected and stored for subsequent use. Each deployed management-system agent is associated with a twin training agent that uses the collected traces produced by the deployed management-system agent for optimizing its policy and value functions. When the optimized policy is determined to be more robust, stable, and effective than the policy of the corresponding deployed management-system agent, the optimized policy is transferred to the deployed management-system agent.

Abstract: The current document is directed to methods and systems that determine workload characteristics of computational entities from stored data and that evaluate deployment/configuration policies in order to facilitate deploying, launching, and controlling distributed applications, distributed-application components, and other computational entities within distributed computer systems. Deployment/configuration policies are powerful tools for assisting managers and administrators of distributed applications and distributed computer systems, but constructing deployment/configuration policies and, in particular, evaluating the relative effectiveness of deployment/configuration policies in increasingly complex distributed-computer-system environments may be difficult or practically infeasible for many administrators and managers and may be associated with undesirable or intolerable levels of risk.