Abstract: Network switches are devices that connect multiple devices together on a computer network, using packet switching to receive, process, and forward data to the destination device. Each switch typically contains multiple ports, which are the points of connection for network cables. These ports can be in an active state, where they are ready to transmit data, or in an idle state, where they consume less power. Power consumption in datacenters has been a topic of concern due to the increasing demand for data processing and storage. One approach to reducing power consumption involves managing the power state of the switch ports. However, current power saving policies focus on making decisions for one type of traffic pattern or for a single port at a time, and therefore cannot intelligently or dynamically adapt to a multitude of network parameters affecting traffic flows. The present disclosure uses artificial intelligence to more intelligently transition ports between different modes of operation.

Abstract: Systems, computer program products, and methods are described for advanced congestion control using multiple congestion indicators in a networking environment. An example system may include an intelligent agent configured to learn congestion control policies. The agent may interact with real-world or simulated environments replicating real-world benchmarks. Congestion indicators such as telemetry information, packet drop metrics, congestion notification packet rate, pause frame rate, port utilization metrics, and/or the like form a comprehensive state representation of the network, enabling congestion state of the network environment. The intelligent agent evaluates these conditions using a reward function to optimize network performance. The intelligent agent may then implement a behavioral policy in response to the captured congestion indicators, thereby changing the congestion state of the network environment.

Abstract: A method for performing a Tree-Search (TS) on an environment is provided. The method comprises generating a tree for a current state of the environment based on a TS policy, determining a corrected TS policy, and determining an action to apply to the environment based on the corrected TS policy. The tree comprises a plurality of nodes including a root node among the plurality of nodes corresponding to the current state of the environment. Each node other than the root node among the plurality of nodes corresponding to an estimated future state of the environment. The plurality of nodes in the tree are connected by a plurality of edges. Each edge among the plurality of edges is associated with an action causing a transition from a first state to a different sate of the environment.

Abstract: Reinforcement learning, which is a machine learning technique where a model learns to make decisions that maximize a reward, has shown great promise in various domains that involve sequential decision making, including for many real-world tasks, such as inventory management, traffic signal optimization, network optimization, resource allocation, and robotics. However, current neural network (NN) based solutions for reinforcement learning struggle with interpretability, handling categorical data, and supporting light implementations suitable for low-compute devices. The present disclosure provides a gradient boosting trees (GBT) framework that is tailored for reinforcement learning, which may enable interpretability, may be well suited for real-world tasks with structured data, and may be capable of deployment on low-compute devices.

Abstract: Network switches are devices that connect multiple devices together on a computer network, using packet switching to receive, process, and forward data to the destination device. Each switch typically contains multiple ports, which are the points of connection for network cables. These ports can be in an active state, where they are ready to transmit data, or in an idle state, where they consume less power. Power consumption in datacenters has been a topic of concern due to the increasing demand for data processing and storage. One approach to reducing power consumption involves managing the power state of the switch ports. However, current power saving policies focus on making decisions for one type of traffic pattern or for a single port at a time, and therefore cannot intelligently or dynamically adapt to a multitude of network parameters affecting traffic flows. The present disclosure uses artificial intelligence to more intelligently transition ports between different modes of operation.

Abstract: Methods, systems, devices, and computer program products for machine learning in datacenter applications are provided. An example method includes receiving, by a centralized computing device, data packets from a networked device communicably coupled with the centralized computing device. The networked device is associated with performance of at least a first machine learning based task, and each of the data packets include data entries generated by the networked device based on data traffic associated with the at least one networked device and/or one or more modifications thereto. The method further includes generating updated operational parameters associated with the first machine learning based task based on the data entries forming the plurality of data packets where the updated operational parameters are generated locally by the centralized computing device. The method also includes transmitting, by the centralized computing device, the updated operational parameters to the networked device.

Abstract: Systems and devices for network data collection and processing are provided. An example system includes a first networked device and a centralized computing device communicably coupled with the at least one networked device. The first networked device operates to generate event-driven data entries associated with the first networked device and generate first data packets including the event-driven data entries and/or manipulated outputs generated based on manipulations to the event-driven data entries. The centralized computing device receives the first data packets from the first networked device and determines configuration updates based on the first data packets. The configuration updates are generated locally by the centralized computing device, and the centralized computing device transmits the one or more configuration updates to the first networked device.

Abstract: One embodiment of a method for controlling a character includes receiving a state of the character, a path to follow, and first information about a scene, generating, via a trained machine learning model and based on the state of the character, the path, and the first information, a first action for the character to perform, wherein the first action comprises a first type of motion included in a plurality of types of motions for which the trained machine learning model is trained to generate actions, and causing the character to perform the first action.

Abstract: One embodiment of a method for controlling a character includes receiving a state of the character, a path to follow, and first information about a scene, generating, via a trained machine learning model and based on the state of the character, the path, and the first information, a first action for the character to perform, wherein the first action comprises a first type of motion included in a plurality of types of motions for which the trained machine learning model is trained to generate actions, and causing the character to perform the first action.

Abstract: A system, method, and apparatus of power management for computing systems are included herein that optimize individual frequencies of components of the computing systems using machine learning. The computing systems can be tightly integrated systems that consider an overall operating budget that is shared between the components of the computing system while adjusting the frequencies of the individual components. An example of an automated method of power management includes: (1) learning, using a power management (PM) agent, frequency settings for different components of a computing system during execution of a repetitive application, and (2) adjusting the frequency settings of the different components using the PM agent, wherein the adjusting is based on the repetitive application and one or more limitations corresponding to a shared operating budget for the computing system.

Abstract: In various embodiments, a congestion control modelling application automatically controls congestion in data transmission networks. The congestion control modelling application executes a trained neural network in conjunction with a simulated data transmission network to generate a training dataset. The trained neural network has been trained to control congestion in the simulated data transmission network. The congestion control modelling application generates a first trained decision tree model based on an initial loss for an initial model relative to the training dataset. The congestion control modelling application generates a final tree-based model based on the first trained decision tree model and at least a second trained decision tree model. The congestion control modelling application executes the final tree-based model in conjunction with a data transmission network to control congestion within the data transmission network.

Abstract: A method is performed by an agent operating in an environment. The method comprises computing a first value associated with each state of a number of states in the environment, determining a lookahead horizon for each state of the number of states in the environment based on the computed first value for each state of the number of states, applying a first policy to compute a second value associated with each state of at least one state in the number of states in the environment for the at least one state in the number of states based on the determined lookahead horizons for the number of states, and determining a second policy based on the first policy and the second value for each state of the number of states in the environment.

Abstract: Introduced herein is a technique that uses ML to autonomously find a cache management policy that achieves an optimal execution of a given workload of an application. Leveraging ML such as reinforcement learning, the technique trains an agent in an ML environment over multiple episodes of a stabilization process. For each time step in these training episodes, the agent executes the application while making an incremental change to the current policy, i.e., cache-residency statuses of memory address space associated with the workload, until the application can be executed at a stable level. The stable level of execution, for example, can be indicated by performance variations, such as standard deviations, between a certain number of neighboring measurement periods remaining within a certain threshold. The agent, who has been trained in the training episodes, infers the final cache management policy during the final, inferring episode.

Abstract: A system, method, and apparatus of power management for computing systems are included herein that optimize individual frequencies of components of the computing systems using machine learning. The computing systems can be tightly integrated systems that consider an overall operating budget that is shared between the components of the computing system while adjusting the frequencies of the individual components. An example of an automated method of power management includes: (1) learning, using a power management (PM) agent, frequency settings for different components of a computing system during execution of a repetitive application, and (2) adjusting the frequency settings of the different components using the PM agent, wherein the adjusting is based on the repetitive application and one or more limitations corresponding to a shared operating budget for the computing system.

Abstract: A reinforcement learning agent learns a congestion control policy using a deep neural network and a distributed training component. The training component enables the agent to interact with a vast set of environments in parallel. These environments simulate real world benchmarks and real hardware. During a learning process, the agent learns how maximize an objective function. A simulator may enable parallel interaction with various scenarios. As the trained agent encounters a diverse set of problems it is more likely to generalize well to new and unseen environments. In addition, an operating point can be selected during training which may enable configuration of the required behavior of the agent.

Abstract: A method for performing a Tree-Search (TS) on an environment is provided. The method comprises generating a tree for a current state of the environment based on a TS policy, determining a corrected TS policy, and determining an action to apply to the environment based on the corrected TS policy. The tree comprises a plurality of nodes including a root node among the plurality of nodes corresponding to the current state of the environment. Each node other than the root node among the plurality of nodes corresponding to an estimated future state of the environment. The plurality of nodes in the tree are connected by a plurality of edges. Each edge among the plurality of edges is associated with an action causing a transition from a first state to a different sate of the environment.

Abstract: A reinforcement learning agent learns a congestion control policy using a deep neural network and a distributed training component. The training component enables the agent to interact with a vast set of environments in parallel. These environments simulate real world benchmarks and real hardware. During a learning process, the agent learns how maximize an objective function. A simulator may enable parallel interaction with various scenarios. As the trained agent encounters a diverse set of problems it is more likely to generalize well to new and unseen environments. In addition, an operating point can be selected during training which may enable configuration of the required behavior of the agent.