Abstract: A reinforcement-learning based queue-management method for undersea networks. In one embodiment, a computer-implemented method to shape traffic in a dynamic network, comprising measuring network states, classifying each of a plurality of packets into one of a plurality of traffic types based on quality-of-service requirements, defining a decision epoch interval, determining a policy with a plurality of neural networks and with a deep reinforcement learning agent that maximizes entropy-regularized long-term reward and produces a plurality of weighted fair-queuing weights based on an observation of the state of the queues at the decision epoch intervals, and using the policy to allocate predefined portions of the available transmission bandwidth across the plurality of traffic types by defining each type with a WFQ weight.

Abstract: A reinforcement-learning based queue-management method for undersea networks. In one embodiment, a computer-implemented method to shape traffic in a dynamic network, comprising measuring network states, classifying each of a plurality of packets into one of a plurality of traffic types based on quality-of-service requirements, defining a decision epoch interval, determining a policy with a plurality of neural networks and with a deep reinforcement learning agent that maximizes entropy-regularized long-term reward and produces a plurality of weighted fair-queuing weights based on an observation of the state of the queues at the decision epoch intervals, and using the policy to allocate predefined portions of the available transmission bandwidth across the plurality of traffic types by defining each type with a WFQ weight.