Abstract: An interconnect topology for communication between GPUs in a computing system is determined. A quantity of directed spanning trees are generated for transmitting data between the GPUs using the interconnect topology and packed. The directed spanning trees define the connections between GPUs that are to be utilized for the transmission and the amount of data to be transmitted on each connection. Program code is generated for implementing the data transfer defined by the directed spanning trees. When the program code is executed, the directed spanning trees are used to pipeline the transmission of chunks of data, such as model parameters used during data-parallel DNN training, between the GPUs. The program code can also determine an optimal chunk size for data to be transferred between the GPUs.

Abstract: Technologies are described herein for event delivery and stream processing utilizing virtual processing agents. Upon receiving an event publication in a queue, a runtime system identifies one or more virtual processing agents that might be interested in, but have not explicitly subscribed to, the published event. Event information of the published event is then delivered to the identified virtual processing agents. Prior to the actual delivery, the runtime system further determines if the virtual processing agents have been activated and activates those processing agents that have not been activated. Based on the received event information, some of the virtual processing agents might decide to explicitly submit subscriptions to receive more events from the queue. The explicit subscriptions will trigger the runtime system to deliver the subscribed events to the processing agents, which might include past events that have been published in the queue before the explicit subscription is received.

Abstract: Technologies are disclosed herein for dynamically generating communication primitives for use in model parameter synchronization during data-parallel DNN training by packing directed spanning trees. An interconnect topology for communication between GPUs in a computing system is determined. A quantity of directed spanning trees are generated for transmitting data between the GPUs using the interconnect topology and packed. The directed spanning trees define the connections between GPUs that are to be utilized for the transmission and the amount of data to be transmitted on each connection. Program code is generated for implementing the data transfer defined by the directed spanning trees. When the program code is executed, the directed spanning trees are used to pipeline the transmission of chunks of data, such as model parameters used during data-parallel DNN training, between the GPUs. The program code can also determine an optimal chunk size for data to be transferred between the GPUs.

Abstract: Technologies are described herein for event delivery and stream processing utilizing virtual processing agents. Upon receiving an event publication in a queue, a runtime system identifies one or more virtual processing agents that might be interested in, but have not explicitly subscribed to, the published event. Event information of the published event is then delivered to the identified virtual processing agents. Prior to the actual delivery, the runtime system further determines if the virtual processing agents have been activated and activates those processing agents that have not been activated. Based on the received event information, some of the virtual processing agents might decide to explicitly submit subscriptions to receive more events from the queue. The explicit subscriptions will trigger the runtime system to deliver the subscribed events to the processing agents, which might include past events that have been published in the queue before the explicit subscription is received.

Abstract: Technologies are described herein for event delivery and stream processing utilizing virtual processing agents. Upon receiving an event publication in a queue, a runtime system identifies one or more virtual processing agents that might be interested in, but have not explicitly subscribed to, the published event. Event information of the published event is then delivered to the identified virtual processing agents. Prior to the actual delivery, the runtime system further determines if the virtual processing agents have been activated and activates those processing agents that have not been activated. Based on the received event information, some of the virtual processing agents might decide to explicitly submit subscriptions to receive more events from the queue. The explicit subscriptions will trigger the runtime system to deliver the subscribed events to the processing agents, which might include past events that have been published in the queue before the explicit subscription is received.

Abstract: The subject disclosure is directed towards virtual components, e.g., comprising software components such as virtual components of a distributed computing system. Virtual components are available for use by distributed computing system applications, yet managed by the distributed computing system runtime transparent to the application with respect to automatic activation and deactivation on runtime-selected distributed computing system servers. Virtualization of virtual components is based upon mapping virtual components to their physical instantiations that are currently running, such as maintained in a global data store.

Abstract: Technologies are described herein for event delivery and stream processing utilizing virtual processing agents. Upon receiving an event publication in a queue, a runtime system identifies one or more virtual processing agents that might be interested in, but have not explicitly subscribed to, the published event. Event information of the published event is then delivered to the identified virtual processing agents. Prior to the actual delivery, the runtime system further determines if the virtual processing agents have been activated and activates those processing agents that have not been activated. Based on the received event information, some of the virtual processing agents might decide to explicitly submit subscriptions to receive more events from the queue. The explicit subscriptions will trigger the runtime system to deliver the subscribed events to the processing agents, which might include past events that have been published in the queue before the explicit subscription is received.

Abstract: The subject disclosure is directed towards virtual components, e.g., comprising software components such as virtual components of a distributed computing system. Virtual components are available for use by distributed computing system applications, yet managed by the distributed computing system runtime transparent to the application with respect to automatic activation and deactivation on runtime-selected distributed computing system servers. Virtualization of virtual components is based upon mapping virtual components to their physical instantiations that are currently running, such as maintained in a global data store.