Abstract: System and method for optimizing data-transfer among multiple compute units in a data-parallel computing system. A topological communications configurator (TCC) determines a connections-optimized configuration of processors associated with compute nodes of the computing system. The processors can execute dataflow workers of an application and form intranodal segments of an internodal interconnection topology coupling the intranodal segments. The TCC determines the connections-optimized configuration based on internodal communications costs corresponding to communications routes among the internodal segments via the internodal interconnection fabric.

Abstract: A system for a data-parallel execution of at least two implementations of an application on reconfigurable processors with different layouts is presented. The system comprises a pool of reconfigurable data flow resources with data transfer resources that interconnect first and second reconfigurable processors having first and second layouts that impose respective first and second constraints for the data-parallel execution of the application.

Abstract: A coarse-grained reconfigurable processor (CGRP) system. The CGRP system includes a set of coarse-grained reconfigurable units (CGRUs) in a first coarse-grained reconfigurable processor that is coupled to a first memory, a network interface including an external direct memory access (DMA) engine coupled to the first memory, and a work queue associated with the external DMA engine.

Abstract: A system for a data-parallel execution of at least two implementations of an application on reconfigurable processors with different layouts is presented. The system comprises a pool of reconfigurable data flow resources with data transfer resources that interconnect first and second reconfigurable processors having first and second layouts that impose respective first and second constraints for the data-parallel execution of the application. The system further comprises an archive of configuration files and a host system that is operatively coupled to the first and second reconfigurable processors. The host system comprises first and second compilers that generate for the application, based on the respective first and second constraints, first and second configuration files that are stored in the archive of configuration files and adapted to be executed data-parallel compatible on respective first and second reconfigurable processors.

Abstract: A coarse-grained reconfigurable processor (CGRP) system for implementing data-parallel training of a neural network is presented. The CGRP system includes a set of coarse-grained reconfigurable units (CGRUs) in a first CGRP configured to implement at least a portion of the neural network, to determine first and second gradients, respectively, of first and second model parameters based on a batch of training data, and to store the first and second gradients in a memory, a network interface including an external direct memory access (DMA) engine coupled between the memory and a network, and a work queue associated with the external DMA engine, wherein completion of determining the first gradient triggers a first work queue entry of the work queue that directs the external DMA engine to transfer the first gradient from the memory over the network to another memory coupled to a second CGRP for a gradient reduction operation.

Abstract: System and method for optimizing data-transfer among multiple compute units in a data-parallel computing system. A topological communications configurator (TCC) determines a connections-optimized configuration of processors associated with compute nodes of the computing system. The processors can execute dataflow workers of an application and form intranodal segments of an internodal interconnection topology coupling the intranodal segments. The TCC determines the connections-optimized configuration based on internodal communications costs corresponding to communications routes among the internodal segments via the internodal interconnection fabric.

Abstract: A host system for executing an application on first and/or second reconfigurable processors is presented. The host system is operatively coupled to the first and second reconfigurable processors, whereby the first reconfigurable processors have a first architecture, and the second reconfigurable processors have a second architecture that is different than the first architecture. The host system allocates reconfigurable processors of the first and/or second reconfigurable processors for executing the application and includes an auto-discovery module that is configured to determine whether the allocated reconfigurable processors include at least one of the first reconfigurable processors.

Abstract: A computer-implemented method comprises a topological communications configurator (TCC) of a computing system determining a connections-optimized configuration of processors among compute nodes of the system. Processors included in the compute nodes can execute compute workers of an application of the system and can form intranodal segments of an internodal interconnection topology communicatively coupling the intranodal segments. The intranodal segments can be interconnected via an internodal interconnections fabric. The TCC can determine the connections-optimized configuration based on internodal communications costs corresponding to communications routes among the internodal segments via the internodal interconnection fabric. The computing system can comprise the TCC and can comprise a data-parallel computing system.

Abstract: In a computer-implemented method a Dynamic Transfer Engine (DTE) included in a computing system receives a dynamic stimulus associated with transfer of stage data during execution of a dataflow application by the system. The DTE determines, based on source and destination devices of the transfer, a transfer method and a transfer channel to transfer the stage data between memories coupled to the source and destination devices. The DTE acquires, hardware resources of the computing system to transfer the stage using the channel and, initiates the transfer. A computer program product can cause one or more processors to perform the method. A computing system can comprise source and destination processors and memories, hardware channels to transfer data between the memories, a resource manager, and a DTE configured to perform the method.

Abstract: A system for executing an application on a pool of reconfigurable processors with first and second reconfigurable processors having first and second architectures that are different from each other is presented. The system comprises an archive of configuration files with first and second configuration files for executing the application on the first and second reconfigurable processors, respectively, and a host system that is operatively coupled to the first and second reconfigurable processors. The host system comprises a runtime processor that allocates reconfigurable processors for executing the application and an auto-discovery module that is configured to perform discovery of whether the reconfigurable processors include at least one of the first reconfigurable processors and whether the reconfigurable processors include at least one of the second reconfigurable processors.

Abstract: In a method an orchestrator of a computing system determines that results of Machine Learning model computations are available and dispatches a worker to perform model computations that include computing gradients of the results. The orchestrator determines that a set of gradients of the results is available and dispatches a gradient worker to compute a sum of the gradients. The orchestrator determines that a second set of gradients of the results is available and dispatches a second gradient worker to compute a sum of the second set of gradients. The orchestrator determines that the sums of the first and second gradients are available and dispatches a third gradient worker to compute synchronized gradients. The gradient workers compute the sums and synchronized gradients concurrent with training workers computing additional model computations results and/or gradients. A computer program product can include the method and a computing system can include the orchestrator.

Abstract: A system for a data-parallel execution of at least two implementations of an application on reconfigurable processors with different layouts is presented. The system comprises a pool of reconfigurable data flow resources with data transfer resources that interconnect first and second reconfigurable processors having first and second layouts that impose respective first and second constraints for the data-parallel execution of the application. The system further comprises an archive of configuration files and a host system that is operatively coupled to the first and second reconfigurable processors. The host system comprises first and second compilers that generate for the application, based on the respective first and second constraints, first and second configuration files that are stored in the archive of configuration files and adapted to be executed data-parallel compatible on respective first and second reconfigurable processors.

Abstract: A system for executing an application on a pool of reconfigurable processors with first and second reconfigurable processors having first and second architectures that are different from each other is presented. The system comprises an archive of configuration files with first and second configuration files for executing the application on the first and second reconfigurable processors, respectively, and a host system that is operatively coupled to the first and second reconfigurable processors. The host system comprises a runtime processor that allocates reconfigurable processors for executing the application and an auto-discovery module that is configured to perform discovery of whether the reconfigurable processors include at least one of the first reconfigurable processors and whether the reconfigurable processors include at least one of the second reconfigurable processors.

Abstract: A computer-implemented method comprises a topological communications configurator (TCC) of a computing system determining a connections-optimized configuration of processors among compute nodes of the system. Processors included in the compute nodes can execute compute workers of an application of the system and can form intranodal segments of an internodal interconnection topology communicatively coupling the intranodal segments. The intranodal segments can be interconnected via an internodal interconnections fabric. The TCC can determine the connections-optimized configuration based on internodal communications costs corresponding to communications routes among the internodal segments via the internodal interconnection fabric. The computing system can comprise the TCC and can comprise a data-parallel computing system.