Abstract: Some embodiments provide a method of reliably forwarding a result of an operation executed by a source EPU to a destination EPU connected to the source EPU through a network. At the source EPU, the method executes the operation to produce the result. The method divides the result into multiple blocks each having multiple segments identified by a segment ID. The method forwards, through the network to the destination EPU, a data message flow having data messages that store the segments of the result, each data message including a header that stores the segment ID and a payload that stores the segment associated with the segment ID. The method designates the forwarding of the result as incomplete until a block-complete notification is received, at the source EPU from the destination EPU, for each block to identify that each segment ID in the block has been received at the destination EPU.

Abstract: Some embodiments provide a method of reliably executing a single distributed application across multiple geographic sites. The method assigns a first set of operations of the distributed application to a first set of EPUs at a first geographic site to execute. The method assigns a second set of operations of the distributed application to a second set of EPUs at a second geographic site to execute. The method configures a network to forward executed operation results among the EPUs. The method configures the EPUs to use a TLP that ensures reliable forwarding of EPU operation results between EPUs in different sites. The TLP includes a mechanism for a destination EPU in the first site to use to confirm to a source EPU in the second site that the destination EPU has successfully received all segments of a result that the source EPU computed and then forwarded to the destination EPU.

Abstract: A network interface card (NIC) can be configured to monitor a first central processing unit (CPU) core mapped to a first receive queue having a receive queue length. The NIC can also be configured to determine whether the CPU core is overloaded based on the receive queue length. The NIC can also be configured to redirect data packets that were targeted from the first receive queue to the CPU core to another CPU core responsive to a determination that the CPU core is overloaded.

Abstract: Some embodiments provide a method of managing communication between endpoint processing units (EPUs) through a network having multiple forwarding elements. Sets of the EPUs execute operations associated with multiple distributed applications. The method provides a first set of infrastructure services to configure multiple forwarding elements to implement multiple virtual private clouds (VPCs) with each VPC connecting different sets of EPUs that execute different distributed applications. The method provides, for each VPC, a second set of scheduling services to schedule forwarding of results computed by the VPC's EPUs through the forwarding elements that form the VPC.

Abstract: Some embodiments provide a method of scheduling a forwarding of a first result of a first operation executed by the first endpoint processing unit (EPU) to a second EPU along a network connecting a plurality of EPUs. At the first EPU, the method forwards, for the first result, a scheduling request as an in-band control message through the network that forwards between the EPUs data message flows that store results of operations executed by the EPUs. In response to the scheduling request, the method receives a set of scheduling parameters from a control plane agent that obtains the in-band forwarded control message and provides the set of scheduling parameters back to the first EPU in an in-band reply message. The method uses the set of scheduling parameters to forward the result of the operation to the second EPU through the network.

Abstract: Some embodiments provide a method of executing a distributed application with multiple EPUs that perform computations for the distributed application. The EPUs are associated with network interfaces that connect the EPUs to forwarding elements that form a network for connecting the EPUs. The method configures a set of network interfaces of a set of EPUs to forward, to a set of forwarding elements, results of a first set of computations after completion of the first set of computations. The method configures the set of network interfaces of the set of EPUs (i) to request scheduling parameters for forwarding results of a second set of computations after completing the second plurality of computations, and (ii) to forward the results of the second set of computations to the set of forwarding elements based on the requested scheduling parameters.

Abstract: Some embodiments provide a method of managing communication between graphics processing units (GPUs) through a network having multiple forwarding elements. The method uses a set of servers to generate instructions for specifying forwarding behaviors for the forwarding elements to forward data messages containing results of computations performed by the GPUs. The GPUs perform the computations in order to collectively execute a distributed application. The method distributes the generated instructions to the forwarding elements to configure the forwarding elements to implement the forwarding behavior.

Abstract: Some embodiments provide a method of executing a distributed application with multiple EPUs that perform computations for the distributed application. The method configures multiple network interfaces of the EPUs to forward, among the EPUs, results of the computations performed by the EPUs for the distributed application. The configured network interfaces forward the EPU computation results through multiple forwarding elements that form a network that communicatively connects the EPUs. The method collects data regarding forwarding operations that the forwarding elements perform to forward results of the EPU computations. The method analyzes the collected data to detect that a particular forwarding element is not properly operating. The method re-configures at least a set of one or more network interfaces to circumvent the particular forwarding element as the set of network interfaces are forwarding their respective EPU computation results.

Abstract: Some embodiments provide a method of executing a distributed application with multiple EPUs that perform computations for the application. Each EPU is associated with a set of network interfaces with ports that connect the EPU to a network through which the EPUs forward results of the computations. The method generates, for first and second computations performed by a first EPU, first and second sets of configuration data to configure how first and second results are the forwarded through the set of ports of the first EPU. The method provides the configuration data to the network interfaces associated with the first EPU. The first set of configuration data directs the first EPU's network interfaces to forward the first result through one port. The second set of configuration data directs the first EPU's network interface set to forward the second result through two or more ports.

Abstract: Some embodiments provide a method of forwarding data messages from a source EPU to other endpoint processing units EPUs that are connected to the source EPU through a network. The EPUs perform computations to collectively execute a distributed application. At the source EPU that performs a computation to be sent to at least one destination EPU, the method uses an identity of the destination EPU to select a tag and associates the tag with a data message flow having multiple data messages to store a result of the computation. Each data message has a payload that contains a portion of the result and a header. At least one header of a data message stores the tag. The method uses the tag to identify an egress port of a network interface that connects the source EPU to the network, and forwards the data message flow through the identified egress port.

Abstract: Some embodiments provide a method of forwarding data messages between endpoint processing units (EPUs) that are connected through a network having multiple forwarding elements and that perform computations to collectively execute a distributed application. For each of multiple source EPUs, the method identifies a set of one or more paths through the network to forward results of computations performed by the source EPU to a set of destination EPUs. Each path includes a set of one or more hops along the network from the source EPU to a destination EPU. Each hop includes a forwarding element. For each identified path, the method identifies a tag to associate with the path, at each hop in the network along the path, to the next hop in the network, and distributes a record to each hop's forwarding element to associate the tag with a next hop along the identified path.

Abstract: Some embodiments provide a method, for multiple GPUs that perform computations to execute a distributed application, for forwarding a first result of a first computation performed by a source GPU to a destination GPU through a network connecting the GPUs. The method identifies first and second portions of the first result. The method selects, respectively for the first and second portions, first and second paths through the network from the source GPU to the destination GPU. The method forwards, from the source GPU, the first portion as a first data message flow that traverses along the first path to the destination GPU. The method forwards, from the source GPU, the second portion as a second data message flow that traverses along the second path to the destination GPU.

Abstract: Some embodiments provide a method for connecting multiple endpoint processing units (EPUs) that perform computations that are associated with the execution of a distributed application. The method configures multiple forwarding elements that are part of a network to communicatively connect each particular EPU to a set of two or more other EPUs to pass the results of the particular EPU's computations to other EPUs in the set of EPUs of the particular EPU. The method configures a set of one or more network interfaces of each particular EPU to forward results of the particular EPU's computations to the particular EPU's set of other EPUs through one or more configured forwarding elements.

Abstract: Some embodiments provide a method of forwarding data messages between EPUs that are connected through a network and that perform computations to collectively execute a distributed application. The method generates, for each computation of each source EPU, a data message flow having one or more data messages that store a result of the computation. The generating includes specifying, for each data message, an Ethernet header having a field that identifies the data message as an Ethernet frame and a MAC field that instead of a MAC address stores a tag for forwarding the data message to a destination in the network. The method uses, at each forwarding element that forwards each data message, the tag stored in the MAC field of the data message's header to identify an egress port of the forwarding element to forward the data message along a path through the network to the data message's destination.

Abstract: Some embodiments provide a method of executing a distributed application with multiple endpoint processing units (EPUs) that perform operations necessary for executing the distributed application. The method configures multiple forwarding elements that are part of a network to forward data message flows between the EPUs in order to allow the EPUs to share results of distributed-application operations that the EPUs execute. The method repeatedly collects and analyzes telemetry data from the forwarding elements. Upon detecting, based on the telemetry analysis, performance degradation at a particular forwarding element, the method reconfigures one or more forwarding elements to forward fewer data message flows through the particular forwarding element in order to mitigate the detected performance degradation.

Abstract: Some embodiments provide a method of executing a distributed application with multiple endpoint processing units (EPUs) that perform computations for the distributed application. The EPUs are connected through a network having multiple network elements. The method iteratively provides instructions to the EPUs to perform computations associated with the distributed application. The method stores a result of each EPU computation at the EPU until the result has been forwarded to a destination in the network and a confirmation has been received that the result has successfully been received at its destination in the network.

Abstract: Some embodiments provide a method of managing communication between endpoint processing units (EPUs) through a network having multiple forwarding elements. The EPUs collectively execute a distributed application. For a first operation that a first EPU executes for the distributed application and that depends on a set of two or more operations of the distributed application that are executed earlier by a set of two or more other EPUs, the method waits to receive confirmation that all EPUs in the set of the EPUs have completed their operations and have results that can be forwarded to the first EPU. After receiving the confirmations from all EPUs in the set of EPUs, the method sends a set of one or more instructions that direct the set of EPUs to forward their results to the first EPU.

Abstract: Some embodiments provide a network for communicatively coupling multiple endpoint processing units (EPUs). The network includes multiple intra-cluster switches each of which connects two or more EPUs within one cluster. The network includes multiple inter-cluster switches each to connect at least two different sets of EPUs organized in two different clusters. Each of a set of EPUs has one or more endpoint interfaces (EPIs) to connect the EPU to an intra-cluster switch and inter-cluster switch. Each EPI in at least a subset of EPIs also used to connect at least one inter-cluster switch to an inter-cluster switch.

Abstract: Some embodiments provide a method of managing communication between EPUs organized into groups. Each EPU group has multiple EPUs connected through at least one intra-group forwarding element and connects to at least one other EPU group through at least one inter-group forwarding element. The method uses a set of servers to generate instructions to specify forwarding behavior of intra-group forwarding elements to forward data messages containing results of computations performed by the EPUs to other EPUs within their same group. The method uses the same set of servers to generate instructions to specify forwarding behavior of inter-group forwarding elements to forward data messages containing results of computations performed by the EPUs to other EPUs in other groups. The method distributes, to each particular inter-group or intra-group forwarding element, the instructions generated for the particular forwarding element to use to forward data messages exchanged between the EPUs.

Abstract: Some embodiments provide a method of executing a distributed application with multiple endpoint processing units (EPUs) that perform computations for the distributed application. At a set of one or more scheduling servers, the method receives multiple API (application programming interface) commands related to computations assigned to a set of EPUs. The method generates, based on the API commands, different sets of scheduling parameters for different sets of computations assigned to the EPUs. The method distributes the generated sets of scheduling parameters to multiple network elements of a network that communicatively couples the set of EPUs. The distributed sets of scheduling parameters control the network elements forwarding of results of the assigned EPU computations through the network.