Abstract: Apparatuses, systems, and techniques to generate one or more graph code nodes to deallocate memory. In at least one embodiment, one or more graph code nodes to deallocate memory are generated, based on, for example, CUDA or other parallel computing platform code.

Abstract: Apparatuses, systems, and techniques to generate one or more graph code nodes to allocate memory. In at least one embodiment, one or more graph code nodes to allocate memory are generated, based on, for example, CUDA or other parallel computing platform code.

Abstract: System and methods to train a neural network to systematically find a cross-over point, given the number of devices (e.g., Graphical Processing Units) used to train a deep learning (DL) model, that indicates which parallelization strategy to implement when optimizing the training of the DL model on a particular system to achieve maximum efficiency gains.

Abstract: A method for data deduplication during execution of an application on a plurality of computing nodes, including: generating, by a first processor in a first computing node executing the application, a first message to process application data owned by a second computing node executing the application; receiving, by a first network interface (NI) of the first computing node, the first message; extracting, by the first NI, a first key from the first message; determining, by the first NI, the first key is not a duplicate; and placing, by the first NI and in response to the first key not being a duplicate, the first message on a network connecting the first computing node to the second computing node.

Abstract: Each computing node of a distributed computing system may implement a hardware mechanism at the network interface for message driven prefetching of application data. For example, a parallel data-intensive application that employs function shipping may distribute respective portions of a large data set to main memory on multiple computing nodes. The application may send messages to one of the computing nodes referencing data that is stored locally on the node. For each received message, the network interface on the recipient node may extract the reference, initiate the prefetching of referenced data into a local cache (e.g., an LLC), and then store the message for subsequent interpretation and processing by a local processor core. When the processor core retrieves a stored message for processing, the referenced data may already be in the LLC, avoiding a CPU stall while retrieving it from memory. The hardware mechanism may be configured via software.

Abstract: Each computing node of a distributed computing system may implement a hardware mechanism at the network interface for message driven prefetching of application data. For example, a parallel data-intensive application that employs function shipping may distribute respective portions of a large data set to main memory on multiple computing nodes. The application may send messages to one of the computing nodes referencing data that is stored locally on the node. For each received message, the network interface on the recipient node may extract the reference, initiate the prefetching of referenced data into a local cache (e.g., an LLC), and then store the message for subsequent interpretation and processing by a local processor core. When the processor core retrieves a stored message for processing, the referenced data may already be in the LLC, avoiding a CPU stall while retrieving it from memory. The hardware mechanism may be configured via software.

Abstract: A method for data deduplication during execution of an application on a plurality of computing nodes, including: generating, by a first processor in a first computing node executing the application, a first message to process application data owned by a second computing node executing the application; receiving, by a first network interface (NI) of the first computing node, the first message; extracting, by the first NI, a first key from the first message; determining, by the first NI, the first key is not a duplicate; and placing, by the first NI and in response to the first key not being a duplicate, the first message on a network connecting the first computing node to the second computing node.

Abstract: In a multi-chip module (MCM), integrated circuits are coupled by optical waveguides that convey optical signals. The optical waveguides provide dedicated point-to-point optical links between all pairs of the integrated circuits. Moreover, for a given point-to-point optical link between a given pair of integrated circuits, other integrated circuits in the integrated circuits steal access on the given point-to-point optical link when communicating information to one of the given pair of integrated circuits so that the given point-to-point optical link is shared by more than the given pair of integrated circuits. Furthermore, the integrated circuits recover errors in messages in the optical signals corrupted by collisions on the given point-to-point optical link using erasure coding. In this way, the MCM may provide an optical network with increased bandwidth relative to a point-to-point optical network.