Abstract: A memory management method performed by a multi-node computing system includes: transmitting, by a first node, to a second node, first data stored in a first data space of memory of the first node; receiving, by the first node, a write request from the second node, the write request including first modified data and a node identifier identifying the second node, and the first modified data being generated by the second node modifying the first data; determining, by the first node, whether to activate multi-node coherence verification with respect to the first data space based on a bit value of a first verification activation bit; and determining whether to store, by the first node, the first modified data in the first data space without activating the multi-node coherence verification based on the bit value of the first verification activation bit.

Abstract: An electronic device and method with task allocation are provided. The electronic device includes one or more processors configured to determine a target computing node to which a task is to be allocated among a plurality of computing nodes, based on a degradation degree of the plurality of computing nodes, and allocate the task to the target computing node, in which the plurality of computing nodes includes processing units of different types configured to execute the task.

Abstract: An electronic device includes one or more processors configured to determine whether to activate a message split circuitry included in a network interface card (NIC) based on either one or both of information on one or more running applications and an estimated latency, and store the determination of whether to activate the message split circuitry in a split register included in the NIC, and the NIC configured to obtain a message transmission request for transmitting one or more portions of a target message to a destination based on the one or more running applications, split, using the message split circuitry of the NIC, the one or more portions of the target message into two or more packets using the message split circuitry and transmit the two or more packets to the destination in response to a value indicating activation of the message split circuitry being stored in the split register, and transmit the one or more portions of the target message, as a single packet, to the destination in response to a val

Abstract: A processor-implemented method includes allocating tensors to a memory to perform an initial iteration of training of a deep learning application, wherein the initial iteration is performed through execution of a plurality of kernels and, in response to each kernel being executed, tensors corresponding to the each kernel are used to execute the kernels, storing pattern information for allocating the tensors and the kernels to the memory in the initial iteration, and prefetching the tensors based on the pattern information to perform a next iteration of training of the deep learning application.

Abstract: A method including updating memory allocation information of a UVM based on block information of model data blocks used for an execution of a deep learning model by a deep learning framework, and performing a least recently used (LRU) eviction based on the updated memory allocation information.

Abstract: A method and apparatus for managing a unified virtual memory (UVM) are provided. The UVM is backed by a main processor memory and a coprocessor memory, and the method includes: checking properties of data blocks of the UVM used to execute a deep learning model; based on a first of the data blocks storing weight data of the deep learning model, storing the first data block in the main processor memory among the main processor memory and the coprocessor memory; and performing an operation of the deep learning model based on the first data block using a coprocessor while directly loading at least a portion of the first data block from the main processor memory into a cache memory of the coprocessor without migration of the first data block from the main processor memory to the coprocessor memory.

Abstract: A method and apparatus for managing a unified virtual memory (UVM) are provided. The UVM is backed by a main processor memory and a coprocessor memory, and the method includes: checking properties of data blocks of the UVM used to execute a deep learning model; based on a first of the data blocks storing weight data of the deep learning model, storing the first data block in the main processor memory among the main processor memory and the coprocessor memory; and performing an operation of the deep learning model based on the first data block using a coprocessor while directly loading at least a portion of the first data block from the main processor memory into a cache memory of the coprocessor without migration of the first data block from the main processor memory to the coprocessor memory.

Abstract: A method and apparatus for managing a unified virtual memory (UVM) are provided. The UVM is backed by a main processor memory and a coprocessor memory, and the method includes: checking properties of data blocks of the UVM used to execute a deep learning model; based on a first of the data blocks storing weight data of the deep learning model, storing the first data block in the main processor memory among the main processor memory and the coprocessor memory; and performing an operation of the deep learning model based on the first data block using a coprocessor while directly loading at least a portion of the first data block from the main processor memory into a cache memory of the coprocessor without migration of the first data block from the main processor memory to the coprocessor memory.

Abstract: A method including updating memory allocation information of a UVM based on block information of model data blocks used for an execution of a deep learning model by a deep learning framework, and performing a least recently used (LRU) eviction based on the updated memory allocation information.

Abstract: Disclosed is an SLO-aware artificial intelligence inference scheduler technology in a heterogeneous processor-based edge system. A scheduling method for a machine learning (ML) inference task, which is performed by a scheduling system, may include receiving inference task requests of multiple ML models with respect to an edge system composed of heterogeneous processors and operating heterogeneous processor resources of the edge system based on a service-level objective (SLO)-aware-based scheduling policy in response to the received inference task requests.