Abstract: An electronic device and method for partitioning an accelerator and scheduling batches are disclosed. An electronic device includes one or more processors, and a memory storing instructions configured to cause the one or more processors to, for a first partitioning of an accelerator into partitions of different sizes, based on resource utilization of the partitions batch of different sizes input to the partition, determine correspondences between the sizes of the batches and the sizes of the partitions in the first partitioning, determine numbers of partitions for the respective determined sizes of the partitions based on the correspondences between the sizes of the batches and the sizes of the partitions in the first partitioning, and partition the accelerator into a second partitioning based on the determined numbers of the respective sizes of the partitions.

Abstract: An electronic device to co-locating models and a method of operating the electronic device is provided. The electronic device includes one or more of processors configured to analyze computational characteristics in response to a plurality of models being located to an accelerator, determine an affinity representing a utilization of the accelerator in response to two models among the plurality of models being co-located based on the computational characteristics of the plurality of models, and co-locate the two models among the plurality of models to the accelerator based on the affinity.

Abstract: A method of operating the accelerator includes receiving a request for preemption during an execution of a first task using one or more processing elements included in the accelerator, in response to the request for preemption, moving context information of the first task stored in an internal memory of the accelerator to an external memory of the accelerator, and executing a second task associated with the request for preemption using the processing elements.

Abstract: Disclosed are a neural network acceleration system and an operating method of the same. The neural network acceleration system includes a first memory module that generates a first reduced embedding segment through a tensor operation, based on a first segment of a first embedding and a second segment of a second embedding, a second memory module that generates a second reduced embedding segment through the tensor operation, based on a third segment of the first embedding and a fourth segment of the second embedding, and a processor that processes a reduced embedding including the first reduced embedding segment and the second reduced embedding segment, based on a neural network algorithm.

Abstract: A computing system includes a host processor, a plurality of accelerators that communicate with the host processor based on a communication interface, and a plurality of memory nodes that are connected with the plurality of accelerators through an interconnection network. A first data link is established between a first accelerator of the plurality of accelerators and a first memory node of the plurality of memory nodes, and a second data link is established between the first accelerator and a second memory node of the plurality of memory nodes.

Abstract: A system, method, and computer program product are provided for providing prioritized access for multithreaded processing. The method includes the steps of allocating threads to process a workload and assigning a set of priority tokens to at least a portion of the threads. Access to a resource, by each one of the threads, is based on the priority token assigned to the thread and the threads are executed by a multithreaded processor to process the workload.

Abstract: A system, method, and computer program product are provided for providing prioritized access for multithreaded processing. The method includes the steps of allocating threads to process a workload and assigning a set of priority tokens to at least a portion of the threads. Access to a resource, by each one of the threads, is based on the priority token assigned to the thread and the threads are executed by a multithreaded processor to process the workload.