Abstract: Aspects of the present disclosure provide an apparatus. For example, the apparatus can include a compiler configured to compile a neural network (NN) model to generate a plurality of operations/threads and determine whether each of the operations/threads is compute bound or memory bound, and a memory coupled to the compiler and configured to store the operations/threads. The apparatus can also include a thread scheduler coupled to the memory and configured to schedule the operations/threads of the NN model. The apparatus can also include a multi-engine processing unit that includes a plurality of compute units (CUs), and an executor coupled between the thread scheduler and the multi-engine processing unit. The executor can be configured to allocate the operations/threads of the NN model and activate a number of the CUs of the multi-engine processing unit for each of the operations/threads based on whether the operation/thread is compute bound or memory bound.

Abstract: Aspects of the present disclosure provide a method for controlling a processing device to execute an application that runs on a neural network (NN). The processing device can include a plurality of processing units that are arranged in a network-on-chip (NoC) architecture. For example, the method can include obtaining compiler information relating the application and the NoC, controlling the processing device to employ a first routing scheme to process the application when the compiler information does not meet a predefined requirement, and controlling the processing device to employ a second routing scheme to process the application when the compiler information meets the predefined requirement.

Abstract: Aspects of the present disclosure provide a method for controlling a processing device to execute an application that employs a neural network (NN). The processing device includes a plurality of processing units arranged in a network-on-chip (NoC) to which the NN is mapped. For example, the method can include obtaining compiler information. The compiler information can include computing loads of the application on the processing units. The computing loads can relate a dataflow type of the NN. The method can also include determining a scaling factor for computing time of each of the processing units based on the computing loads, adjusting the computing time of the processing units based on the scaling factors, and enabling the processing units to perform their respective tasks of the application within their respective adjusted computing time.