Abstract: Aspects of the present disclosure provide an apparatus. For example, the apparatus can include a compiler configured to compile a neural network (NN) model to generate a plurality of operations/threads and determine whether each of the operations/threads is compute bound or memory bound, and a memory coupled to the compiler and configured to store the operations/threads. The apparatus can also include a thread scheduler coupled to the memory and configured to schedule the operations/threads of the NN model. The apparatus can also include a multi-engine processing unit that includes a plurality of compute units (CUs), and an executor coupled between the thread scheduler and the multi-engine processing unit. The executor can be configured to allocate the operations/threads of the NN model and activate a number of the CUs of the multi-engine processing unit for each of the operations/threads based on whether the operation/thread is compute bound or memory bound.