Abstract: Small and compact Deep Learning models are required for embedded Al in several domains. In many industrial use-cases, there are requirements to transform already trained models to ensemble embedded systems or re-train those for a given deployment scenario, with limited data for transfer learning. Moreover, the hardware platforms used in embedded application include FPGAs, AI hardware accelerators, System-on-Chips and on-premises computing elements (Fog/Network Edge). These are interconnected through heterogenous bus/network with different capacities. Method of the present disclosure finds how to automatically partition a given DNN into ensemble devices, considering the effect of accuracy—latency power—tradeoff, due to intermediate compression and effect of quantization due to conversion to AI accelerator SDKs.