Abstract: Aspects of the present disclosure provide accelerated transfer learning for machine learning models, such as neural networks, as a service. In doing so, the aspects disclosed herein decouple the conversion process for use of machine learning models from deployment of the models in a production environment through the creation of one or more common generic models that execute on a hardware accelerator, such as a FPGA. Additional aspects of the disclosure relate to the creation of an accelerated machine learning model. The accelerated machine learning model is created by identifying common portions of a machine learning model that can be leveraged by a plurality of different machine learning models. Still further aspects of the disclosure relate to training scenario specific machine learning models for use with an accelerated machine learning model.