Abstract: Machine learning version management method for a prediction service includes receiving a prediction request, determining application metadata for the request that defines routing logic and a machine learning framework version, determining model metadata for the request that defines at least one model and at least one model version, forwarding the prediction request to the at least one model with the at least one model version, and returning a prediction from the at least one model to a requestor.

Abstract: Methods and systems for machine learning inferencing based on directed acyclic graphs are presented. A request for a machine learning application is received from a tenant application. A tenant identifier that identifies one of the tenants is determined from the request. Based on the tenant identifier and a type of the machine learning application, configuration parameters and a graph structure are determined. The graph structure defines a flow of operations for the machine learning application. Nodes of the graph structure are executed based on the configuration parameters to obtain a scoring result. Execution of a node causes a machine learning model generated for the first tenant to be applied to data related to the request. The scoring result is returned in response to the request.

Abstract: Methods and systems for multi-model scoring in a multi-tenant system are presented. A request for a machine learning application is received from a tenant application. A tenant identifier that identifies one of the multiple tenants is determined. Based on the tenant identifier and a type of the machine learning application, a first and a second machine learning models are determined. The first machine learning model was generated based on a first training data set associated with the tenant identifier. The second machine learning model that was generated based on a second training data set associated with the tenant identifier. A flow of operations that includes running the first and second machine learning models with data related to the request is executed to obtain a scoring result. The scoring result is returned to the tenant application in response to the request.

Abstract: A method of a base scorer in a scoring service container includes sending a model identifier to a model loader of an application specific scorer in the scoring service container, receiving a model object from the model loader in response to sending the model identifier, sending a request for a scoring from a client application to a scoring function of the application specific scorer, receiving the scoring from the application specific scorer, and returning the scoring to the client application.

Abstract: Machine learning version management method for a prediction service includes receiving a prediction request, determining application metadata for the request that defines routing logic and a machine learning framework version, determining model metadata for the request that defines at least one model and at least one model version, forwarding the prediction request to the at least one model with the at least one model version, and returning a prediction from the at least one model to a requestor.