Abstract: A system is disclosed that includes capabilities for generating a Machine Learning (ML) inference pipeline for deploying an ML model using artifacts received from one or more training stages in an ML training pipeline. The system receives one or more artifacts for one or more training stages in a set of training stages in a ML training pipeline associated with an ML process. The system then identifies one or more inference stages in an ML inference pipeline that correspond to the one or more training stages in the ML training pipeline. For each inference stage that corresponds to a training stage, the system associates the artifact received for the training stage with the inference stage. The system then generates the ML inference pipeline comprising the inference stages and their associated artifacts, where the artifacts include the artifacts received for the training stages in the ML training pipeline.

Abstract: Techniques discussed herein include dynamically providing synchronous and/or asynchronous data processing by a machine-learning model service. The machine-learning model service (“the service”) executes a stream manager application, a web interface, and a machine-learning model via a common container. The stream manager application can obtain input data (e.g., from an input data stream, a partition of an input data stream, etc.) and provide the data to the machine-learning model through the web interface using a local communication channel (e.g., a loopback interface that bypasses local network interface hardware of the computing device on which the model executes). Prediction results from the model may be provided as output data (e.g., to an output data stream, to a partition of an output data stream, etc.).