Abstract: Provided is a system for generating an inference based on real-time selection of a machine learning model using a machine learning model framework that includes at least one processor programmed or configured to receive a request for inference, wherein the request includes a payload, select a machine learning model of a plurality of machine learning models based on the request for inference, determine an aggregation of data based on the machine learning model and the payload of the request, transform the aggregation of data into inference data, wherein the inference data has a configuration that is capable of being processed by the machine learning model, and generate an inference based on the inference data using the machine learning model. Methods and computer program products are also provided.

Abstract: A method, system, and computer program product for dynamically scheduling machine learning inference jobs receive or determine a plurality of performance profiles associated with a plurality of system resources, wherein each performance profile is associated with a machine learning model; receive a request for system resources for an inference job associated with the machine learning model; determine a system resource of the plurality of system resources for processing the inference job associated with the machine learning model based on the plurality of performance profiles and a quality of service requirement associated with the inference job; assign the system resource to the inference job for processing the inference job; receive result data associated with processing of the inference job with the system resource; and update based on the result data, a performance profile of the plurality of the performance profiles associated with the system resource and the machine learning model.

Abstract: Provided are systems for generating a machine learning model and a prediction based on encoded time series data using model reduction techniques that include a processor to receive a training dataset of a plurality of data instances, wherein each data instance includes a time series of data points, perform an encoding operation on the training dataset to provide an encoded dataset having a lower dimension space than a dimension space of the training dataset, generate one or more prediction models based on the encoded dataset, determine an output of the one or more prediction models in the lower dimension space based on an input provided to the one or more prediction models, and perform a decoding operation on the output to project the output from the lower dimension space to the dimension space of the training dataset. Methods and computer program products are also provided.

Abstract: A method, system, and computer program product for dynamically scheduling machine learning inference jobs receive or determine a plurality of performance profiles associated with a plurality of system resources, wherein each performance profile is associated with a machine learning model; receive a request for system resources for an inference job associated with the machine learning model; determine a system resource of the plurality of system resources for processing the inference job associated with the machine learning model based on the plurality of performance profiles and a quality of service requirement associated with the inference job; assign the system resource to the inference job for processing the inference job; receive result data associated with processing of the inference job with the system resource; and update based on the result data, a performance profile of the plurality of the performance profiles associated with the system resource and the machine learning model.

Abstract: A machine learning system includes a training platform and an inference platform, where the inference platform is coupled to receive the output of the training platform. Based upon an updating of hyperparameters in the training platform, an optimized inference model is configured to be deployed to the inference platform from the training platform. The optimized inference model is further optimized in the inference platform by using an observation difference between a client observation and a prediction response to update the optimized inference model. The updated optimized inference model is used to provide a prediction response to a client.

Abstract: A method, system, and computer program product for dynamically scheduling machine learning inference jobs receive or determine a plurality of performance profiles associated with a plurality of system resources, wherein each performance profile is associated with a machine learning model; receive a request for system resources for an inference job associated with the machine learning model; determine a system resource of the plurality of system resources for processing the inference job associated with the machine learning model based on the plurality of performance profiles and a quality of service requirement associated with the inference job; assign the system resource to the inference job for processing the inference job; receive result data associated with processing of the inference job with the system resource; and update based on the result data, a performance profile of the plurality of the performance profiles associated with the system resource and the machine learning model.

Abstract: A method, system, and computer program product for dynamically scheduling machine learning inference jobs receive or determine a plurality of performance profiles associated with a plurality of system resources, wherein each performance profile is associated with a machine learning model; receive a request for system resources for an inference job associated with the machine learning model; determine a system resource of the plurality of system resources for processing the inference job associated with the machine learning model based on the plurality of performance profiles and a quality of service requirement associated with the inference job; assign the system resource to the inference job for processing the inference job; receive result data associated with processing of the inference job with the system resource; and update based on the result data, a performance profile of the plurality of the performance profiles associated with the system resource and the machine learning model.

Abstract: A method, system, and computer program product for maintaining model state at model data centers hosting a same machine learning model may receive first input data input, at a first time, to a first implementation of a model to generate first output data, the first implementation of the model being associated with a first model state at a time before the first time; receive second input data input, at a second time different than the first time, to a second implementation of the model to generate second output data, the second implementation of the model being associated with a second model state at a time before the second time; determine, based on the first input data and the second input data, update data for the first model state of the first implementation and the second model state of the second implementation; and provide, at a third time subsequent to the first time and the second time, the update data.