Abstract: Embodiments of the present invention disclose a method, computer program product, and system for mitigating machine learning performance digression due to insufficient test data availability. A set of data is received, wherein the received set of data is parsed into a set of training data and a set of test data. A trained model is generated and the trained model is applying to the set of test data. A first set of performance values of the tested trained model are recorded and, if above a threshold, associated with a performance baseline value. A set of modified test data is generated and the trained model is applied to the set of modified test data. A second set of performance values are recorded and a performance difference value is calculated based on the performance baseline value and second set of recorded performance values. A table of results is generated, for display.

Abstract: A processor may acquire a trained predictive computational model from a database. The processor may apply a trained reduced complexity model to the trained predictive computational model. The trained reduced complexity model may be associated with the trained predictive computational model. The processor may select at least one metric. The processor may determine a quality indicator related to the at least one metric by identifying the type of the at least one metric, evaluating the output of the trained predictive computational model in relation to the type of the at least one metric, and generating, based on the evaluation of the trained predictive computational model, a threshold associated with the at least one metric. The processor may determine the accuracy of the trained predictive computational model based on the quality indicator.

Abstract: Embodiments of the present invention provide a method, system and computer program product for automated early anomaly detection in a continuous learning model. In an embodiment of the invention, a method includes training a continuous learning model with a training data set of different records and a known target class for each of the different records, deploying the model, and monitoring performance of the model. The method further includes prior to receiving a complete feedback data set for the model, computing a metric in the model based upon unseen records in the model that had not been present in the training data set, determining poor quality of the model for a metric computed to exceed a threshold value and displaying a recommendation in the host server to retrain the model responsive to the determination of poor quality of the model.

Abstract: A current data set with multiple records is fed into a data analysis model. The current data set is limited to data occurring in a current time window with a predetermined window size. The model is run on the current data set and a current data prediction result is generated. Limited historical data sets having multiple records are selected. Each record has values for several features. Each historical data set is limited to data occurring in a historical time window having the window size. A historical class label distribution is determined for the historical data sets and an upper and lower control limit are determined for the historical data sets using the historical class label distribution. A current class label distribution having a mean value is determined for the current prediction result. An alert is provided when the mean value is not between the upper and lower control limit.

Abstract: A processor may acquire a trained predictive computational model from a database. The processor may apply a trained reduced complexity model to the trained predictive computational model. The trained reduces complexity model may be associated with the trained predictive computational model. The processor may select at least one metric. The processor may determine a quality indicator related to the at least one metric by identifying the type of the at least one metric, evaluating the output of the trained predictive computational model in relation to the type of the at least one metric, and generating, based on the evaluation of the trained predictive computational model, a threshold associated with the at least one metric. The processor may determine the accuracy of the trained predictive computational model based on the quality indicator.

Abstract: Embodiments of the present invention disclose a method, computer program product, and system for mitigating machine learning performance digression due to insufficient test data availability. A set of data is received, wherein the received set of data is parsed into a set of training data and a set of test data. A trained model is generated and the trained model is applying to the set of test data. A first set of performance values of the tested trained model are recorded and, if above a threshold, associated with a performance baseline value. A set of modified test data is generated and the trained model is applied to the set of modified test data. A second set of performance values are recorded and a performance difference value is calculated based on the performance baseline value and second set of recorded performance values. A table of results is generated, for display.

Abstract: A current data set with multiple records is fed into a data analysis model. The current data set is limited to data occurring in a current time window with a predetermined window size. The model is run on the current data set and a current data prediction result is generated. Limited historical data sets having multiple records are selected. Each record has values for several features. Each historical data set is limited to data occurring in a historical time window having the window size. A historical class label distribution is determined for the historical data sets and an upper and lower control limit are determined for the historical data sets using the historical class label distribution. A current class label distribution having a mean value is determined for the current prediction result. An alert is provided when the mean value is not between the upper and lower control limit.