Abstract: There is provided a method of monitoring performance of a machine learning model externally to the machine learning model, comprising: monitoring data elements being fed into a machine learning model trained on a training dataset of historical training data elements, wherein the data elements are each associated with a respective time after the time associated with the training dataset, analyzing the data elements for identifying shift(s) between at least two subsets of the data elements, computing according to the shift(s), measurement(s) denoting an expected effect on output of the model, and detecting a misclassification event by the model when the measurement(s) exceeds a threshold of the model, wherein the monitoring, the analyzing, the computing, and the detecting are performed externally to the model, without accessing at least one of: data stored within the machine learning model, an implementation of the model, and data structures of the model.

Abstract: There is provided a method of monitoring performance of a machine learning model externally to the machine learning model, comprising: monitoring data elements being fed into a machine learning model trained on a training dataset of historical training data elements, wherein the data elements are each associated with a respective time after the time associated with the training dataset, analyzing the data elements for identifying shift(s) between at least two subsets of the data elements, computing according to the shift(s), measurement(s) denoting an expected effect on output of the model, and detecting a misclassification event by the model when the measurement(s) exceeds a threshold of the model, wherein the monitoring, the analyzing, the computing, and the detecting are performed externally to the model, without accessing at least one of: data stored within the machine learning model, an implementation of the model, and data structures of the model.