Abstract: This disclosure relates to method and system to estimate model performance based on model decision boundaries. Absence of ground truths in a production scenario makes difficult to estimate the model behavior and performance. The disclosed method receives a set of unlabeled datapoints as input and a set of features are extracted from a training dataset associated with a model for which the performance is being estimated. Further, a nearest neighbor point of each unlabeled datapoint is computed from the training dataset with a first distance and a second distance and an associated ground truth. The performance of the model is estimated by evaluating a label assigned to each unlabeled datapoint from the set of unlabeled datapoints. Then, a decision boundary radius of the model is computed for the nearest neighbor point of each unlabeled datapoint based on the training dataset and the tolerance value using a Bayesian optimization technique.

Abstract: Performance of a machine learning (ML) model in production, is heavily dependent on underlying distribution of data or underlying process generating labels from attributes. Any change in either one or both impacts the ML model performance heavily and inhibits knowledge of true labels. This in turn affects ML model uncertainty. Thus, performance monitoring of ML models in production becomes necessary. Embodiments of the present disclosure provide estimates operating model accuracy at production stage by constructing the correlations between the model accuracy, model uncertainty and deviation of the distributions in absence of ground truth. In the method of present disclosure, the model performance of the machine learning (ML) model deployed in production is estimated in absence of ground truths. Moreover, this can be done without retraining the model, thus saving computational costs and resources. The method of the present disclosure can be used and performed in real time.