Abstract: A computer-implemented performance evaluation method includes specifying a group of comparable entities and a benchmark against which the comparable entities are evaluated. The entities evaluated may be a process, technology, strategy, treatment, organization, individual, or other identifiable unit. A primary data matrix is arranged by data indices, and the primary matrix is sampled with replacement N times to bootstrap N observation matrices. Alternatively, a Monte Carlo approach can be used. Then, all the matrices are filled with measurement criteria, with each criterion being representative of a respective data index and a respective entity. A p-value estimate is returned that measures the statistical significance of the best of the comparable entities relative to the benchmark, where the p-value represents the probability of wrongly rejecting the null hypothesis that a best of the comparable entities has expected performance no better than that of a benchmark.

Abstract: A computer-implemented prediction model evaluation method includes specifying many prediction models and a benchmark model against which the prediction models will be evaluated. A primary data matrix is arranged by data indices, and the primary matrix is sampled with replacement N times to bootstrap N observation matrices. Then, all the matrices are filled with measurement criteria, with each criteria being representative of a respective data index and a respective model. A p-value estimate is returned that measures the statistical significance of the best prediction model relative to the benchmark, where the p-value represents the probability of wrongly rejecting the null hypothesis that a best prediction model has expected performance no better than that of a benchmark. The p-value accounts for the examination of all of the prediction models, i.e., the p-value depends on the examination of all of the models as a group, and not simply on a single model.