Abstract: Disclosed are methods and systems for creating, evaluating, and tuning a predictive engine for machine learning, and for replaying performance of the predictive engine, including steps to deploy a variant of the predictive engine; receive a plurality of queries to the predictive engine variant from an end-user device; apply a query segment filter to the plurality of queries to generate a replay group of queries, wherein for each query in the replay group, the engine variant generates a corresponding predicted result, receives a corresponding actual result, and computes a corresponding prediction score; group queries in the replay group into one or more segments; and generate, as a replay of the performance of the engine variant, an accumulated prediction score over each of the one or more query segments of the replay group by applying an accumulation function. The present invention substantially improves on systems that utilize predictive engines.

Abstract: Disclosed are methods and systems of creating, evaluating, and tuning a predictive engine for machine learning, including steps to deploy the predictive engine with an initial parameter set; receive queries to the deployed engine variant and in response, generate predicted results; receive corresponding actual results; associate the queries, the predicted results, and the actual results with a replay tag; evaluate the performance of the deployed engine variant; generate a new engine parameter set based on tuning of one or more parameters of the initial engine parameter set, according to the evaluation results; deploy the new engine variant to replace the initial engine variant; receive a replay request from an operator specifying the currently or a previously deployed engine variant; and in response to the replay request, replay at least one of the queries, the corresponding predicted results, the actual results, and the evaluation results.