Abstract: A machine learning (ML) system and methods are provided that are configured to provide a performance evaluation of an online ML model using an evaluation framework with an offline ML model. The system includes a processor and a computer readable medium operably coupled thereto, the computer readable medium comprising a plurality of instructions stored in association therewith that are accessible to, and executable by, the processor, to perform model comparison operations which include accessing, for a performance evaluation of a first ML model, a second ML model using the evaluation framework, determining a batch size for the performance evaluation, calculating first model scores for an analytical metric during an online run using the batch size, calculating decayed weights applied to the first model scores, comparing the first model scores with second model scores for the second ML model, and outputting the performance evaluation based on the comparing.

Abstract: A rule training system and methods are provided that are configured to automatically generate machine learning (ML) rules for intelligent decision-making by a policy manager platform. The system includes a processor and a computer readable medium operably coupled thereto, the computer readable medium comprising a plurality of instructions stored in association therewith that are accessible to, and executable by, the processor, to perform rule training operations which include accessing rule training data, iteratively generating a plurality of decision rules based on the rule training data and a plurality of ML model training techniques, testing each of the plurality of decision rules, filtering the plurality of decision rules by corresponding performances, selecting a set of the plurality of decision rules based on alert metrics, evaluating the set of the plurality of decision rules, and generating a decision ruleset for the ML task.