Abstract: A system, method, and computer-program product includes obtaining, via an application programming interface (API), a test object that includes application usage data of a deployed AI application for a target time span, executing, in real-time by one or more computer processors, one or more application behavior tests that assess an operational behavior of the deployed AI application, detecting, by the one or more computer processors, that a misbehavior occurred in the deployed AI application during the target time span and one or more deviant features contributing to the misbehavior in response to executing the one or more application behavior tests, and returning, by the one or more computer processors, the one or more deviant features contributing to the misbehavior to a subscribing entity associated with the deployed AI application.

Abstract: A system, method, and computer-program product includes obtaining, via an application programming interface (API), a test object that includes application usage data of a deployed AI application for a target time span, executing, in real-time by one or more computer processors, one or more application behavior tests that assess an operational behavior of the deployed AI application, detecting, by the one or more computer processors, that a misbehavior occurred in the deployed AI application during the target time span and one or more deviant features contributing to the misbehavior in response to executing the one or more application behavior tests, and returning, by the one or more computer processors, the one or more deviant features contributing to the misbehavior to a subscribing entity associated with the deployed AI application.

Abstract: Systems and methods for tuning hyperparameters of a model include receiving a tuning request for tuning hyperparameters, the tuning request includes a first and a second objective function for the machine learning model. The first and second objective functions may output metric values that do not improve uniformly. Systems and methods additionally include defining a joint tuning function that is based on a combination of the first and second objective functions; executing a tuning operation; identifying a Pareto efficient frontier curve defined by a plurality of distinct hyperparameter values; applying metric thresholds to the Pareto efficient frontier curve; demarcating the Pareto efficient frontier curve into at least a first infeasible section and a second feasible section; searching the second feasible section of the Pareto efficient frontier curve for one or more proposed hyperparameter values; and identifying at least a first set of proposed hyperparameter values based on the search.

Abstract: Systems and methods for tuning hyperparameters of a model include receiving a tuning request for tuning hyperparameters, the tuning request includes a first and a second objective function for the machine learning model. The first and second objective functions may output metric values that do not improve uniformly. Systems and methods additionally include defining a joint tuning function that is based on a combination of the first and second objective functions; executing a tuning operation; identifying a Pareto efficient frontier curve defined by a plurality of distinct hyperparameter values; applying metric thresholds to the Pareto efficient frontier curve; demarcating the Pareto efficient frontier curve into at least a first infeasible section and a second feasible section; searching the second feasible section of the Pareto efficient frontier curve for one or more proposed hyperparameter values; and identifying at least a first set of proposed hyperparameter values based on the search.