Abstract: A method for comparing systems includes running, simultaneously, a first system and a second system, wherein the first system and the second system process events, collecting first data from the first system based on the processing of the events, collecting second data from the second system based on the processing of the events, wherein the second system includes at least one feature different than the first system, and performing a sequential probability ratio test based on the first data and the second data.

Abstract: A method for determining piloting a first system includes receiving a first hypothesis, receiving first test parameters of a decision state space defined on a sequential probability ratio test plot of a number of failures of the first system versus a number of failures of a reference system, identifying for a arbitrary distribution of events, a first number of events to be processed by the first system and the reference system that will satisfy the first test parameters, determining a coefficient of variation of the arbitrary distribution of events, and determining whether to perform the sequential probability ratio test plot using the arbitrary distribution of events or historical data based on the coefficient of variation.

Abstract: Test characteristics are determined at vertices of a rectangle defined on a sequential probability ratio test plot of a number of failures of a new physical system versus a number of failures of a baseline physical system. Based on the test characteristics, errors of first and second order are approximated as planes. Based on the approximating, intercepts of accept and reject boundaries are estimated. Based on slopes of the planes and actual values of the errors of the first and second orders, a starting point is determined on a graph of values of an intercept of the reject boundary and values of an intercept of the accept boundary. A minimum of an operating characteristic function is determined by alternative advances, commencing at the starting point, parallel to an axis representing the accept boundary intercept and an axis representing the reject boundary intercept.

Abstract: Test characteristics are determined at vertices of a rectangle defined on a sequential probability ratio test plot of a number of failures of a new physical system versus a number of failures of a baseline physical system. Based on the test characteristics, errors of first and second order are approximated as planes. Based on the approximating, intercepts of accept and reject boundaries are estimated. Based on slopes of the planes and actual values of the errors of the first and second orders, a starting point is determined on a graph of values of an intercept of the reject boundary and values of an intercept of the accept boundary. A minimum of an operating characteristic function is determined by alternative advances, commencing at the starting point, parallel to an axis representing the accept boundary intercept and an axis representing the reject boundary intercept.