Abstract: The claimed subject matter includes techniques for providing an application testing service with a user interface that enables a user to evaluate performance data for computer implemented decision policies. An example method includes displaying a first reward statistic comprising an actual performance result for a policy implemented by an application. The method also includes obtaining experimental data corresponding to previously implemented policies, computing a second reward statistic for a hypothetical policy using a reward function applied to the experimental data. The method also includes displaying the second reward statistic together with the first reward statistic to enable a user to compare the first reward statistic and the second first reward statistic.

Abstract: The claimed subject matter includes techniques for providing an application testing service with a user interface that enables a user to evaluate performance data for computer implemented decision policies. An example method includes displaying a first reward statistic comprising an actual performance result for a policy implemented by an application. The method also includes obtaining experimental data corresponding to previously implemented policies, computing a second reward statistic for a hypothetical policy using a reward function applied to the experimental data. The method also includes displaying the second reward statistic together with the first reward statistic to enable a user to compare the first reward statistic and the second first reward statistic.

Abstract: Anomalies detection in error signals of a cloud based service is provided. An application such as an analysis application identifies a machine learning algorithm that matches error signals of components of a cloud based service. A periodic pattern from the error signals is removed with the machine learning algorithm to filter the periodic pattern from an error count in the error signals. The error signals are processed with the machine learning algorithm to detect one or more anomalies with the components. The machine learning algorithm is updated while processing new data to detect new patterns.

Abstract: A control system is described which receives a live data steam of time stamped sensor data observed from a system. The control system accesses a store of time-stamped sensor data from the live data stream. A plurality of pipeline configurations is generated for analyzing the live data stream. Each pipeline configuration comprises a plurality of components for analyzing data, an order of the components, and values of one or more parameters of each component. The pipeline configurations are evaluated by applying the pipeline configurations to data from the store. A ground truth selector is configured to receive user input comprising ground truth data being labeled data items from the store of time-stamped sensor data. The pipeline configurations are re-evaluated using the ground truth data to select one of the pipeline configurations. Control is achieved using output of the selected one of the pipeline configurations executing on the live data stream.

Abstract: Anomalies detection in error signals of a cloud based service is provided. An application such as an analysis application identifies a machine learning algorithm that matches error signals of components of a cloud based service. A periodic pattern from the error signals is removed with the machine learning algorithm to filter the periodic pattern from an error count in the error signals. The error signals are processed with the machine learning algorithm to detect one or more anomalies with the components. The machine learning algorithm is updated while processing new data to detect new patterns.