Abstract: Techniques for detecting regressions with respect to the accuracy of an anomaly detection compute service in detecting anomalies in users' time series data. The techniques include providing an instrumented time series instrumented with a set of one or more anomalies to the anomaly detection service. The anomaly detection service detects a set of one or more anomalies in the instrumented time series. The precision and recall of the detected anomalies with respect to the instrumented anomalies is computed. From the computed precision and recall, an anomaly detection accuracy is computed as an F-score or F-measure. It is then determined whether a regression in anomaly detection accuracy has occurred by comparing the computed accuracy score to a threshold. If a regression has occurred, an alert can be generated or a recent change to the anomaly detection service can be rolled back.

Abstract: A service to provide anomaly detection receives a request to back-test the service. The request includes information for accessing a dataset of historical data. The service executes workflows to ingest the data, train a plurality of machine learning models to perform anomaly detection, and detect anomalies in the dataset. A representation of the detect anomalies is generated and presented to a user. The service receives an indication to activate the service to provide ongoing anomaly detection services.