Abstract: A method includes obtaining time series data for a usage or performance metric for computing resources in a service provider network comprising a plurality of observations recorded in a plurality of respective time steps. A prediction error is determined for a previous prediction of an observation in the time series data. The prediction error is used to update a standard deviation of a set of predication errors for the usage or performance metric. The standard deviation and the prediction error are then used to update a confidence coefficient. A prediction limit for the usage or performance metric is then determined based on an expected value, the confidence coefficient, and the standard deviation. One or more events may be generated based on the prediction limit, which may be used to trigger a reconfiguration or auto-scaling of the computing resources.

Abstract: A leader host obtains individual distributions of data sets ingested by individual hosts of a fleet of hosts over a domain. The leader host compiles the individual distributions over the domain to generate a compiled distribution. The leader host then partitions the domain based at least in part on the generated compiled distribution. These partitions of the partitioned domain are distributed to individual hosts of the fleet of hosts, which causes the individual hosts to process a portion of the distributed date set according to their respective partitions.

Abstract: Fault tolerant data export is provided through the checkpointing of various export tasks. Data entries, such as event log records, to be exported are assigned to a set of discrete export tasks. A snapshot of the data can be taken to indicate the state of the data to be exported. The tasks can be performed using appropriate resources, and for each task that completes successfully an entry can be made in a checkpoint data file or a virtual cursor can be updated for the snapshot indicating completion of that task. In the event of a failure of one or more tasks, a workflow manager or other such component can determine which tasks failed using the checkpointing data and retry only those specific tasks, without having to retry the entire export.

Abstract: Technology is described for testing applications. Multiple testing threads that are to be used for testing an application may be identified. The multiple testing threads may be dispatched for execution using a unit testing framework. The multiple testing threads may be executed in parallel to test the application using testing context information that is related to each testing thread. Performance metrics may be collected for the application that is tested. The performance metrics may be transmitted from the multiple testing threads after parallel execution of the multiple testing threads.

Abstract: A prediction model testing system includes a test environment that is used to test a prediction model under test (PMUT). A metrics collector in a production environment collects and stores production metrics data generated from computing resources in a production environment. A production predictor in the production environment generates production predictions for the metrics, using a production prediction model. A test manager may make the production metrics data available in a test environment. Test predictions are generated in the test environment from the metrics data using the PMUT. The test manager may then calculate respective prediction errors of the production prediction model and the PMUT, and generate a report indicating the differences between the two sets of prediction errors. The report may be used by the test management system to determine whether a test of the PMUT was successful.

Abstract: Techniques for acquiring performance analysis metrics for computer-implemented functions are described herein. A computer-implemented function is instantiated on a computer system. A monitoring agent running on the computer system monitors outbound calls from the computer-implemented function and, when an outbound function call is intercepted, the agent calculates a performance metric for the outbound function. An aggregated set of performance metrics is produced, and that aggregated set of performance metrics is presented for further analysis.

Abstract: A technology is provided for enabling a partitioned search to be performed on log events from multiple log streams that are stored by multiple hosts. A search query may be submitted to identify the log streams whose log events are to be searched and to indicate a time interval in which log events are to have occurred as indicated by the log events' time stamps. The multiple hosts may search stored log events in parallel and return a set of log-event search results satisfying the search query. A pagination token can be included with the set of log event search results. The pagination token may be used to resume the search if the multiple hosts were not able to completely finish searching the stored log events before the set of log-event search results had to be returned to prevent a timeout of a search client.

Abstract: A technology is described for processing a log message. An example method may include identifying a log template in a log statement associated with a computer program. Extracting the log template from the log statement and creating a log template file that includes a template identifier referencing the log template. The computer program and the log template file may be deployed to a server that hosts the computer program, and a log message output by the computer program executing on the server may include the template identifier and a log data value that is stored to a log data store.

Abstract: Systems and methods are provided for organizing data channels and processing hosts included in a system into clusters. A cluster management service may receive data from a steam of data and may route the data to a cluster associated with the data stream. A data channel routing service included in the cluster may route the data to the set of processing hosts included in the cluster through multiple data channels included in the cluster. In some instances, the data channel routing service may use any of the data channels to send data to the set of processing hosts. Because incoming data may be distributed among multiple data channels, the cluster may experience less congestion. Further, the system may also process the stream of data using the same processing hosts by routing the stream of data to the same cluster, thereby avoiding split processing of the data stream.