Abstract: To enhance the scaling of data processing systems in a computing environment, a number of data objects indicated in an allocation queue and a first attribute of the allocation queue are determined, where the allocation queue is accessible to a plurality of data processing systems. A number of data objects indicated in the allocation queue at a subsequent time is predicted based on the determined number of data objects and the first attribute. It is determined whether the active subset of the plurality of data processing systems satisfies a criterion for quantity adjustment based, at least in part, on the predicted number of data objects indicated in the allocation queue and a processing time goal. Based on determining that the active subset of data processing systems satisfies the criterion for quantity adjustment, a quantity of the active subset of data processing systems is adjusted.

Abstract: A cloud monitoring system is disclosed herein that uses a filtering paradigm after metric data aggregation and before storing in a repository that allows querying the metric data to significantly reduce the raw data stored into the repository. A persistence filter identifies extremal cloud metrics that are persistent across time windows, increasing the confidence that extremal metrics correspond to abnormal behavior. A cloud metric filter comprising a sequence of logical and statistical filters that are interchangeable in order and use allows for dynamic filtering of cloud data. The cloud metric filter has a feedback control loop to update the order and parameters of individual filters based on properties of filtered metric values. Intelligent filtering of cloud metric data yields a focused set of statistically significant metric data for monitoring and eliminates noisy metric data for normal behavior.

Abstract: A cloud monitoring system is disclosed herein that uses a filtering paradigm after metric data aggregation and before storing in a repository that allows querying the metric data to significantly reduce the raw data stored into the repository. A persistence filter identifies extremal cloud metrics that are persistent across time windows, increasing the confidence that extremal metrics correspond to abnormal behavior. A cloud metric filter comprising a sequence of logical and statistical filters that are interchangeable in order and use allows for dynamic filtering of cloud data. The cloud metric filter has a feedback control loop to update the order and parameters of individual filters based on properties of filtered metric values. Intelligent filtering of cloud metric data yields a focused set of statistically significant metric data for monitoring and eliminates noisy metric data for normal behavior.

Abstract: To enhance the scaling of data processing systems in a computing environment, a number of data objects indicated in an allocation queue and a first attribute of the allocation queue are determined, where the allocation queue is accessible to a plurality of data processing systems. A number of data objects indicated in the allocation queue at a subsequent time is predicted based on the determined number of data objects and the first attribute. It is determined whether the active subset of the plurality of data processing systems satisfies a criterion for quantity adjustment based, at least in part, on the predicted number of data objects indicated in the allocation queue and a processing time goal. Based on determining that the active subset of data processing systems satisfies the criterion for quantity adjustment, a quantity of the active subset of data processing systems is adjusted.

Abstract: To enhance the scaling of data processing systems in a computing environment, a number of data objects indicated in an allocation queue and a first attribute of the allocation queue are determined, where the allocation queue is accessible to a plurality of data processing systems. A number of data objects indicated in the allocation queue at a subsequent time is predicted based on the determined number of data objects and the first attribute. It is determined whether the active subset of the plurality of data processing systems satisfies a criterion for quantity adjustment based, at least in part, on the predicted number of data objects indicated in the allocation queue and a processing time goal. Based on determining that the active subset of data processing systems satisfies the criterion for quantity adjustment, a quantity of the active subset of data processing systems is adjusted.

Abstract: To enhance the scaling of data processing systems in a computing environment, a number of data objects indicated in an allocation queue and a first attribute of the allocation queue are determined, where the allocation queue is accessible to a plurality of data processing systems. A number of data objects indicated in the allocation queue at a subsequent time is predicted based on the determined number of data objects and the first attribute. It is determined whether the active subset of the plurality of data processing systems satisfies a criterion for quantity adjustment based, at least in part, on the predicted number of data objects indicated in the allocation queue and a processing time goal. Based on determining that the active subset of data processing systems satisfies the criterion for quantity adjustment, a quantity of the active subset of data processing systems is adjusted.

Abstract: Described herein are systems, methods, and software to enhance the scaling of data processing systems in a computing environment. In one implementation, a method of operating a data processing management system includes monitoring a queue length in an allocation queue for data processing system, and generating a prediction of the allocation queue based on the monitored queue length. Once the prediction is generated, the data processing management system may modify an operational state of at least one data processing system based on the prediction of the queue length and a processing time requirement for data objects in the allocation queue.

Abstract: Techniques for predictively scaling a distributed application are described. Embodiments could monitor performance of an application within a cloud computing environment over a first window of time to collect historical performance data. Here, the application comprises a plurality of application instances. A workload of the application could be monitored over a second window of time to collect historical workload data. Embodiments could analyze both the historical performance data and the historical workload data to determine one or more scaling patterns for the application. Upon determining a present state of the application matches one of the one or more scaling patterns, a plan for predictively scaling the application could be determined. Embodiments could then predictively scale the plurality of application instances, based on the determined plan.

Abstract: Described herein are systems, methods, and software to enhance the scaling of data processing systems in a computing environment. In one implementation, a method of operating a data processing management system includes monitoring a queue length in an allocation queue for data processing system, and generating a prediction of the allocation queue based on the monitored queue length. Once the prediction is generated, the data processing management system may modify an operational state of at least one data processing system based on the prediction of the queue length and a processing time requirement for data objects in the allocation queue.

Abstract: Techniques for evaluating a second version of software. Embodiments selectively route incoming requests to software instances within a plurality of baseline instances and a plurality of canary instances, where the baseline instances run a first software version and the canary instances run the second software version. The software instances are monitored to collect performance data for a plurality of performance metrics. Embodiments calculate aggregate baseline performance metrics, where each of the aggregate baseline performance metrics is calculated based on the collected performance data for the plurality of baseline instances. For each of the performance metrics and canary instances, embodiments calculate a relative performance value that measures the collected performance data for the respective canary instance and for the respective performance metric, relative to the corresponding aggregate baseline performance metric.

Abstract: Techniques are described for identifying a root cause of a pattern of performance data in a system including a plurality of services. Embodiments provide dependency information for each of the plurality of services, where at least one of the plurality of services is dependent upon a first one of the plurality of services. Each of the plurality of services is monitored to collect performance data for the respective service. Embodiments further analyze the performance data to identify a cluster of services that each follow a pattern of performance data. The first one of the services in the cluster of services is determined to be a root cause of the pattern of performance data, based on the determined dependency information for each of the plurality of services.

Abstract: Techniques for predictively scaling a distributed application are described. Embodiments could monitor performance of an application within a cloud computing environment over a first window of time to collect historical performance data. Here, the application comprises a plurality of application instances. A workload of the application could be monitored over a second window of time to collect historical workload data. Embodiments could analyze both the historical performance data and the historical workload data to determine one or more scaling patterns for the application. Upon determining a present state of the application matches one of the one or more scaling patterns, a plan for predictively scaling the application could be determined. Embodiments could then predictively scale the plurality of application instances, based on the determined plan.

Abstract: Techniques are described for identifying a root cause of a pattern of performance data in a system including a plurality of services. Embodiments provide dependency information for each of the plurality of services, where at least one of the plurality of services is dependent upon a first one of the plurality of services. Each of the plurality of services is monitored to collect performance data for the respective service. Embodiments further analyze the performance data to identify a cluster of services that each follow a pattern of performance data. The first one of the services in the cluster of services is determined to be a root cause of the pattern of performance data, based on the determined dependency information for each of the plurality of services.

Abstract: Techniques for evaluating a second version of software. Embodiments selectively route incoming requests to software instances within a plurality of baseline instances and a plurality of canary instances, where the baseline instances run a first software version and the canary instances run the second software version. The software instances are monitored to collect performance data for a plurality of performance metrics. Embodiments calculate aggregate baseline performance metrics, where each of the aggregate baseline performance metrics is calculated based on the collected performance data for the plurality of baseline instances. For each of the performance metrics and canary instances, embodiments calculate a relative performance value that measures the collected performance data for the respective canary instance and for the respective performance metric, relative to the corresponding aggregate baseline performance metric.