Abstract: The techniques disclosed herein enable systems to perform repeatable and iterative load testing and performance benchmarking for artificial intelligence models deployed in a cloud computing environment. This is achieved by utilizing load profiles and representative workloads generated based on the load profiles to evaluate an artificial intelligence model under various workload contexts. The representative workload is then executed by the artificial intelligence model utilizing available computing infrastructure. Performance metrics are extracted from the execution and analyzed to provide insight into various performance dynamics such as the relationship between latency and data throughput. In addition, load profiles and input datasets are dynamically adjusted to evaluate different scenarios and use cases enabling the system to automatically test the artificial intelligence model across diverse applications.

Abstract: A cloud computing capacity management system can include a fine-grained admission control layer, a policy engine, and an enforcement layer. The fine-grained admission control layer can be configured to ingest capacity signals and create a capacity mitigation policy, based at least in part on the capacity signals, to protect available capacity of a cloud computing system for prioritized users. The capacity mitigation policy can be directed to users of the cloud computing system. The policy engine can be configured to control how the capacity mitigation policy is applied to the cloud computing system. The enforcement layer can be configured to handle incoming resource requests and to enforce resource limits based on the capacity mitigation policy as applied by the policy engine.

Abstract: To improve the reliability of nodes that are utilized by a cloud computing provider, information about the entire lifecycle of nodes can be collected and used to predict when nodes are likely to experience failures based at least in part on early lifecycle errors. In one aspect, a plurality of failure issues experienced by a plurality of production nodes in a cloud computing system during a pre-production phase can be identified. A subset of the plurality of failure issues can be selected based at least in part on correlation with service outages for the plurality of production nodes during a production phase. A comparison can be performed between the subset of the plurality of failure issues and a set of failure issues experienced by a pre-production node during the pre-production phase. A risk score for the pre-production node can be calculated based at least in part on the comparison.

Abstract: An automated root-cause analysis (RCA) system may provide a fully automated platform that provides dependency and execution order modeling for tasks included in a capacity provisioning process, anomaly detection, ticket correlation, root-cause analysis, monitoring and feedback, and data visualization. The automated RCA system may continuously collect and store data for use in determining a root cause of a blockage on a capacity provisioning process. The blockage may be identified in a ticket generated by a cloud-computing system. The automated RCA system may receive the ticket and attempt to determine the root cause of the blockage based on root causes associated with previous tickets generated by the cloud-computing system. The automated RCA system may identify a true root cause, recommend repair items based on the true root cause, identify one or more responsible teams to drive a fix, and provide an estimated time for completion.

Abstract: To improve the reliability of nodes that are utilized by a cloud computing provider, information about the entire lifecycle of nodes can be collected and used to predict when nodes are likely to experience failures based at least in part on early lifecycle errors. In one aspect, a plurality of failure issues experienced by a plurality of production nodes in a cloud computing system during a pre-production phase can be identified. A subset of the plurality of failure issues can be selected based at least in part on correlation with service outages for the plurality of production nodes during a production phase. A comparison can be performed between the subset of the plurality of failure issues and a set of failure issues experienced by a pre-production node during the pre-production phase. A risk score for the pre-production node can be calculated based at least in part on the comparison.

Abstract: A method for minimizing allocation failures in a cloud computing system without overprovisioning may include determining a predicted supply for a virtual machine series in a system unit of the cloud computing system during an upcoming time period. The predicted supply may be based on a shared available current capacity and a shared available future added capacity for the virtual machine series in the system unit. The method may also include predicting an available capacity for the virtual machine series in the system unit during the upcoming time period. The predicted available capacity may be based at least in part on a predicted demand for the virtual machine series in the system unit during the upcoming time period and the predicted supply. The method may also include taking at least one mitigation action in response to determining that the predicted demand exceeds the predicted supply during the upcoming time period.

Abstract: A cloud computing capacity management system can include a fine-grained admission control layer, a policy engine, and an enforcement layer. The fine-grained admission control layer can be configured to ingest capacity signals and create a capacity mitigation policy, based at least in part on the capacity signals, to protect available capacity of a cloud computing system for prioritized users. The capacity mitigation policy can be directed to users of the cloud computing system. The policy engine can be configured to control how the capacity mitigation policy is applied to the cloud computing system. The enforcement layer can be configured to handle incoming resource requests and to enforce resource limits based on the capacity mitigation policy as applied by the policy engine.

Abstract: An automated root-cause analysis (RCA) system may provide a fully automated platform that provides dependency and execution order modeling for tasks included in a capacity provisioning process, anomaly detection, ticket correlation, root-cause analysis, monitoring and feedback, and data visualization. The automated RCA system may continuously collect and store data for use in determining a root cause of a blockage on a capacity provisioning process. The blockage may be identified in a ticket generated by a cloud-computing system. The automated RCA system may receive the ticket and attempt to determine the root cause of the blockage based on root causes associated with previous tickets generated by the cloud-computing system. The automated RCA system may identify a true root cause, recommend repair items based on the true root cause, identify one or more responsible teams to drive a fix, and provide an estimated time for completion.

Abstract: To improve the reliability of nodes that are utilized by a cloud computing provider, information about the entire lifecycle of nodes can be collected and used to predict when nodes are likely to experience failures based at least in part on early lifecycle errors. In one aspect, a plurality of failure issues experienced by a plurality of production nodes in a cloud computing system during a pre-production phase can be identified. A subset of the plurality of failure issues can be selected based at least in part on correlation with service outages for the plurality of production nodes during a production phase. A comparison can be performed between the subset of the plurality of failure issues and a set of failure issues experienced by a pre-production node during the pre-production phase. A risk score for the pre-production node can be calculated based at least in part on the comparison.

Abstract: A method for minimizing allocation failures in a cloud computing system without overprovisioning may include determining a predicted supply for a virtual machine series in a system unit of the cloud computing system during an upcoming time period. The predicted supply may be based on a shared available current capacity and a shared available future added capacity for the virtual machine series in the system unit. The method may also include predicting an available capacity for the virtual machine series in the system unit during the upcoming time period. The predicted available capacity may be based at least in part on a predicted demand for the virtual machine series in the system unit during the upcoming time period and the predicted supply. The method may also include taking at least one mitigation action in response to determining that the predicted demand exceeds the predicted supply during the upcoming time period.

Abstract: Various embodiments of a planning and execution process used to forecast contact volumes that will occur over the course of a season and determine resources necessary to handle the contact volumes are disclosed. The planning and execution process may be used to determine staffing resources necessary to service contact volumes across a plurality of support channels. The contact volume forecast may be continually updated over the course of the season, and different models may be used to generate the contact volume forecast at different times during the season.

Abstract: Various embodiments of a system and method for determining staffing requirements or other resource requirements for a plurality of contact channels are disclosed. A contact volume forecast may be performed at a beginning of a season. The contact volume forecast may predict variability in contact volume over a plurality of time periods in the season for each of the contact channels. A resource plan may be determined based on the contact volume forecast. The resource plan may indicate an amount of resources required for each contact channel for each of the plurality of time periods in the season. For at least one time period, the resource plan may specify that resources are to be shifted into a first contact channel from one or more other contact channels.

Abstract: Various embodiments of a planning and execution process used to forecast contact volumes that will occur over the course of a season and determine resources necessary to handle the contact volumes are disclosed. The planning and execution process may be used to determine staffing resources necessary to service contact volumes across a plurality of support channels. The contact volume forecast may be continually updated over the course of the season, and different models may be used to generate the contact volume forecast at different times during the season.

Abstract: Various embodiments of a system and method for determining staffing requirements or other resource requirements for a plurality of contact channels are disclosed. A contact volume forecast may be performed at a beginning of a season. The contact volume forecast may predict variability in contact volume over a plurality of time periods in the season for each of the contact channels. A resource plan may be determined based on the contact volume forecast. The resource plan may indicate an amount of resources required for each contact channel for each of the plurality of time periods in the season. For at least one time period, the resource plan may specify that resources are to be shifted into a first contact channel from one or more other contact channels.