A Statistical Approach to Capacity Planning for On-Demand Computing Services

In on-demand computing services, the customer pays based on actual usage and the service provider is free to allocate unused capacity to other customers. Capacity planning becomes an important issue in such a shared environment. Capacity Planning encompasses the process of planning for adequate IT resources required to fulfill current and future resource requirements so that the customer's workload requirements are met and the service provider's costs are recovered. This requires the capacity planner to plan for efficient use of existing IT resources which will cost-effectively maximize the use of those resources and to identify any change in the type and quantity of IT resources necessary to fulfill service commitments.

Typical customer workloads have short-lived surge that determines the capacity requirements. For a given risk level such as the likelihood of the workload exceeding the capacity allocation, the required capacity is estimated using historical data and statistical models that capture the workload volatility.

For some applications, workload volatility can be so high that the required capacity may exceed the service provider's recovery limits, making the service unprofitable. Server virtualization is the cutting-edge technology aimed at solving this problem by allowing consolidation of multiple applications to dynamically share the resources of a single server while protecting the applications from each other. We developed a statistical method to quantify the benefit of workload consolidation at different risk levels. The figure below shows the estimated capacity savings (in normalized capacity units --- 100 units per CPU), with 95% confidence limits, for consolidating two workloads from stand-alone servers to a virtualized one.



Based on this approach of capacity savings estimation, we can address the ultimate question of workload consolidation: how to partition a given pool of candidate workloads into groups (portfolios) so that the total capacity requirement is minimized when the workloads in each group are consolidated to share a single virtualized server?

This optimization problem virtually requires an exhaustive search and the computational complexity is very high even for a small number of workloads. For example, over 4,000 possible portfolios need to be evaluated for consolidating 8 workloads. In this research project, we developed a number of fast iterative algorithms to find a (possibly suboptimal) solution to the problem. Moreover, since the capacity requirement is estimated from data, statistical uncertainty must be taken into account when evaluating the potential capacity savings of a portfolio. We solved the problem by using bootstrap-based statistical hypothesis testing techniques.

The figure below shows the result of an algorithm, called the hierarchical clustering algorithm (HCA), that solves the optimization problem by iteratively consolidating two portfolios at a time until the capacity requirement cannot be further reduced. The number shown at each node is the number of processors required by the corresponding portfolio and the number in parenthesis is the average processor utilization. CPC is the normalized capacity per CPU; ALPHA and BETA are design parameters representing the desired availability level and capacity reserve level, respectively. The algorithms require that the capacity of all servers be normalized using standard performance ratings.



Stand-alone PC software was developed to implement this and several other algorithms. The tool also provides automatic data acquisition, workload forecasting, graphical analysis, and many other capabilities. For more information, contact Dr. Ta-Hsin Li (thl@us.ibm.com).