SHIP: A Scalable Hierarchical Power Control ... - IEEE Xplore

WANG ET AL.: SHIP: A SCALABLE HIERARCHICAL POWER CONTROL ARCHITECTURE FOR LARGE-SCALE DATA CENTERS 169years, feedback control theory has been identified as aneffective tool for power control due to its theoreticallyguaranteed control accuracy and system stability. Controltheory also provides well-established controller designapproaches, e.g., standard ways to choose the right controlparameters, such that exhaustive iterations of tuning andtesting can be avoided. Furthermore, control theory can beapplied to quantitatively analyze control performance (e.g.,stability, settling time) even when the system is sufferingunpredictable workload variations. This rigorous designmethodology is in sharp contrast to heuristic-based adaptivesolutions that heavily rely on extensive manual tuning.For example, recent work [8], [5] has shown that controltheoreticpower management outperforms commonly usedheuristic solutions by having more accurate power controland better application performance.There are several challenges in developing scalablepower control algorithms. First, the global control problem(i.e., power control for an entire data center) needs to bedecomposed into a set of control subproblems for scalability.The decomposition strategy must comply with thedata centers’ power distribution hierarchy. Second, thelocal controller designed for each decomposed subproblemneeds to achieve local stability and control accuracydespite significantly varying workloads. Third, each localcontroller needs to coordinate with other controllers atdifferent levels for global stability and control accuracy.Finally, the system performance of the data center needs tobe optimized based on optimal control theory, subject tovarious system constraints.In this paper, we present SHIP, a highly scalablehierarchical power control architecture for large-scale datacenters composed of thousands of servers. Our controlarchitecture is systematically designed based on advancedoptimal control theory for theoretically guaranteed controlaccuracy and system stability. Specifically, the contributionsof this paper are four-fold:. We decompose the problem of power control for adata center into control subproblems at the threelevels of the common power distribution hierarchy,and then model the power consumption of each level.. We design and analyze Multi-Input-Multi-Output(MIMO) power control algorithms for different levelsbased on Model Predictive Control (MPC) theory tooptimize system performance, while controlling thetotal power to stay within the desired constraints.. We implement our control architecture on a physicaltestbed and provide the implementation details ofeach component in the control loops.. We present empirical results on a physical testbed todemonstrate that our solution can provide precisepower control and desired power differentiation foroptimized system performance and desired serverpriorities. With scalability constraints, our controlsolution outperforms a state-of-the-art centralizedpower controller by having better benchmark performance.We also present simulation results based on areal trace file of 5,415 servers to show the effectivenessof our solution in large-scale data centers.The rest of the paper is organized as follows: Section 2introduces the overall architecture of our hierarchicalpower control solution. Section 3 describes the systemmodeling, controller design and analysis of the PDU-levelpower controller. Section 4 discusses the coordinationamong controllers at different levels. Section 6 providesthe implementation details of our control architecture andour empirical results on a physical testbed. Section 7highlights the distinction of our work by discussing therelated work. Section 8 concludes the paper.2 HIERARCHICAL POWER CONTROL ARCHITECTUREIn this section, we provide a high-level description of theSHIP power control architecture, which features a threelevelpower control solution. First, the rack-level powercontroller adaptively manages the power consumption of arack by manipulating the CPU frequency (e.g., via DynamicVoltage and Frequency Scaling (DVFS)) of the processors ofeach server in the rack. Second, the PDU-level powercontroller manages the total power consumption of a PDUby manipulating the power budget of each rack in the PDU.Similar to the PDU-level controller, the data center-levelcontroller manages the total power consumption of theentire data center by manipulating the power budget of eachPDU. Our control architecture is directly applicable to datacenters, where applications (e.g., scientific computing andbackground data processing) can allow degraded performancewhen power must be controlled to stay below abudget at runtime (e.g., due to thermal emergency). Fordata centers, where applications need to achieve specifiedservice-level agreements (SLAs) (e.g., response time), oursolution can be integrated with application-level performancecontrol solutions (e.g., [9], [10], [11]) for simultaneouscontrol of power and application performance.We assume that the power limit of the upper level (e.g.,the data center) is lower than the sum of the maximumpower limits of all the lower-level units (e.g., PDUs). Thisassumption is based on two key observations of data centeroperation. First, many data centers are rapidly increasingtheir number of hosted servers to support new business inthe short term, while infrastructure upgrades at upperlevels happen over much longer time scales due to costconsiderations. Second, lower-level units commonly havenonuniform workloads and so can rarely reach their powerlimits simultaneously.There are several reasons for us to use processorfrequency (and voltage) scaling as our actuation methodat the rack level. First, processors commonly contribute alarge portion of the total power consumption of a server[12]. As a result, the processor power difference betweenthe highest and lowest power states is large enough tocompensate for the power variation of other components,and can thus provide an effective way for server powercontrol. Second, frequency scaling has a small overheadwhile some other actuation methods, like turning serverson/off, may lead to service interruption and undesired longdelays. Finally, current processors support frequencyscaling by DVFS or clock modulation [8], while there arestill very few real disks or memory devices that aredesigned for servers and allow runtime transition amongdifferent active power modes. Note that other actuationmethods can also be included in our control architecture,which is our future work.