SHIP: A Scalable Hierarchical Power Control ... - IEEE Xplore

More documents

Recommendations

Info

WANG ET AL.: SHIP: A SCALABLE HIERARCHICAL POWER CONTROL ARCHITECTURE FOR LARGE-SCALE DATA CENTERS 175raised back to 1,000 W. The PDU-level controller thenincreases the power budgets of the racks to achieve the newset point. This experiment demonstrates that SHIP canprovide robust power control despite power budgetreduction at runtime.7 RELATED WORKPower is one of the most important design constraints forenterprise servers. Much of the prior work has attempted toreduce power consumption by improving the energyefficiencyof individual server components [16]. There hasbeen some work on system-level power and thermalmanagement [23], [24], [25]. For example, Nathuji andSchwan have proposed heuristic solutions for powerbudgeting in virtualized environments [26]. In contrast toexisting work, which relies on heuristic-based controlschemes, we adopt a rigorous design methodology thatfeatures a control-theoretic framework for systematicallydeveloping control strategies with analytical assurance ofcontrol accuracy and system stability.Several research projects [27], [8], [28] have successfullyapplied control theory to explicitly control power ortemperature of a single enterprise server. Some recent workhas proposed heuristic-based control strategies at the racklevel [4], [29]. Control-theoretic solutions have also beendesigned to control rack-level power consumption foroptimized system performance [5]. However, those solutionscannot be directly applied to control a PDU or an entire datacenter because the overhead of their centralized controlschemes becomes prohibitive when the system size increasesto a certain extent. In contrast, our hierarchical controlarchitecture is highly scalable for large-scale data centers.A recent study [6] indicates the possibility of having ageneral group power manager that can be extended tocontrol a data center. Our work is different in three aspects:1) our control scheme is designed specifically based on datacenters’ three-level power supply hierarchy, 2) our solutionfeatures a MIMO control strategy with rigorous stabilityanalysis, and 3) our work is evaluated on a physical testbed,while only simulation results are presented in [6]. Inaddition, we also present simulation results in large-scaledata centers with a trace file of 5,415 servers while only180 servers are simulated in [6]. At the PDU level, Govindanet al. [30] propose statistical profiling-based techniques toprovision servers under a power constraint. At the datacenter level, Fan et al. [2] investigate the aggregate powerusage characteristics of a warehouse-sized data center. Incontrast, we dynamically control the power consumption ofan entire data center and optimize system performance byshifting power among racks and PDUs. Pelley et al. proposea method of distributing PDU power feeds to reduce thenumber of PDUs required to tolerate PDU failures [31].Their technique requires a power capping component, suchas SHIP, to prevent long-term overload conditions.8 CONCLUSIONSPower control for an entire data center has becomeincreasingly important. However, existing server powercontrol solutions are not scalable for large-scale data centersbecause they are designed for a single server or a rackenclosure. In this paper, we presented SHIP, a highly scalablehierarchical control architecture that controls the total powerconsumption of a large-scale data center to stay within aconstraint imposed by its power distribution capacity. Thecontrol architecture is designed based on rigorous controltheory for analytical assurance of control accuracy andsystem stability. Empirical results on a physical testbed showthat our control solution can provide precise power control,as well as power differentiations for optimized systemperformance and desired server priorities. In addition, ourextensive simulation results based on a real trace filedemonstrate the efficacy of our control solution in largescaledata centers composed of thousands of servers.ACKNOWLEDGMENTSThis is a significantly extended version of a conferencepaper [1]. The authors at the University of Tennessee aresupported by National Science Foundation (NSF) under aCSR grant CNS-0720663 and an NSF CAREER Award CNS-0845390, and by Microsoft Research under a power-awarecomputing award in 2008.REFERENCES[1] X. Wang, M. Chen, C. Lefurgy, and T.W. Keller, “SHIP: ScalableHierarchical Power Control for Large-Scale Data Centers,” Proc.18th Int’l Conf. Parallel Architectures and Compilation Techniques(PACT ’09), 2009.[2] X. Fan, W.-D. Weber, and L.A. Barroso, “Power Provisioning for aWarehouse-Sized Computer,” Proc. 34th Ann. Int’l Symp. ComputerArchitecture (ISCA ’07), 2007.[3] S. Gorman, “Power Supply Still a Vexation for the NSA,” TheBaltimore Sun, June 2007.[4] P. Ranganathan, P. Leech, D. Irwin, and J.S. Chase, “Ensemble-Level Power Management for Dense Blade Servers,” Proc. 33rdAnn. Int’l Symp. Computer Architecture (ISCA ’06), 2006.[5] X. Wang, M. Chen, and X. Fu, “MIMO Power Control for High-Density Servers in an Enclosure,” IEEE Trans. Parallel andDistributed Systems, vol. 21, no. 10, pp. 1412-1426, Oct. 2010.[6] R. Raghavendra, P. Ranganathan, V. Talwar, Z. Wang, and X. Zhu,“No Power Struggles: Coordinated Multi-Level Power Managementfor the Data Center,” Proc. 13th Int’l Conf. Architectural Supportfor Programming Languages and Operating Systems (ASPLOS ’08),2008.[7] X. Wang, D. Jia, C. Lu, and X. Koutsoukos, “DEUCON:Decentralized End-to-End Utilization Control for DistributedReal-Time Systems,” IEEE Trans. Parallel and Distributed Systems,vol. 18, no. 7, pp. 996-1009, July 2007.[8] C. Lefurgy, X. Wang, and M. Ware, “Power Capping: A Prelude toPower Shifting,” Cluster Computing, vol. 11, no. 2, pp. 183-195,2008.[9] T. Horvath, T. Abdelzaher, K. Skadron, and X. Liu, “DynamicVoltage Scaling in Multi-Tier Web Servers with End-to-End DelayControl,” IEEE Trans. Computers, vol. 56, no. 4, pp. 444-458, Apr.2007.[10] Y. Chen et al., “Managing Server Energy and Operational Costs inHosting Centers,” Proc. ACM SIGMETRICS, 2005.[11] Y. Wang, X. Wang, M. Chen, and X. Zhu, “Power-EfficientResponse Time Guarantees for Virtualized Enterprise Servers,”Proc. Real-Time Systems Symp. (RTSS, 08), 2008.[12] P. Bohrer, E.N. Elnozahy, T. Keller, M. Kistler, C. Lefurgy, C.McDowell, and R. Rajamony, “The Case for Power Managementin Web Servers,” Power Aware Computing, Kluwer AcademicPublishers, 2002.[13] J.M. Maciejowski, Predictive Control with Constraints. Prentice Hall,2002.[14] X. Fu et al., “Dynamic Thermal and Timeliness Guarantees forDistributed Real-Time Embedded Systems,” Proc. 15th IEEE Int’lConf. Embedded and Real-Time Systems Symp. (RTCSA ’09), 2009.
176 IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, VOL. 23, NO. 1, JANUARY 2012[15] Nat’l Fire Prevention Assoc. “NFPA 70: National Electrical Code,”2008.[16] C. Lefurgy, K. Rajamani, F. Rawson, W. Felter, M. Kistler, andT.W. Keller, “Energy Management for Commercial Servers,” IEEEComputer, vol. 36, no. 12, pp. 39-48, Dec. 2003.[17] Rockwell Automation, “Bulletin 1489 Circuit Breakers SelectionGuide, Publication 1489-SG001B-EN-P,” Jan. 2007.[18] SPEC, “Power and Temperature Measurement Setup GuideSPECpower v1.1,” 2010.[19] M. Ware et al., “Architecting for Power Management: ThePOWER7 Approach,” Proc. IEEE 16th Int’l Symp. High PerformanceComputer Architecture (HPCA ’10), 2010.[20] Intel Corporation, “Voltage Regulator Module (VRM) andEnterprise Voltage Regulator-Down (EVRD) 11.1 Design Guidelines,”Sept. 2009.[21] AMD, “White Paper Publication 26094: BIOS and Kernel Developer’sGuide for AMD Athlon 64 and AMD Opteron Processors,Revision 3.30,” Feb. 2006.[22] W. Kim, M. Gupta, G.-Y. Wei, and D. Brooks, “System LevelAnalysis of Fast, Per-Core DVFS Using On-Chip SwitchingRegulators,” Proc. IEEE 14th Int’l Symp. High Performance ComputerArchitecture (HPCA ’08), 2008.[23] H. Zeng et al., “ECOSystem: Managing Energy as a First ClassOperating System Resource,” Proc. 10th Int’l Conf. ArchitecturalSupport for Programming Languages and Operating Systems(ASPLOS ’02), 2002.[24] Y.-H. Lu, L. Benini, and G.D. Micheli, “Operating-System DirectedPower Reduction,” Proc. Int"l Symp. Low Power Electronics andDesign (ISLPED ’00), 2000.[25] D. Brooks and M. Martonosi, “Dynamic Thermal Management forHigh-Performance Microprocessors,” Proc. Seventh Int’l Symp.High Performance Computer Architecture (HPCA ’01), 2001.[26] R. Nathuji and K. Schwan, “Vpm Tokens: Virtual Machine-AwarePower Budgeting in Data centers,” Proc. 17th Int’l Symp. HighPerformance Distributed Computing (HPDC ’08), 2008.[27] R.J. Minerick, V.W. Freeh, and P.M. Kogge, “Dynamic PowerManagement Using Feedback,” Proc. Workshop on Compilers andOperating Systems for Low Power (COLP ’02), Sept. 2002.[28] K. Skadron, T. Abdelzaher, and M.R. Stan, “Control-TheoreticTechniques and Thermal-RC Modeling for Accurate and LocalizedDynamic Thermal Management,” Proc. Eighth Int’l Symp.High Performance Computer Architecture (HPCA ’02), 2002.[29] M.E. Femal and V.W. Freeh, “Boosting Data Center Performancethrough Non-Uniform Power Allocation,” Proc. Second Int’l Conf.Automatic Computing (ICAC ’05), 2005.[30] S. Govindan, J. Choi, B. Urgaonkar, A. Sivasubramaniam, and A.Baldini, “Statistical Profiling-Based Techniques for EffectiveProvisioning of Power Infrastructure in Consolidated DataCenters,” Proc. Fourth ACM European Conf. Computer Systems(EuroSys ’09), 2009.[31] S. Pelley et al., “Power Routing: Dynamic Power Provisioning inthe Data Center,” Proc. 15th Int’l Conf. Architectural Support forProgramming Languages and Operating Systems (ASPLOS ’10), 2010.Xiaorui Wang received the PhD degree fromWashington University in St. Louis in 2006. Heis an associate professor in the Department ofElectrical and Computer Engineering at TheOhio State University. He is the recipient of theUS Office of Naval Research (ONR) YoungInvestigator (YIP) Award in 2011, the USNational Science Foundation (NSF) CAREERAward in 2009, the Power-Aware ComputingAward from Microsoft Research in 2008, andthe IBM Real-Time Innovation Award in 2007. He also received theBest Paper Award from the 29th IEEE Real-Time Systems Symposium(RTSS) in 2008. He is an author or coauthor of more than 60 refereedpublications. From 2006 to 2011, he was an assistant professor at theUniversity of Tennessee, Knoxville, where he received the EECS EarlyCareer Development Award, the Chancellor’s Award for ProfessionalPromise, and the College of Engineering Research Fellow Award in2008, 2009, and 2010, respectively. In 2005, he worked at the IBMAustin Research Laboratory, designing power control algorithms forhigh-density computer servers. From 1998 to 2001, he was a seniorsoftware engineer and then a project manager at Huawei TechnologiesCo. Ltd., China, developing distributed management systems foroptical networks. His research interests include power-aware computersystems and architecture, real-time embedded systems, and cyberphysicalsystems. He is a member of the IEEE and the IEEEComputer Society.Ming Chen received the BEng and MEngdegrees in electrical engineering from NorthwesternPolytechnic University, Xian, China, in2002 and 2005, respectively. He received thePhD degree in computer engineering from theDepartment of Electrical Engineering and ComputerScience at the University of Tennessee,Knoxville in 2010, under the supervision of Dr.Xiaorui Wang. He is a performance engineer atthe NetApp, Inc. in Sunnyvale, CA. From 2005 to2006, he worked as a software engineer at ZTE Corp., China,developing embedded multimedia software. His research interestsinclude power-aware computing.Charles Lefurgy received the PhD degree incomputer science and engineering from theUniversity of Michigan. He is a research staffmember at the IBM Austin Research Laboratory.His research interests include power managementfor servers and data centers and contributedto the development of the IBM activeenergy management product. He is a member ofthe ACM, IEEE, and IEEE Computer Society.Tom W. Keller received the PhD degree incomputer sciences from the University of Texas.He is a IBM’s ranking technical executive in thearea of power-aware computing and he is adistinguished engineer in IBM Research, Austin,where research in energy-efficient data centers,microprocessors, and computing systems isconducted. He has previously worked at theUT Computation Center, MCC and the LosAlamos National Laboratory.. For more information on this or any other computing topic,please visit our Digital Library at www.computer.org/publications/dlib.
Page 1 and 2: 168 IEEE TRANSACTIONS ON PARALLEL A
Page 3: 170 IEEE TRANSACTIONS ON PARALLEL A
Page 6 and 7: WANG ET AL.: SHIP: A SCALABLE HIERA

SHIP: A Scalable Hierarchical Power Control ... - IEEE Xplore

Create successful ePaper yourself

Delete template?

Save as template?