A-Case-Studies-Hsu-and-Mulay

Site Plan - Prineville• 124 Acres Greenfield SiteCurrent Phase▪ Phase 3Phase 2

Electrical - Prineville

Electrical Overview• Removed• Centralized UPS by implementing 48VDC UPSSystem• 480V to 208V transformation by using 480VACdistribution to IT equipment

DC UPS Backup Scheme48VDC BatteryCabinet277VAC48VDCBatteryChargerServer PS277VACN+1battery4x12VBatt4x12VBatt4x12VBatt48VDCstandbyBack upconverter48VDC-12VDCAC to DC277VAC –12VDC12VDCL E G E N DNormal PowerBackup Power4x12VBatt4x12VBattMOBO

Battery Cabinet• Custom DC UPS• 56kW or 85kW, 480VAC,3-phase input• Six 48VDC output• 45 sec back-up time at fullload EOL• Rectifier to charge batteries• 20 sealed VRLA batteries• Battery monitoring system –impedance, voltage,temperature• Two 50A 48VDC aux outputs

Reactor Power Panel• Custom Fabricated RPP• Delivers 165kW, 480/277V, 3-phase to CABlevel• Contains Cam-Lock connector for wrap around• Line Reactor• Reduces short circuit current < 10kA• Corrects leading power factor towards unity (3%improvement)• Reduces THD for improved electrical systemperformance (iTHD 2% improvement)• Power consumption = 360 Watt

Mechanical - Prineville

Mechanical Overview• Removed• Centralized chiller plant by implementing 100% outside air economizationwith evaporative cooling system• HVAC ductwork by using cold aisle space as supply air plenum• System Basis of Design:• 50 year worst case 105°F Dry Bulb / 70.3°F Wet Bulb based on BIN data• Cold aisle temperature range: 65°F to 80°F• 65% RH maximum, 41.9°F min dew point temperature• Built-up penthouse air handling system• In the winter, the server waste heat is used for office space heating

Data Center Design Innovation – Mechanical SystemTraditional Data Center CoolingRETURNDUCTWORKCT CHILLER AHUSUPPLYDUCTDATA CENTERFacebook Prineville Data Center CoolingDUCTLESSRELIEF AIRRELIEF AIRRETURNAIRDUCTLESS RETURN AIRReturnAir Plenum100%OUTSIDEAIR INTAKEFILTERWALLMISTINGSYSTEMFANWALLDUCTLESSSUPPLYDATA CENTER

Data Center Design Innovation – Mechanical SystemTraditional Data Center CoolingRETURNDUCTWORKCT CHILLER AHUSUPPLYDUCTDATA CENTERFacebook Forest City Data CenterCoolingRETURNAIRDUCTLESS RETURN AIRDUCTLESSRELIEF AIRRELIEF AIRReturnAir Plenum100%OUTSIDEAIR INTAKEFILTERWALLDXSYSTEMMISTINGSYSTEMFANWALLDUCTLESSSUPPLYDATA CENTER

Mechanical PenthouseVertical drainable louversOutside air intake corridor

Mechanical PenthouseAir mixing section - Return air / out side air /filter corridorEvaporative cooling /humidification corridor

Forest City OnlyDX Cooling Coil• Added to the Forest Citymechanical design• 50 year extreme weatherconditions• Sized to dehumidifying OAduring extreme weatherconditions• Per BIM weather data, DX is notrequired during normaloperating conditions

Mechanical PenthouseReverse Osmosis (RO) waterpurification system for directevaporative cooling /humidification systemEvaporative cooling/humidification pump skid chiller-less design

Mechanical PenthouseMisteliminatormediaDirect evaporative nozzle arraySpraying nozzle

Mechanical PenthouseFan wall array inlet corridorFan array outlet / supply air corridor

Data SuiteCold aisle pressurization – ductless

Data SuiteHot aisle containment – ductless return

Mechanical PenthouseRelief air fansRelief air corridor

Prineville Datacenter Achievements▪ PUE of 1.07 at full load – verified during commissioning▪ WUE of 0.31 liters/kWh – calculated value, real data being collected▪ Reduced System Components▪ Higher Reliability▪ Lowered datacenter construction cost▪ Lowered operating cost

PUE Update for PrinevilleAvailable PUE Data: for period 14 Apr, 2011- 30 Sept, 2011Bin Frequency1.01 421.02 131.03 971.04 1091.05 24381.06 156941.07 568381.08 768151.09 282031.1 146161.11 92601.12 83841.13 60921.14 23351.15 3481.16 771.17 811.18 1251.19 221.2 4More 6221599Cumulative PUE: 1.083

Lessons Learned - Electrical

In-rush Current with PSU• Higher in-rush current experienced under repeatedpower outage• Circuit Breakers see the high in-rush and trip onGround Fault• Cause due to PS input capacitor• Adjusted setting accordingly to resolve issue

Lessons Learned - Mechanical

Lessons Learned• Controls issues with a 100% OA economization withdirect evaporative cooling• We found an error in the controls sequence• Control loop tuning (14) independently controlled AHU lineUps

Lessons Learned• A perforated target plate is currently being implemented forimproved air distribution within the datacenter

Next Steps - Electrical

Power Path: SwedenNationalGridMultiple PowerGeneration SourcesRegionalGridHydro PowerPlantsRegionalGridPrimaryRedundant▪ 145kV Lines to Primary and Redundant Substations from separatesource and path▪ Lines connected to robust National Grid with hydro generation▪ National Grid has not go down since the early 1970’s

Initial Sweden DesignPrimaryUtility▪ Typical single utility feedPrimary Bus▪ Standard design with generatorsTXSubstationTransformerGENPTXMSB-RGENPTXGENPTXMSB

Final Sweden Design145kV Primary Bus22kV BusPrimaryUtility145kV Redundant BusRedundantUtilityTXTX TX TXSubstation Transformers22kV Bus▪ Fully redundant system▪ 145kV Utility lines fromseparate sources andpaths▪ Reduces generatorrequirements by 70%▪ Lower operating costsGENPTXGENPTX▪70% less fuel storageMSB-RMSB-S▪Less annual maintenancecostPTXPTXMSBMSB

Next Steps - Mechanical

Datacenter Basis of Design Comparison80ºF inlet85ºF inlet85ºF inlet85ºF inlet65% humidity80% humidity90% humidity80% humidity20ºF ΔT22ºF ΔT22ºF ΔT22ºF ΔTPRN1A1BPRN1C1DFRC1A1B SWD1A1B

Modifications to MechanicalDesign Parameters▪ The initial mechanical design of PRN1C1D and FRC1A1B▪ 35ºF ΔT for the datahall, basing the server deployment on 100%OCP servers▪ Upon final datahall floor layout design▪ 22ºF ΔT for the datahall, basing the server deployment on a mixof OCP and OEM servers▪ Increased (8) AHU line ups to (13) AHU line ups in order to meeta server 22ºF ΔT requirement

Psychrometric Chart – PRN1A1B• 65F – 80F cold aisletemperature• 65% relativehumidity max• 41.9F min dew pointtemp• Summer: 110ºFDB,70.3ºFWB• Winter: -30.8ºFDB,

Psychrometric Chart – PRN1C1D• 65F – 85F cold aisletemperature• 80% relativehumidity max• 41.9F min dew pointtemp• We reduced the evappump skidredundancy from 2Nto N• Summer: 110ºFDB,

Psychrometric Chart – FRC1A1B• 65F – 85F cold aisletemperature• 90% relative humiditymax• 41.9F min dew pointtemp• DX added for extremeweather criteria into condition the airwithin acceptableserver conditions• Also reduced the evappump skid redundancyfrom 2N to N

Psychrometric Chart – FRC1A1B• BIN weather datacharted on thisgraph shows thatDX co0ling isnot necessaryfor a typicalyear

Psychrometric Chart – Sweden• 65F – 85F coldaisle temperature• 80% relativehumidity max• 41.9F min dewpoint temp• DX not required• Summer: 95.1ºFDB,71.2ºFWB• Winter: -39.1ºFDB, 50%RH

http://opencompute.org/

Thank You!