Shekhar Borkar, Intel Corp. Exascale Computing—a fact or ... - IPDPS

OutlineCompute roadmap & technology outlookChallenges & solutions for:– Compute,– Memory,– Interconnect,– Resiliency, and– Software stackSummary2

Relative Proc FreqFrom Giga to Exa, via Tera & Peta1000Peta1.E+08Exa100101Tera30X 250XG1986 1996 2006 20161.E+061.E+041.E+021.E+00GTera36XPeta2.5M X4,000XConcurrencyProcessor Performance1986 1996 2006 20161.E+081.E+061.E+041.E+021.E+00PetaTeraG 80XPower1M X4,000XExa1986 1996 2006 2016System performanceincreases fasterParallelism continues toincreasePower & energy challengecontinues4

The UHPC* Challenge*DARPA, Ubiquitous HPC Program20MW, Exa20W, Tera20KW, Peta20 mW, Mega20 pJ/Operation20 mW, Giga2W, 100 Giga6

RelativeRelativeRelativeRelativeTechnology Scaling Outlook40Transistor Density1.5Frequency3020101.75 – 2X10.5Almost flat045nm 32nm 22nm 14nm 10nm 7nm5nm045nm 32nm 22nm 14nm 10nm 7nm5nm1.21Supply Voltage1Energy0.80.6Almost flat0.1Some scaling0.40.20.01Ideal045nm 32nm 22nm 14nm 10nm 7nm5nm0.00145nm 32nm 22nm 14nm 10nm 7nm5nm7

Energy (pJ)Energy per Compute Operation300250200100 pJ/bitCommunicationpJ/bit CompJ/bit DRAMDP RF OppJ/DP FP15075 pJ/bitDRAM100Operands50FP Op10 pJ/bit25 pJ/bit045nm 32nm 22nm 14nm 10nm 7nmSource: Intel8

NormalizedVoltage Scaling1.2When designed to voltage scale1010.80.60.40.2Energy EfficiencyLeakageFreqTotal Power864200.3 0.5 0.7 0.90Vdd (Normal)9

Maximum Frequency (MHz)4 OrdersTotal Power (mW)Energy Efficiency (GOPS/Watt)Subthreshold Region< 3 OrdersActive Leakage Power (mW)Near Threshold-Voltage (NTV)10 465nm CMOS, 50°C10 245040065nm CMOS, 50°C10 110 310 1350300112502009.6X15010 -110 110 2 10 -210 -11001320mV0.2 0.4 0.6 0.8 1.0 1.2 1.4Supply Voltage (V)H. Kaul et al, 16.6: ISSCC08500320mV0.2 0.4 0.6 0.8 1.0 1.2 1.4Supply Voltage (V)10 -210

SD Leakage PowerSubthreshold Leakage at NTV60%50%40%30%20%10%0%40% VddIncreasingVariations50% Vdd75% Vdd100% Vdd45nm 32nm 22nm 14nm 10nm 7nm5nmNTV operation reduces total power, improves energy efficiencySubthreshold leakage power is substantial portion of the total11

Mitigating Impact of Variation1.Variation control with body biasingBody effect is substantially reduced in advancedtechnologiesEnergy cost of body biasing could become substantialFully-depleted transistors have no body left2. Variation tolerance at the system levelExample: Many-core Systemfffff/2f/4ff/2fffff/4 f f/2 f/4ffffff/2f/4fRunning all cores at fullfrequency exceeds energybudgetRun cores at the native frequencyLaw of large numbers—averaging12

ENERGYEFFICENCYWide Dynamic RangeHIGHSubthresholdNTVNormal operating range~5xDemonstratedLOWZEROVOLTAGEMAXUltra-low Power Energy Efficient High Performance280 mV 0.45 V 1.2 V3 MHz 60 MHz 915 MHz2 mW 10 mW 737 mW1500 Mips/W 5830 Mips/W 1240 Mips/W14

Observations100%80%60%40%Mem LkgMem DynLogic LkgLogic Dyn20%0%Sub-Vt NTV FullVddLeakage power dominatesFine grain leakage power management is required15

Efficiency [%]Integration of Power DeliveryFor efficiency and managementIntegrated Voltage Regulator Testchip5mmConverterchipLoad chipRFLaunchStandard OLGAPackagingTechnology9085L = 1.9nHL = 0.8nH2.4V to 1.5VTOP80BOTTOMInductorsInputCapacitorsOutputCapacitors757060MHz80MHz2.4V to 1.2V100MHz0 5 10 15 20Load Current [A]Power delivery closer to the load for1. Improved efficiency2. Fine grain power managementSchrom et al, “A 100MHz 8-Phase Buck Converter Delivering 12A in 25mm2 Using Air-Core Inductors”, APEC 200716

Fine-grain Power ManagementMode Power Saving Wake upNormal All active - -StandbySleepLogic offMemory onLogic andMemory offDynamic Chip LevelSTANDBY:• Memory retains data• 50% less power/tileFULL SLEEP:• Memories fully off• 80% less power/tile50% Fast80% SlowDynamic, within a core21 sleep regions per tileData MemorySleeping:57% less powerRouterInstructionMemorySleeping:56% less powerRouterSleeping:10% less power(stays on topass traffic)FPEngine 1Sleeping:90% lesspowerFPFPEngine 2Sleeping:90% lesspowerEnergy efficiency increases by 60%,Vangal et al, An 80-Tile Sub-100-W TeraFLOPS Processor in 65-nm CMOS, JSSC, January 200817

Memory & Storage Technologies10000010000Cost/bit(Pico $)1000100101Energy/bit(Pico Joule)Capacity(G Bit)0.10.01SRAM DRAM NAND, PCM Disk(Endurance issue)Source: Intel18

Compare Memory TechnologiesDRAM for first level capacity memoryNAND/PCM for next level storageSource: Intel19

3D-Integration of DRAM and LogicLogic Buffer ChipTechnology optimized for:High speed signalingEnergy efficient logic circuitsImplement intelligenceDRAM StackTechnology optimized for:Memory densityLower costLogic BufferPackage3D Integration provides best of both worlds

1Tb/s HMC DRAM Prototype• 3D integration technology• 1Gb DRAM Array• 512 MB total DRAM/cube• 128GB/s Bandwidth•

Energy/bit (pJ/bit)Communication Energy10010Betweencabinets10.1On DieOn DieBoard to BoardChip to chipBoard to Board0.010.1 1 10 100 1000Interconnect Distance (cm)23

On-die Interconnect1.21Compute energy0.80.60.4On die IC energy0.2090 65 45 32 22 14 10 7Source: IntelTechnology (nm)Interconnect energy (per mm) reduces slower than computeOn-die data movement energy will start to dominate24

21.72mmDDR3 MCDDR3 MC21.4mmDDR3 MCDDR3 MCNetwork On Chip (NoC)80 Core TFLOP Chip (2006)12.64mmI/O Areasingle tile1.5mm2.0mm48 Core Single Chip Cloud (2009)TILE26.5mmCCCCPLLTILEJTAGPLLI/O AreaTAPClockdist.11%8 X 10 Mesh32 bit links320 GB/sec bisectionBW @ 5 GHzDualFPMACs36%VRCSystem Interface + I/O2 Core clusters in 6 X 4 Mesh(why not 6 x 8?)128 bit links256 GB/sec bisection BW @ 2 GHzMC &DDR3-80019%Cores70%Router +Links28%10-portRF4%IMEM +DMEM21%Routers& 2Dmesh10%GlobalClocking1%25

Delay (ps)pJ/BitOn-chip Interconnects1000001000010001000.3u pitch, 0.5VRepeated wire delayRouter DelayWire Delay0 5 10 15 20 25Length (mm)Wire EnergyRouter Energy0.90.80.70.60.50.40.30.20.1026

Circuit Switched NoCClock8x8 NoCI/OTraffic Generationand MeasurementPortsCoreNorthSouthEastWestNarrow, high freq packet switchednetwork establishes a circuitData is transferred using wide andslower established circuit switched busDifferential, low swing bus improvesenergy efficiencyProcess45nm Hi-K/MGCMOS, 9 Metal CuNominal Supply 1.1VArbitration andRouter LogicNumber ofTransistorsSupports 512b data2.85MDie Area 6.25mm 22 to 3X increased energy efficiency over packet switched networkAnders et al, A 4.1Tb/s bisection-bandwidth 560Gb/s/W streaming circuit-switched 8×8 mesh network-on-chip in 45nm CMOS, ISSCC 201027

Hierarchical & HeterogeneousCCCCCCBus RBus RBusCCCCCCCCCCBus to connect overshort distancesCBus R2 nd Level BusCCBus RCHierarchy of BussesOr hierarchical circuitand packet switchednetworks28

Electrical Interconnect < 1 Meter1000100Energy (pJ/bit)10Data rate (Gb/sec)1Source: ISSCC papers0.11.2u 0.5u 0.25u 0.13u 65nm 32nmBW and Energy efficiency improves,but not enough29

Energy (pJ/bit)Electrical Interconnect AdvancesTop of the packageconnectorEmploy, new, low-loss, non-traditional interconnectsLow-loss flexconnector43State of the artLow-loss twinax210FlexHDITwinax01010-300-6-9-10-10-0.5 -0.25 0 0.25 0.5-0.5 -0.25 0 0.25-120.50 100 200 300 400Channel length (cm)Co-optimization of interconnectsand circuits for energy efficiencyO’Mahony et al, “A 47x10Gb/s 1.4mW/(Gb/s) Parallel Interface in 45nm CMOS “, ISSCC 2010; and J. Jaussi, RESS004, IDF 2010

Straw-man System InterconnectAssume: 40 Gbps, 10 pJ/b, $0.6/Gbps, 8B/FLOP, naïve tapering1,000fiberseach. ..8,000fiberseach. ..$35M217 MW............... 35 Clusters.......Clusterof 35~0.8 PF................ 35 Clusters32

Byte/FLOPDataMovementPower (MW)DataMovementPower (MW)Bandwidth Tapering1.E+021.E+011.E+001.E-011.E-021.E-031.E-041.E-05246.658 Byte/Flop total1.130.19Severe0.03Naïve, 4X0.00450.00051.5110001001010.1Total DM Power = 217 MWL1 L2 Chip Board Cab Sys0.00005 Total DM Power = 3 MWCore L1 L20.5Chip Board Cab L1SysL2Sys0L1 L2 Chip Board Cab SysIntelligent BW tapering is necessary33

Road to Unreliability?From Peta to ExaReliability Issues1,000X parallelism More hardware for something to go wrong>1,000X intermittent faults due to soft errorsAggressive Vcc scalingto reduce power/energyDeeply scaledtechnologiesGradual faults due to increased variationsMore susceptible to Vcc droops (noise)More susceptible to dynamic temp variationsExacerbates intermittent faults—soft errorsAging related faultsLack of burn-in?Variability increases dramaticallyResiliency will be the corner-stone34

n-SER/cell (sea-level)Relative to 130nmSoft Errors and Reliability10.80.60.40.2065nm90nm130nm180nm250nm0.5 1 1.5 2Voltage (V)Soft error/bit reduces each generationNominal impact of NTV on soft error rate101Assuming 2X bit/latchcount increase pergenerationLatchMemory180 130 90 65 45 32Positive impact of NTV on reliabilityTechnology (nm)Soft error at the system level willcontinue to increaseLow V lower E fields, low power lower temperatureDevice aging effects will be less of a concernLower electromigration related defectsN. Seifert et al, "Radiation-Induced Soft Error Rates of Advanced CMOS Bulk Devices", 44th Reliability Physics Symposium, 200635

ResiliencyFaultsExamplePermanent faults Stuck-at 0 & 1Gradual faultsVariabilityTemperatureIntermittent faults Soft errorsVoltage droopsAging faultsDegradationFaults cause errors (data & control)Datapath errors Detected by parity/ECCSilent data corruption Need HW hooksControl errors Control lost (Blue screen)Minimal overhead for resiliencyApplicationsSystem SoftwareProgramming systemMicrocode, PlatformMicroarchitectureCircuit & DesignError detectionFault isolationFault confinementReconfigurationRecovery & Adapt36

Architecture needs a Paradigm ShiftArchitect’s past and present priorities—Single thread performanceProgramming productivityConstraintsFrequencyLegacy, compatibilityArchitecture features for productivity(1) Cost(2) Reasonable Power/EnergyArchitect’s future priorities should be—Throughput performancePower/EnergyConstraintsParallelism, application specific HWArchitecture features for energySimplicity(1) Programming productivity(2) CostMust revisit and evaluate each (even legacy)architecture feature37

HW-SW Co-designApplications and SW stackprovide guidance for efficientsystem designApplicationsSystem SWProgramming SysArchitectureCircuits & DesignLimitations, issues andopportunities to exploit38

RelativePerformanceBottom-up GuidanceSD Leakage PowerMW1. NTV reduces energy but exacerbatesvariationsSmall & Fast coresRandom distributionTemp dependent2. Limited NTV for arrays (memory)due to stability issues3. On-die Interconnect energy (permm) does not reduce as much ascompute1.210.80.60.40.20MemoryComputeVoltageCompute EnergyInterconnect Energy45 32 22 14 10 7Technology (nm)DisproportionateMemory arrayscan be madelarger6X compute1.6X interconnect4. At NTV, leakage power is substantialportion of the total power60%50%40%30%20%10%0%40% Vdd50% Vdd75% Vdd100% VddIncreasingVariations45nm 32nm 22nm 14nm 10nm 7nm 5nmExpect 50%leakageIdle hardwareconsumes energy5. DRAM energy scales, but not enough1000100101DRAM Energy (pJ/b)3D Hybrid Memory Cube90nm 65nm 45nm 32nm 22nm 14nm 10nm 7nm50 pJ/b today8 pJ/bdemonstratedNeed < 2pJ/b6. System interconnect limited by laserenergy and cost30025020015010050040 Gbps Photonic links @ 10 pJ/bData Movement Power SystemCabinetBoardsDieClustersIslands45nm 32nm 22nm 14nm 10nm 7nm 5nmBW tapering andlocality awarenessnecessary39

Today’s HW System ArchitectureSmallRFCoreX8 FP32K I$32K D$128K I$Processor…X 16…~ 3GHz~100 WSmallRFCoreX8 FP32K I$32K D$128K I$4GBDRAMSocketSocket4GBDRAM4GBDRAMSocketSocket4GBDRAMCoherent domain1.5 TF Peak660 pJ/F 660 MW/Exa10 mB of DRAM/F128K D$16 MB L3128K D$.Non-coherent domain384 GF/s Peak260 pJ/F 260 MW/Exa55 mB of local memory/FToday’s programming modelcomprehends this systemarchitecture40

Service CoreService CoreService CoreService CoreService CoreService CoreService CoreService CoreLogicService CoreService CoreService CoreService CoreService CoreService CoreService CoreService CoreService CoreService CoreService CoreService CoreStraw-man Exascale ProcessorSimplest CoreRF*Logic600K TransistorsFirst level of hierarchyC C C CShared CacheC C C CNext level of hierarchyPE PE PE PE ………..1MB L2PE PE PE PE………..PE PE PE PE1MB L2PE PE PE PEInterconnectNext level cache………..PE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2………..PE PE PE PEProcessorNext level of hierarchyPE PE PE PE ………..1MB L2PE PE PE PE………..PE PE PE PE1MB L2PE PE PE PEInterconnectNext level cache………..PE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2Next level of hierarchyPE PE PE PE ………..1MB L2PE PE PE PE………..PE PE PE PE1MB L2PE PE PE PE………..InterconnectNext level cache………..………..Next level of hierarchyPE PE PE PE ………..1MB L2PE PE PE PE………..PE PE PE PEPE PE PE PEPE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2………..………..InterconnectLast level cache………..1MB L2InterconnectNext level cache………..PE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2………..PE PE PE PENext level of hierarchyPE PE PE PE ………..1MB L2PE PE PE PE………..PE PE PE PEPE PE PE PEPE PE PE PE1MB L2InterconnectNext level cache………..………..PE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2………..PE PE PE PETechnology 7nm, 2018Die area500 mm2Cores 2048Frequency 4.2 GHzTFLOPs 17.2Power600 WattsE Efficiency 34 pJ/FlopComputations alone consume 34 MW for Exascale41

LogicService CoreService CoreService CoreService CoreService CoreService CoreService CoreService CorePE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEService CoreService CoreService CoreService CorePE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEService CoreService CoreService CoreService CorePE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEService CoreService CoreService CoreService CorePE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEStraw-man Architecture at NTVProcessorSimplest CoreRF*Logic600K TransistorsFirst level of hierarchyC C C CShared CacheC C C CNext level of hierarchyPE PE PE PE ………..1MB L2PE PE PE PE………..PE PE PE PE1MB L2PE PE PE PEInterconnectNext level cache………..PE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2………..PE PE PE PENext level of hierarchy………..………..InterconnectNext level cache………..Next level of hierarchy………..………..………..InterconnectNext level cache………..………..………..………..InterconnectLast level cache………..Next level of hierarchy………..………..InterconnectNext level cache………..Next level of hierarchy………..………..InterconnectNext level cache………..………..………..………..Full Vdd 50% VddTechnology 7nm, 2018Die area 500 mm2Cores 2048Frequency 4.2 GHz 600 MHzTFLOPs 17.2 2.5Power 600 Watts 37 WattsE Efficiency 34 pJ/Flop 15 pJ/FlopReduced frequencyand flopsReduced power andimproved E-efficiencyCompute energy efficiency close to Exascale goal42

Micro-Byte/FLOPsLocal Memory Capacity1000100101TraditionalExa-StrawmanAt NTVL1 L2 L3 L4Memory HierarchyHigher local memory capacity promotes data locality43

Interconnect StructuresBuses over short distanceShared memoryCross Bar SwitchShared busMulti-ported MemoryX-Bar1 to 10 fJ/bit0 to 5mmLimited scalability10 to 100 fJ/bit1 to 5mmLimited scalability0.1 to 1pJ/bit2 to 10mmModerate scalabilityPacket Switched Network Board CabinetSystemSecond LevelSwitch. ...First levelSwitch. ..SwitchCluster………………………….CabinetCluster………………………….…………………………1 to 3pJ/bit>5 mm, scalable44

Programming modelExecution modelSW Challenges1.Extreme parallelism (1000X due to Exa,additional 4X due to NTV)2.Data locality—reduce data movement3.Intelligent scheduling—move thread todata if necessary4.Fine grain resource management(objective function)5.Applications and algorithms incorporateparadigm change45

Programming & Execution ModelEvent driven tasks (EDT)Dataflow inspired, tiny codelets (self contained)Non blocking, no preemptionProgramming model:Separation of concerns: Domain specification & HW mappingExpress data locality with hierarchical tilingGlobal, shared, non-coherent address spaceOptimization and auto generation of EDTs (HW specific)Execution model:Dynamic, event-driven scheduling, non-blockingDynamic decision to move computation to dataObservation based adaption (self-awareness)Implemented in the runtime environmentSeparation of concerns:User application, control, and resource management46

Over-provisioning, Introspection, Self-awarenessService CoreService CoreService CoreService CorePE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEService CoreService CoreService CoreService CorePE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEService CoreService CoreService CoreService CorePE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEService CoreService CoreService CoreService CorePE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEPE PE PE PE1MB L2PE PE PE PEAddressing variationsFine grain resource mgmtSensors for introspectionFMFFMSSSS M F MF S S S1. Provide more computeHW2. Law of large numbers3. Static profileFMFFMSSSS M F MF S S SDynamic reconfiguration:1. Energy efficiency2. Latency3. Dynamic resourcemanagementInterconnect8MB Shared LLC………..………..Interconnect8MB Shared LLC………..Processor Chip (16 Clusters)………..Interconnect64MB Shared LLC………..Interconnect8MB Shared LLC………..Interconnect8MB Shared LLC………..………..1. Energy consumption2. Instantaneous power3. Computations4. Data movement1. Schedule threads based on objectives and resources2. Dynamically control and manage resources3. Identify sensors, functions in HW for implementationSystem SW implements introspective execution model47

MWMWOver-provisioned Introspectively ResourceManaged SystemOver-provisioned in design140120100806040200Dynamically tuned for the given objectiveSystem PowerData MovementPower w/o DMSys Goal20 MW45nm 32nm 22nm 14nm 10nm 7nm 5nm1412108642040 Gbps Photonic links @ 10 pJ/bSystemCabinetBoardsDieClustersIslandsData Movement Power3.23 MW45nm 32nm 22nm 14nm 10nm 7nm 5nm48

SummaryPower & energy challenge continuesOpportunistically employ NTV operation3D integration for DRAMCommunication energy will far exceed computationData locality will be paramountRevolutionary software stack needed to makeExascale real49