Introduction to High-Performance Computing

Introduction toHigh-Performance ComputingGerolf Ziegenhain – TU Kaiserslautern, Germany

Definitions - General Part

What is HPC?●●●●●HPC = High-Performance Computing~ SupercomputingHigh computer power for– Calculations– Storage– High availabilityHPC: concentration on calculationsFastest high-performance computers top500.orgNB: Ghz ≠ GFLOPS (Floating Point Operations)

Current Status (November 2009)top500.org●IBM, HP, Cray dominate high-performance market

Vendors of the Fastest Computers●IBM, HP, Cray increased their dominationtop500.org

Operating Systems● All POSIX ⇒ core environment similartop500.org

●●POSIX ≠ W*nd0wsOperating Systems– Portable Operating System Interface [for UniX]"– Some POSIX OS●●●LinuxFreeBSD, OpenBSDSolaris, AIXWhere do POSIX like OS run?– Mobile phone– WIFI-router– GPS navigation system– Desktop– HPC⇒ Key technology

Where Are the Clusters Located?top500.org

Who Uses HPC?google.com

What Can Be Done With HPC?●●Performance / Storage / High availabilityExamples:– Cluster / || computing (P,S)– Google (P, S, H)– Cloud-computing (P, S, H)– Webserver (S, H)– Phone (P, H)– Video (P, S, H)⇒ Key technology

Who Else Uses HPC?●Industry– Product development (material properties)– Research (simulation of crash tests)– Production (robotics, optimization)– Collaboration (world-wide)– Interaction with customer (homebanking, call center)●●Universities– Research (biology, physics, chemistry, mathematics,engineering)Government– Infrastructure– Interaction citizen ↔ government

Cluster Architectures

Von Neumann-Architecture●CPU interacts with memory:●Speedup: multiple instructions at once– Pipelining– Pentium, PowerPC, Cell Processor (PS3), GPGPU●Multiple CPUs, one memory (shared memory)– SMP, vector computer●Multiple computers (distributed memory)– Transputer, cluster

How to Couple the CPUs?●●●Protected memory– I.e. multiple accessCluster filesystems– I.e. mixing local/distributed approachNetworking topology– Depending on problem⇒ Coupling of CPUs depends on coupling of theproblem to solve

Cluster Types●●●Homogenous cluster– SupercomputersHeterogenous cluster– Old hardwareGrid computing– Screensaver → folding@home●Restriction: Here only AMD/Intel alike desktop pcs⇒ Beowolf cluster

Examples of Decompositions

Physical Confinements●System ↔ Hardware ↔ Algorithm– Interaction, ρ({x}, t), Symmetry●Range of interaction– Short: decomposition with all-to-all-communication– Long: tree algorithms●●Density distribution– Homogenous system ⇒ balanced load betweennodesSymmetry– Choice of networking

Example - Strong Coupling●Atomistic simulation● Many particles ⇒ || simulation●●Parallelized in spaceInteraction– Width ⇒ R cut– Interaction of boundary particles⇒ Shared boundaries●Critical: Surface / Volume ratio⇒ Strong couplingZiegenhain 2009

Example - Weak Coupling●Problem:– Infrequent state– Low transition rate– But: full dynamics needed●Idea: use statistics to achieve transition– Parallel replicaZiegenhain 2007⇒ Weak coupling

Paradigm Change●●Algorithm can be decomposed innon-/parallelizable partsGeneral limit for parallelization:– Due to communication– S(p): speedup as function of #cpu– I.e. 100 painters brushing one room⇒ Amdahls Law (1967)● Gustavson (1988):Ziegenhain 2008– Decomposition of problem matters (system has toscale)– I.e. 100 painters brushing 100 rooms

What is a Beowolf Cluster?●●●Developed 1992 @ NASAIdea: Do not built supercomputer, but usecommon hardware!– Usual user-hardware PCs– Networking interconnectAdvantages– Use old pcs (heterogenous cluster)– Use CIP pool in the night– Usual staff-pcs in the night

Key technology: Hardware●Influence with industry– Ha-server– Cloud computing●Influence with infrastructure– Ha-server– E-government

How to Use Computers in Research?

What is Computersimulations?●Objective: description of the reality– Therefore: model systems● Simulation method ⇐ model system– There is no the computer simulation

Computer Simulations●Methodologic branches in science– Theory– Experiment– Simulations●●●●Missing linkLook at difficult-to-extract phenomenaExperimental observation / preparation not possibleTheory can only give approximative solution

Motivation for Simulations● Scientific● Test of model systems● Total control on parameters● Observables easily accessible● Economic● Cheap● Universal● General● (Dis-)advantage depends on method● Computation power still far to small to work without tricks● It's not always worthwhile to run a simulation

Simulations on Different Length Scales●For different scales● Different aspects of physics● Different model systems● Different simulation methods●Problematic: coupling of scales

Dimensions of Simulations●How long can a simulation run– Seconds → ∞– Usually: ~hours → ~days●How much data is created?– MByte → TByte– Usually: ~GByte

Global Experiment●●●●●∃ New chances through open source softwareUse remote clusterCloudApply for cpu time / storageWork from remote

What Can HPC Give to a University?●●●●Results achieved relatively easilyVisibility through open source contributionsWorldwide collaborations with EU / USASource for common problems / fields exists● Many application purposes ⇒ all departments mayshare a cluster●External computation can be used

●Industry– Clusters– Cloud– Web– Phone– TvBenefit for Country●●Reduction of administrative overheadCustomer interactions

Scientific Benefit●●●●●Global experimentGrid computingRelatively easy to achieve new results becausethe field is newExact control over model system and parametersMissing link

Selection of Grand Challenges●Hydrodynamics– Design of bodies for aircraft & automobiles– Turbulence– Weather forecasting / estimating climate change●Plasma dynamics– Fusion energy●Symbolic computation– Speech recognition– Semantic data manipulation●Material science– Nanomaterials

The Big Picture●●●Next wave is comingWeb is revolutionary similar to book printingFree access to information

Key Technologies●●●HardwareOSSoftware

Zuse Z3 (1941)●●~ 0.3 Flops1 node, 2.600 relais● 176 byte memoryDeutsches Museum

Cray Jaguar (2009)●Cray Jaguar●●●2.3 Peta-Flops224.162 nodes, 8.4*10¹² gatesSpecial network, filesystemWikipedia

Thank you!●Acknowledgement:– Prof. Dr. rer. nat. Herbert M. Urbassek,TU Kaiserslautern, Germany