From Beowulf to the HIVE - Linux Clusters Institute
From Beowulf to the HIVE - Linux Clusters Institute
From Beowulf to the HIVE - Linux Clusters Institute
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
Commodity Cluster Computing<br />
at<br />
Goddard Space Flight Center<br />
Dr. John E. Dorband<br />
NASA Goddard Space Flight Center<br />
Earth and Space Data Computing Division<br />
Applied Information Sciences Branch<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
1
The Legacy<br />
MasPar MP1/MP2<br />
1990-2000<br />
16K cus<strong>to</strong>m proc.<br />
$1.5-2 Million<br />
6.2 GFlop<br />
3.7 sustained (PPM)<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
2
<strong>Beowulf</strong> I: Wiglaf<br />
1994<br />
16 - 100Mhz 486DX4<br />
$40,000<br />
50 MFlops sustained (PPM)<br />
Dual 10Mhz e<strong>the</strong>rnet<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
3
<strong>Beowulf</strong> II: Hrothgar<br />
1995<br />
16 - 100Mhz Pentium<br />
$40,000<br />
150 Mflops sustained (PPM)<br />
Dual 100Mhz hub-based e<strong>the</strong>rnet<br />
1999: Dual 100Mhz switched e<strong>the</strong>rnet<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
4
1997<br />
60 - 200Mhz Pentium Pro<br />
100Mhz e<strong>the</strong>rnet<br />
Communications thru<br />
10 distribution nodes<br />
<strong>Beowulf</strong> III: Ecg<strong>the</strong>ow<br />
<strong>Beowulf</strong> Bulk S<strong>to</strong>rage System<br />
1999<br />
128 - 200Mhz Pentium Pro<br />
100Mhz switched e<strong>the</strong>rnet<br />
72 port switch<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
5
<strong>the</strong><strong>HIVE</strong><br />
Highly-parallel Integrated Virtual Environment<br />
Sponsored by:<br />
Mission <strong>to</strong> Planet Earth and<br />
Code S Office of Space Science Advanced Technology<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
6
Goal<br />
• To build a cluster for those who did not<br />
build it<br />
• Requirements<br />
– Reliability<br />
– Ease of use<br />
– Cost performance<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
7
Initial Assumptions<br />
• Low cost<br />
• One or few users<br />
• Low contention for resources<br />
• Implication<br />
– No scheduling - free-for-all<br />
– No accounting<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
8
Reliability<br />
• Long uptime, short down time, & no loss of<br />
data<br />
• Met with<br />
– Commodity hardware<br />
– Well used open source software<br />
– Hot system backup<br />
– RAID technology<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
9
Ease of Use<br />
• Easy/Quick Maintenance<br />
– Installation<br />
– Upgrade<br />
• Easy application development<br />
• Met with<br />
– Parallel maintenance<br />
– Parallel command shell<br />
– Message Passing<br />
– Global Filesystem<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
10
Cost Performance<br />
• Open Source Software<br />
• Commodity Hardware<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
11
Open Source Needs<br />
• Parallel Debugger<br />
• Hardware Diagnostics<br />
• Global parallel programming paradigm<br />
• Low latency network pro<strong>to</strong>col<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
12
The <strong>HIVE</strong> Concept<br />
Dvlp<br />
Dvlp<br />
Console<br />
execute<br />
nodes<br />
S<br />
T<br />
O<br />
R<br />
A<br />
G<br />
E<br />
Dvlp<br />
Dvlp = Developer’s<br />
Dvlp<br />
Dvlp<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
13
<strong>the</strong><strong>HIVE</strong><br />
Highly-Parallel Integrate Virtual Environment<br />
1997<br />
128 - 200Mhz Pentium Pro<br />
$210,000<br />
7.5 Gflops sustained (PPM)<br />
100Mhz switched e<strong>the</strong>rnet<br />
5 - 16 port switches<br />
1999<br />
128 - 200Mhz Pentium Pro<br />
100Mhz switched e<strong>the</strong>rnet<br />
72 port switch<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
14
What is <strong>the</strong><strong>HIVE</strong>?<br />
? Lots of fans<br />
? A <strong>Beowulf</strong>-class Parallel Computer<br />
(i.e. Lots of PCs running LINUX)<br />
? Highly-parallel Integrated Virtual<br />
Environment (<strong>HIVE</strong>)<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
15
The Spec<br />
• To provide a high performance computing<br />
resource (5-10Gflop) for earth and space<br />
science applications at low cost (
<strong>the</strong><strong>HIVE</strong><br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
17
Architecture 1997<br />
Administrative<br />
Host<br />
Q0<br />
D0<br />
User<br />
Host<br />
C0 C1 C2 C3 C4<br />
5 fast<br />
e<strong>the</strong>rnet<br />
switches<br />
b0 - b15 b16 - b31 b32 - b47 b48 - b63<br />
64 dual processor nodes<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
18
Architecture 1998<br />
Administrative<br />
Host<br />
Q0<br />
D0<br />
User<br />
Host<br />
72 port switch<br />
1 fast<br />
e<strong>the</strong>rnet<br />
switch<br />
b0 - b15 b16 - b31 b32 - b47 b48 - b63<br />
64 dual processor nodes<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
19
Architecture 1999<br />
Administrative<br />
Host<br />
User<br />
Host<br />
16 Dual<br />
PIII Xeon<br />
E B G DL<br />
10 Quad<br />
PIII Xeon<br />
Fast<br />
E<strong>the</strong>rnet<br />
switches<br />
64 dual<br />
P-Pro<br />
66 dual<br />
P-Pro<br />
Myrinet<br />
Gigabit<br />
E<strong>the</strong>rnet<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
20
Architecture 2000<br />
Console<br />
Host<br />
66 dual<br />
P-Pro<br />
User<br />
Host<br />
16 Dual<br />
PIII Xeon<br />
B G DL<br />
10 Quad<br />
PIII Xeon<br />
Fast<br />
E<strong>the</strong>rnet<br />
switches<br />
Topaz<br />
Fast E<strong>the</strong>rnet<br />
Myrinet<br />
16 Dual<br />
PIII - 700 Mhz<br />
SGI<br />
Gigabit<br />
E<strong>the</strong>rnet<br />
Myrinet<br />
Orka<br />
VA<strong>Linux</strong><br />
BaByBlue<br />
IBM<br />
<strong>the</strong><strong>HIVE</strong><br />
Fast E<strong>the</strong>rnet<br />
Fast E<strong>the</strong>rnet<br />
Myrinet<br />
Myrinet<br />
16 Dual<br />
PIII - 700 Mhz<br />
16 Dual<br />
PIII - 650 Mhz<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
21
Hardware 1997<br />
• 64 compute nodes (b0-b63)<br />
• 2 host nodes (Q0 and D0)<br />
• 128 - Pentium Pro processors (200Mhz, 256K cache)<br />
• 4 Gbyte of RAM (64 MB per node)<br />
• 160 Gbyte Disk (2.5 GB per node)<br />
• 5 100Mhz Full-duplex Fast e<strong>the</strong>rnet switches (C0-C4)<br />
• 6.4 Gbit/sec maximum inter-processor bandwidth<br />
• 1600 Mbit/sec within a switch<br />
• 400 Mbits/sec between switches<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
22
Hardware 1999<br />
• 64 compute nodes (b0-b63)<br />
• 2 host nodes (Q0 and D0)<br />
• 128 - Pentium Pro processors (200Mhz, 256K cache)<br />
• 28 Gbyte of RAM (448 MB per node)<br />
• 896 Gbyte Disk (14 GB per node)<br />
• 100Mhz Full-duplex Fast e<strong>the</strong>rnet switch (72 port)<br />
• 6.4 Gbit/sec maximum inter-processor bandwidth<br />
• 6400 Mbit/sec within <strong>the</strong> switch<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
23
The Essence of a BEE<br />
• 1 Mo<strong>the</strong>rboard<br />
• 2 CPU Chips + Fans<br />
• 64M Memory<br />
• 3G EIDE Hard Drive<br />
• 1 Floppy Drive<br />
• 1 PCI Video Card<br />
• 1 PCI E<strong>the</strong>rnet Card<br />
• 1 Chassis + Power Supply<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
24
The Essence of <strong>the</strong> QUEEN<br />
• 1 Mo<strong>the</strong>rboard<br />
• 2 CPU Chips + Fans<br />
• 64M Memory<br />
• 10G EIDE Hard Drive<br />
• 1 Floppy Drive<br />
• 1 PCI Video Card<br />
• 2 PCI E<strong>the</strong>rnet Card<br />
• 1 EIDE CDROM Drive<br />
• 1 Display Moni<strong>to</strong>r<br />
• 1 Keyboard<br />
• 1 3 but<strong>to</strong>n Mouse<br />
• 1 Case + Power Supply<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
25
Building <strong>the</strong><strong>HIVE</strong> (unpacking)<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
26
Building <strong>the</strong><strong>HIVE</strong> (chassis)<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
27
Building <strong>the</strong><strong>HIVE</strong> (components)<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
28
Building <strong>the</strong><strong>HIVE</strong> (<strong>the</strong>PILE)<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
29
Building <strong>the</strong><strong>HIVE</strong> (<strong>the</strong>RACK)<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
30
Inside <strong>the</strong><strong>HIVE</strong><br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
31
<strong>the</strong><strong>HIVE</strong><br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
32
The 1999 Extension<br />
1999<br />
16 Gateway Nodes<br />
- dual PIII Xeons<br />
- 8 GB ram<br />
10 Dell Nodes<br />
- quad PIII Xeons<br />
- 5 GB ram<br />
- 1Gb/s Myrinet<br />
- Fast E<strong>the</strong>rnet<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
33
2000<br />
32 - 700Mhz Pentium III<br />
$270,000<br />
1 Gb/s Myrinet<br />
100Mhz E<strong>the</strong>rnet<br />
200 GB Disk<br />
16 GB RAM<br />
1 Console node<br />
2 Developer nodes<br />
4 S<strong>to</strong>rage nodes<br />
16 Compute nodes<br />
Topaz<br />
SGI Commodity Cluster<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
34
Inside Topaz<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
35
BaByBlue<br />
IBM Commodity Cluster<br />
2000<br />
32 - 650Mhz Pentium III<br />
$270,000<br />
1 Gb/s Myrinet<br />
100Mhz E<strong>the</strong>rnet<br />
300 GB Disk<br />
16 GB RAM<br />
1 Console node<br />
2 Developer nodes<br />
2 S<strong>to</strong>rage nodes<br />
16 Compute nodes<br />
QuickTime and a<br />
Pho<strong>to</strong> - JPEG decompressor<br />
are needed <strong>to</strong> see this picture.<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
36
Orka<br />
VA<strong>Linux</strong> Commodity Cluster<br />
2000<br />
32 - 700Mhz Pentium III<br />
$160,000<br />
1 Gb/s Myrinet<br />
100Mhz E<strong>the</strong>rnet<br />
100 GB Disk<br />
16 GB RAM<br />
1 Console/S<strong>to</strong>rage node<br />
1 Developer node<br />
16 Compute nodes<br />
QuickTime and a<br />
Pho<strong>to</strong> - JPEG decompressor<br />
are needed <strong>to</strong> see this picture.<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
37
Inside Orka<br />
QuickTime and a<br />
Pho<strong>to</strong> - JPEG decompressor<br />
are needed <strong>to</strong> see this picture.<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
38
Installing <strong>the</strong>System<br />
• Key components & steps<br />
– Install system on Queen<br />
– Bootp - Remote boot information server<br />
– The Floppy System on BEEs<br />
• Partition Disks & install Initial System<br />
– The Initial System on BEEs<br />
• Install Standard & Recovery Systems<br />
– The Standard & Recovery Systems on BEEs<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
39
How <strong>to</strong> use <strong>the</strong><strong>HIVE</strong><br />
• Data parallel MIMD (i.e. SPMD)<br />
– However one is not restricted <strong>to</strong> SPMD only.<br />
• Drones are user hosts in<strong>to</strong> <strong>the</strong><strong>HIVE</strong><br />
• Identical file system on all BEEs<br />
• SPMD commands<br />
– Most shell commands available on BEES<br />
– Ssh, Scp, Psh, Pcp, Hsh, and Hcp<br />
• All /home direc<strong>to</strong>ries are cross mounted on<br />
all nodes under /homes<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
40
Software<br />
• LINUX operating system<br />
• Simple command distribution<br />
• Message passing (PVM, MPI)<br />
• Distributed shared-memory interface (BSP)<br />
• Programming Languages: C, C++,<br />
Fortran77, Fortran90, aCe, and Lisp<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
41
Applications Areas<br />
• Image Data Base S<strong>to</strong>rage & Retrieval<br />
• Ocean/Atmosphere Wea<strong>the</strong>r Prediction<br />
• Astronomical Image Res<strong>to</strong>ration<br />
• Hyper-spectral image information<br />
extraction<br />
• Physical System Modeling<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
42
Specific Applications<br />
? PPM Fluid Dynamics<br />
? Image Segmentation<br />
? Parallel Adaptive Mesh Refinement<br />
? Plasma Diagnostics and Electromagnetic<br />
Interferometry<br />
? Community Climate Model (wea<strong>the</strong>r)<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
43
Image Segmentation<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
44
Performance<br />
• PPM fluid dynamics -- 7.5 Gflops<br />
• Mandlebrot set calculation -- 10.2 Gflops<br />
• Local Communication -- 5.2 Gbit/sec<br />
• Global Exchange of Data -- 320 Mbits/s<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
45
<strong>the</strong>His<strong>to</strong>ry of <strong>the</strong><strong>HIVE</strong><br />
• March 1, 1997 - Planning began.<br />
• April 1, 1997 - Procurement began<br />
• May 31, 1997 - Procurement complete<br />
• June 31, 1997 - Majority of parts received.<br />
• July 1, 1997 - Assembly began<br />
• July 22, 1997 - Chassis assembly complete (~ 6 man-weeks)<br />
• July 23, 1997 - Software installation began<br />
• August 7,1997 - Cabinet assembly and chassis installation began.<br />
• August 15, 1997 - Assembly complete ( ~ 2 additional man-weeks)<br />
• August 31, 1997 - Software installation complete.<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
46
Problems & Limitations<br />
• Only moderate interprocessor bandwidth<br />
• Long message latency<br />
• LINUX was not designed for parallel<br />
processing<br />
• Stability (software releases)<br />
• Not much parallel software available<br />
• No parallel installation software<br />
• Hard <strong>to</strong> get users <strong>to</strong> write parallel code<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
47
The Future<br />
• Fy00 - 3 small 16 processor testbeds<br />
– (i.e. VA<strong>Linux</strong>, IBM. & SGI )<br />
• Fy02 - a large 512 processor testbed<br />
– (possibly heterogeneous)<br />
• Fy04 - a 2048 processor testbed<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
48
Conclusions<br />
• High performance<br />
• Low Cost<br />
• Very reliable<br />
• Easy <strong>to</strong> use<br />
• Flexible<br />
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
49
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
50
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
51
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
52
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
53
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
54
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
55
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
56
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
57
J. E. Dorband 9/15/00<br />
<strong>From</strong> <strong>Beowulf</strong> <strong>to</strong> <strong>the</strong> <strong>HIVE</strong><br />
HPCC Program<br />
ESS<br />
Earth and Space Science Project<br />
58
Change language