High Performance Computing Cluster Interconnect - Extreme Networks

Extreme Networks White PaperHigh PerformanceComputing ClusterInterconnectAbstractHigh Performance Computing Clusters (HPCC) areno longer exclusive to the scientific community. “BigData Analytics” are big business. Financial Services,Retail and Manufacturing all require real-time analysisfor real-time business decisions. This white paperdiscusses the cluster interconnect, its importanceto HPCC and a lowest latency, higher performingsolution from Extreme Networks®.Make Your Network Mobile© 2012 Extreme Networks, Inc. All rights reserved.

Extreme Networks White Paper: High Performance Computing Cluster InterconnectHPCC is no longer an exclusive solution for the scientificcommunity. As “Big Data Analytics” are required formore businesses, HPCC is being used as a more commercially-orientedsolution. Financial Services, Retail andManufacturing all have the same challenges of analyzingreal time data from many different sources to make realtime decisions. As HPCC becomes more mainstream anda solution is chosen, the cluster interconnect becomesan important decision.As an example, petabytes of unstructured data frominternet applications must now be processed withina short period of time. One approach is the HadoopMapReduce framework along with the HadoopDistributed File System (HDFS), designed to run oncommodity hardware based on Linux. The HPCCsolution is known as a Hadoop Cluster. In a low-latencydeployment, tests have proven that Solid State Disk(SSD) in combination with advanced interconnect like10 GbE, have a significant enhancement to performanceof the cluster.A cluster must have a dedicated cluster interconnect towhich all cluster members are connected. Bandwidthand latency are the most important factors in the overallperformance of the solution. Today, a majority of HPCCinterconnect is Ethernet due to cost effectiveness, ubiquity,and open standards. The most effective solution isbased on a variety of factors such as:• Cost• Cluster availability• Cluster performance• Application requirementsAs 10 GbE and now 40 GbE become a standard inthe data center, more and more clusters are using thetechnology as a Top-of-Rack or End-of-Row architecture,or to connect to the Internet. Extreme Networks offersthe lowest latency with the highest throughput in theindustry, combined with ExtremeXOS®, an extensible andresilient operating system.2© 2012 Extreme Networks, Inc. All rights reserved.

Extreme Networks White Paper: High Performance Computing Cluster InterconnectBlackDiamond® X series:Highest density, lowest latency,data center, cloud, and highperformance computing coreand EOR switchBlackDiamond 8900 series:Cost-effective 1 GbE/10 GbEdata center EOR switchSummit® X670/X670V:Low latency, cost-effective,10 GbE/40 GbE data centerTOR switch• Unmatched switching functionality––20+ Tbps switching capacity––1.28 Tbps bandwidth per slot––192 x 40 GbE or 768 x 10 GbE• Superior availability––N+1 switching, power &fan redundancy––N+N power GRID redundancy• Low latency––

Extreme Networks White Paper: High Performance Computing Cluster InterconnectThe Lippis Report went on to make the following observations:• “The above latency measures are the lowest we have measured for core switches by a large degree. The BDX8forwards packets ten to six times faster than other core switches we’ve tested.”• “The BDX8 delivered the fastest IP Multicast performance measured to date being able to forward IP Multicastpackets between 3 and 13 times faster than previous core switch measures of similar 10 GbE density.”• “This measurement also breaks all previous records as the BDX8 is between 2 and 10 times faster in forwardingcloud based protocols under load. In fact, the Extreme Networks BDX8 was between 2 and 9 times faster atforwarding packets over a wide range of frame sizes than previous core switch measurements conducted duringLippis/Ixia iSimCity test of core switches from Alcatel-Lucent, Arista and Juniper.”• “In other words, the BDX8 is the fastest core switch we have tested with the lowest latency measurements.”A typical BlackDiamond X8 and Summit X670 deployment in an HPCC environment is below. Total two-tier networklatency (Summit X670 -> BlackDiamond X8 -> Summit X670) will be sub-5 µsec at average packet sizes.Campus NetworkCore LayerBlackDiamond ®X8BlackDiamondX8Access LayerTop of the RackSummit ® X670 Summit X670 Summit X670 Summit X670 Summit X670 Summit X670ServerRackServerRackServerRackServerRackServerRackServerRack5597-01In summary, cluster interconnect is an important decision. Ethernet is quickly becoming the de-facto standard dueto not only the CapEx savings but more importantly the OpEx costs it takes to manage a separate network. Thefact that Ethernet is built upon open standards, means customers are assured of interoperability with the rest of thenetwork and additional flexibility as the cluster grows will make it the best option. Finally, with the advancements inlatency and throughput, Ethernet becomes the only logical choice.5© 2012 Extreme Networks, Inc. All rights reserved.

Extreme Networks White Paper: High Performance Computing Cluster InterconnectCustomer ReferencesAtomic WeaponsEstablishment, UKThe Atomic Weapons Establishment’s (AWE) HPCC applications require Extreme Networks modularswitching solutions to offer greater flexibility and meet organizational needs without sacrificingperformance or increasing costs. For AWE, Extreme Networks supports high network availability byoffering advanced features including VRRP and ESRP for fully coordinated Layer 2/3 client routerredundancy. Because AWE regularly moves large amounts of data from disks to the compute area,line rate throughput is a paramount concern—as degradation in performance is unacceptable. Fornetworks processing and managing large amounts of traffic, such as AWE’s clustering applications,Extreme Networks jumbo frames increase efficiency.CEA Saclay Grid Project“Carriocas”The Carriocas grid run by CEA Saclay explores and develops hardware and software components of afiber optic network at high speeds—10 Gbit/s per channel—for developing distributed applications forthe purpose of interactive digital simulation on supercomputers and remote processing and visualizationof very large volumes of data stored remotely and in a distributed manner. The Extreme NetworksHPCC solution supports performance scaling beyond 10 GbE and offers flexibility, both crucial in theresearch fields. Despite analysis demands that have been up to three times larger than expected,production with the Saclay Grid has continued to operate smoothly, underlining the capacity androbust performance of the solution.European SynchrotronRadiation Facility (ESRF)The ESRF is a collaboration between a dozen European countries that have joined forces in Grenoble.Comparable to “super microscopes”, synchrotron radiation sources provide invaluable information toa number of research fields. The ESRF has deployed Extreme Networks modular BlackDiamond 8800switches and Summit fixed Gigabit switches. The high performance network for ESRF has stoodout from its rivals on numerous technical points, including redundancy. The ExtremeXOS modularsoftware also boasts numerous advantages: with software updating takes place in a transparentmanner for the user, while hardware failures are accurately diagnosed and resolved.PetroChinaExtreme Networks is serving the needs of the China National Petroleum Corporation, known asPetroChina, at multiple locations including its Research Institute and on location at the Daqing andLiaohe oil fields. PetroChina’s HPCC systems are supported by Extreme Networks high capacity,low latency Ethernet switches. HPCC underpins core scientific research capabilities that boost theorganization’s energy exploration, where classic computing and GPU clusters are maintained toconduct massive and ongoing computer-generated simulations of the earth. The benefit is faster andmore accurate discovery of the most feasible and energy rich resources for oil and petroleum acrossdiffering regions. Extreme Networks meets PetroChina’s diverse needs for HPCC connectivity, supplyingits versatile BlackDiamond 8800 series modular switches and Summit X450 andSummit X650 fixed switches.Make Your Network MobileCorporateand North America3585 Monroe StreetSanta Clara, CA 95051United StatesPhone +1 408 579 2800Europe, Middle East, Africaand South AmericaPhone +31 30 800 5100Asia PacificPhone +65 6836 5437Latin AmericaPhone +52 55 5557 6465www.extremenetworks.com© 2012 Extreme Networks, Inc. All rights reserved. Extreme Networks, the Extreme Networks logo, BlackDiamond, ExtremeXOS, Summit and SummitStackare either registered trademarks or trademarks of Extreme Networks, Inc. in the United States and/or other countries. All other names and marks are theproperty of their respective owners. Specifications are subject to change without notice. 1821_02 03/12