this T-Platforms

Products and services

T-Platforms | One-stop source for petascale computing

Products and serviceswww.t-platforms.com

Today, when it comes to acquisition and installation ofHPC systems, software and services, many organizationsare looking for just the right balance of innovation,flexibility, performance and investment protection.This demand is driving a wider spread adoption of turnkeyHPC solutions for companies of all sizes.CONTENTS05 Lines of Business06 High End HPC Systems14 Storage Systems20 ClustrX HPC Software24 HPC Services28 Appendix.Product Feature Matrix29 T-Blade 2 Family30 T-Blade V-Class Family32 T-Mini familyFrom configuring a straightforward HPC cluster or largesupercomputer system, to delivering a complete HPCsolution including the design and development of an entiredatacenter facility, T-Platforms has the expertise andresources to be your one-stop technology, system andsolution provider.T-Platforms is a full service HPC vendor that designs,produces, and deploys comprehensive supercomputingsystems and software. Many of our hardware productsare configured and sold as part of turnkey solutions.HPC solutions from T-Platforms are an excellent choicefor high-end systems in fields ranging from energy,environmental research, CAE, and CFD, to seismicprocessing, nanoscience and drug design for commercial,academic and government organizations where extremecomputation and time-to-discovery or time-to-market arekey competitive elements.Lines of Business■■Development and production of various scale HPCsystems, including Petaflops level■■Software development for HPC systems■■HPC specific storage system portfolio■■Turn key datacenter design■■Engineering simulations and on-demand computingservices■■Full cycle outsourcing, from computing capacity leaseto HPC datacenter management services05

High End HPC: TB2 product familyHigh End HPC SystemsThe T-Blade 2 system family is used in datacenters atthe largest universities and scientific centers to solve themost compute-intensive problems. The system familyprovides the highest performance, scalability and density.01 TB2 product family02 V-Class Family03 T-Mini P-ClassThe T-Blade 2 family is comprised of:• TB2-TL. A flagship solution with the highest availablecomputing density delivered by up to 32 NVIDIA TeslaX2070 accelerators installed in a single chassis• TB2-XN. A solution based on 64 high performance Intel XeonX5600 CPUs in a single chassis• TB2W. A concept by T-Platforms, with a hybrid airand a hot water-based cooling system06The T-Blade 2 is the foundation for the Lomonosovsupercomputer deployed at Moscow State University. Withthe latest system’s upgrade, the peak performance exceeds1.3Petaflops, taking 13th place on the TOP500 list of theworld’s most powerful supercomputers (June 2011) whilemaintaining a leading position on the most powerful Russiansupercomputer list. The supercomputer has also becomea leader in the Graph500 test, measuring real applicationperformance in data-intensive environments.T-Platforms | One-stop source for petascale computing

TB2-TL Compute nodeT-Platforms has partnered with NVIDIA to deliverthe powerful and unique TB2-TL – an idealsolution for the CUDA-optimized applications insegments requiring the versatility and scalabilityof heterogeneous systems with the highestlevel of performance, such as Oil and Gas,Bioinformatics, Financial Services, CAE, andClimate Solutions.The TB2-TL system is based on the newestTL-blade design. The TB2-TL supports sustainedpetascale operations with an infrastructurearchitected for the eventual transition toexascale.TB2-TL:• 10 rack cabinets with 60 TB2-TL systemsprovide over 1PF of peak performance (DP)• Up to 16 TL per TB2 enclosure with 32Tesla X2070 GPU SXMs and 32 Intel®Xeon® L5630 quad core CPU’s• 17.5 ТF peak system performance (DP)• Uncompromised throughout with dedicatedPCIe x16 and QDR port per each GPU• 1450MFLOPS/W peak performance-perwattratio• Single SSD per each nodeAs of spring 2011, the Lomonosovsupercomputer is being expanded, usinghundreds of additional TB2-TL nodes.08TB2-XN Compute nodeDesigned to provide maximum density of 64sockets (384 cores) per 7U, TB2-XN is theindustry leader in computing density for x86systems. Each XN compute module containstwo 2 dual-socket compute nodes on a singleboard, using custom-designed DDR3 mezzaninememory modules.The TB2-XN system is the foundation for theLomonosov supercomputer deployed at MoscowState University. Initially number 12 on theTOP500 supercomputer list, it is an extremelydense installation for capability computing,containing 4,446 nodes with 35,778 processorcores, and consuming only 1.36 MW underLINPACK tests while being deployed in a modest252 square meter footprint. The installation isnow being upgraded with the heterogeneousTB2-TL blade systems.TB2-XN:• Each XN system board contains 2 DP Xeon56xx series nodes• 4.7 ТF peak system performance (DP)• Over 27 TFLOPS in 19” 42U rack.• Up to 24GB per DP node• Integrated QDR IB port per DP node• Diskless-only storage configurationto achieve highest computational densityT-Platforms | One-stop source for petascale computing

TB2W ConceptTB2W HighlightsTB2W is T-Platforms’ first foray into hot watercooling system design. As power savings andrenewable energy terms turn from mere buzz-wordsinto practical elements of computer system design,our engineers are looking for the ways to reducesystem and data-center level power consumptionby using hot water and a partial air-coolingcombination approach.TB2 enclosure was modified to accommodate asealed, pressurized water block with a highly reliableone inlet/one outlet water connector system.Both compute nodes feature hotplates – speciallydesigned flat heatsinks, tightly pressed against thewater block chamber rims to exchange the heateffectively.• 7U TB2 modified enclosure• Sealed +40°C temperature water block to cool down16 compute modules• Two fans remain to cool the 2 IB switches and management module• Accepts up to 16 TL nodes with 32 CPUs and 32GPUs• Integrated QDR IB switches and management modules• Up to 600W power reduction on system level compared to TB2• 25% estimated power reduction on the Datacenter level• Possibility of free cooling and hot water re-useUsing just standard hot water, installations canpotentially achieve 25-30% power savings on thedatacenter level, increase their PUE, introducefree-cooling, and abolish the use of water chillersin some climatic regions. This also can lead to ascenario where hot water is literally re-cycled, beingre-used to heat a datacenter building during coldwinter months.09TB2W prototypes pave the wave for our nextgeneration water-based T-Blade A-Class systems,due in 2013.

T-Blade V-СlassUsers with capacity, rather than capability needs,often look for small-to-medium sized clustersystems to meet high performance computingneeds in a variety of disciplines.Our energy-efficient scale-out type cluster systemsfeature industry standard technologies, and areflexible enough to create the right combination ofa balanced hardware and application environment.A 5U TB-V enclosure saves rack space and packsmore computational power by hosting ten 2-waycompute nodes with external IB or GbE ports. It is agreat alternative to the 1U or 2U twin server-basedclusters. With the same density of 2 standard nodesper RU, TB-V does not compromise on fan or PSUredundancy and brings node manageability throughthe in-chassis management switch.10T-Blade V-Class systems can feature differentCPU/GPU configurations, large memory pool andsufficient local storage of up to 2TB per node.Compute node design includes 2 blade form-factors.The thicker F double form factor allows for up to2 NVIDIA Tesla M20x0-series GPU per Intel-basednode, with up to 10 GPUs deployed in a singlechassis.TB-V Enclosure features:• 5U Cabinet• 10 single S blade bays / 5 fat F blade bays• Just three cooling modules• Three or Four 1.6kW PSU’s with n+1 redundancy• Integrated Ethernet-based node management/KVMT-Platforms | One-stop source for petascale computing

System featuresTB-V200S• Up to 10 DP compute nodes (type S)• 4-8-12-16 cores per Intel Xeon E52600-based node• Over 3Tflops of peak performancein a 10-node configuration• 16 DDR3 DIMM slots for upto 256GB of memory per node• Up to 2 SATA/SAS disks (2TB total)per node• 2 external GbE ports, one externaloptional FDR Infiniband/40GbE VPIport and one internal Fast Ethernetmanagement portTB-V200F• Up to 5 DP double width computenodes (type F)• 4-8-12-16 cores per Intel Xeon E52600-based node• Up to 10 NVIDIA Tesla M GPUs• Over 8Tflops of peak performancein a 5-node configuration (x86+GPU)• 16 DDR3 DIMM slots for upto 256GB of memory per node• Up to 2 SATA/SAS disks (2TB total)per node• 2 external GbE ports, one externaloptional FDR Infiniband /40GbE VPIport and one internal Fast Ethernetmanagement port11TB-V205S• Up to 10 DP compute nodes (type S)• 8-12-24-32 cores per AMD Opteron6200/6100-based node• Over 2.9Tflops of peak performance ina 10-node configuration• 16 DDR3 DIMM slots for upto 256GB of memory per node• Up to 2 SATA disks (2TB total) pernode• 2 external GbE ports, one externaloptional QDR Infiniband/10GbE VPIport and one internal Fast Ethernetmanagement portTB-V205F• Up to 5 DP compute nodes (type F)• 8-12-24-32 cores per AMD Opteron6200/6100-based node• Up to 5 NVIDIA Tesla M GPUs• Over 4.7Tflops of peak performance ina 10-node configuration• 16 DDR3 DIMM slots for upto 256GB of memory per node• Up to 2 SATA disks (2TB total) pernode• 2 external GbE ports, one externaloptional QDR Infiniband/10GbE VPIport and one internal Fast Ethernetmanagement port

T-Mini P-Class product familyAs more and more users need to get instantaccess to computational capacity without bookingand configuring capacity at large supercomputercenters, there is an increasing demand forpreconfigured parallel computers.Our upcoming T-Mini P-Class system is an 8compute node/1 management node parallelworkstation, designed for individual use and groupHPC collaboration in engineering, seismic analysis,physics, computational chemistry and other fields ofresearch and applied science.12T-Mini P can come preconfigured in Basic, Advancedand Super-Charged versions, featuring differentCPU, GPU, interconnect, memory and storageoptions. Default configurations are supplied withpreloaded MS Windows® HPC Server 2008 R2 OS,to facilitate easier deployment by the end user.T-Mini Highlights:• Pedestal or 8U rackmount enclosurewith instrumentation LCD• 8 bays for P200 and P205 compute nodes• 3U bay for persistent “fat” head node withup to 4PCI E slots• 2 bays for 20-port GbE Ethernet and/or20 QDR IB switches• Nodes connected to the switches via cablesat the back of system• Up to 2 HDD’s per compute module• Up 16-disk RAID on head node• Integrated management and KVM Switch• High-line and energy efficient PSU optionsT-Platforms | One-stop source for petascale computing

System ConfigurationT-Mini P205 Basic• Four P205 compute nodes, each coming with two AMD Opteron 61xx processorsand 32GB of DDR3, 500GB of SATA storage, 2 GbE ports• One P205M head node with 64GB of DDR3, 6TB SATA RAID, 2 GbE ports• GbE switch with two10GbE uplink portsT-Mini P205 Advanced• Eight P205 compute nodes, each coming with two AMD Opteron 61xx processors,64GB of DDR3, 500GB of SATA storage, 2 GbE & 1 IB QDR port• P205M head node with 128GB of DDR3, 10TB SATA RAID, 2 GbE and oneQDR Infiniband port• GbE switch with two 10GbE uplink ports• QDR InfiniBand switch13T-Mini P205 Super-Charged• Four P205 compute nodes, each coming with two AMD Opteron 61xx processors,64GB of DDR3, 500GB of SDD storage, 2GbE & 1 IB QDR port• 1 NVIDIA® Tesla C2050 / C2070 per node• P205M head node with 128GB of DDR3, 10TB SATA RAID, 500GB of SSD storageand one IB QDR port• GbE switch with two 10GbE uplink ports• QDR InfiniBand switch

Storage SystemsStorage is an integral part of all high performancecomputing systems, and an important element affectingoverall performance, reliability and availability ofcomputational clusters. T-Platforms offers storagesolutions for::■■High Performance Computing■■Additional infrastructure to provide14• Storage Consolidation• Backup• Archiving■■Near-HPC applications• Post production of movies• Video and audio surveillance• Cloud and internet services• Unstructured data storageT-Platforms is marketing the following solutions:■■T-Platforms branded systems – T-Store ActiveStore andT-Store SAN. These OEM systems are based on Panasasand NetApp E-Series technologies, supplied as a part ofour computational systems.■■Vendor systems: Panasas, DataDirect Networks, NetApp,Open-E. These products can be bundled with thecomputational systems or supplied stand-alone.T-Platforms | One-stop source for petascale computing

T-Store ActiveStoreT-Store SANThe T-Store ActiveStore system was designed asan integral part of high performance computationalclusters. Based on Linux, its core is an objectorientedarchitecture, combined with ActiveScale Filesystem to deliver linear scalability, performance andmanageability.T-Store ActiveStore features• Optimized for parallel computing• Easy-to-integrate into existing IT infrastructure• Simultaneous support of multiple clients• Applications: HPC, post-production, video-surveillance anddata-mining• Linear performance and capacity scalability• Throughput – from 600 MB/s to 150GB/s• From 8,000IOPS per chassis and up to 100,000’s IOPSper system• From 8 to 60TB of capacity per chassis and up to 6PBof capacity per system• User-friendly installation and management• Load balancing and resource aggregation• Module, chassis and network infrastructure redundancy• Snapshot supportAn open architecture-based T-Store SANis an optimal choice both for HPC systemsand Storage Area Networks (SAN). Variousperformance and capacity components can bea part of T-Store SAN, using FC, Infiniband andEthernet interfaces. This innovative solution isdifferentiated by the dense component designand fault tolerance of both modules and networkinterfaces. The centralized management systemprovides diagnostics, setup and management ofall components on-line, with the system running.T-Store SAN features• Mixed disk chassis configuration (SATA/FC/SAS/SSD)• Redundant interfaces with automatic loadbalancing improve system’s fault tolerance andguaranteed data availability• Unified management and diagnostics with intuitiveinterface• 2U-4U enclosures for high performanceconfigurations, based on 12-60 disks and forsystems with a larger capacity• A performance range of 2 to 6GB/s per diskenclosure15

PANASASAs a key element of highly effective high performance clusters, blade storage systems by Panasas minimize time-to-discovery for multi-stagemodeling, simulations and applications with extremely large data sets.Panasas features■■Maximal performance and scalability. Panasas bladearchitecture delivers virtually unlimited scalability though theaggregation of additional disk modules. System design allowsfor expanded capacity to 6PB, and the addition of largercapacity disks, once they become available. It is the leadingindustry system in terms of supported clients per gigabyteratio, with consolidated systems serving up to 12000 clients.Panasas product families• ActiveStor 8 is positioned for parallel input-output applicationsand large file operations in science and manufacturing• ActiveStor 11 and 12 families are differentiated from ActiveStor8 by increased throughput performance of up to 150GB/s. Thesesystems facilitate the most complex and resource-dependentapplications, running on the largest computing systems.16■■Panasas® PanFS parallel file system. PanFS providessimultaneous ‘performance with capacity’ growth. A directparallel data access under PanFS improves applicationperformance, eliminating issues of slow access for largenumbers of clients, a traditional NAS caveat.■■Data integrity guarantee. RAID technology is tightly coupledwith the Panasas file-system manage system. ScalableRAID functionality combines both operating system andRAID system management, leading to a higher economiceffectiveness and reliability of a system. A parallel recoveryprocess, driven by the system, minimizes the risk of serialdisk failures, proving the industry’s best recovery time. Paritycontrol implemented on blade-to-blade ‘horizontal’ level andon a separate disk ‘vertical’ level inside one enclosure, ensureread/write error recovery. Network parity technology helps toeliminate a ‘hidden’ data loss, reducing system load during theRAID-based recovery process.■■Manageability. All of the scalable Panasas NAS-systemscome with a GUI, which presents the data pool as a singleobject to manage. This helps to minimize administrationburden, related operational costs and TCO. The same systempresentation is also maintained when a system is expanded.Quick deployment and configuration tools let administratorsquickly expand the system adding new modules. Panasasstorage comes with redundant management modules, loadbalancing, power and cooling, automatically switching toreserved resources in case of component failure.T-Platforms | One-stop source for petascale computing

DataDirect NetworksT-Platforms is actively partnering with DataDirect Networks(DDN), a developer of storage systems with a greattrack record in HPC. Like Panasas, DDN is specializing inhigh level HPC storage systems, also expanding its cloudand NAS offerings. Two high performance hardwareplatforms and four software shells are the base elementsto create storage systems with various performanceand functionality. In this highly modular and scalablearchitecture, expanding the system with additional diskmodules is limited only by the parallel file system. Internalsystem architecture is utilizing SAS 6g interfaces, perfectto achieve low latency and high throughput.DDN solutions are successfully implemented in large andmedium-sized computational projects in the academic fieldand in applied science.Advantages of DDN■■Best in class scalability and aggregate performance:supporting up to 2500 nodes, with throughput of up to300GB/s■■The newest architecture of SFA12000 disk controllersprovides the performance level of up to 40 GB/s per asingle storage system with up to 1,400,000 sustainedIOPS, supports up to 16 GB of cache and hierarchalstorage management■■Open system architecture based on Lustre and GPFSparallel file systems■■High disk density of more than 2PB per 42U rack■■Up to 80% of available disk capacityHardware platforms:• SAF12000 – the most modern and performing architecture, developed for effectivehierarchal management of unstructured data to ensure effortless control of severalpetabytes of information. The data is stored on disks with optimal performance,prioritized by the type of data and common access pattern. To ensure parity controlon various levels, and to check and correct errors, redundant data is generated inthe shadow mode, without affecting overall application performance• S2A6620 offers a balanced throughput of 2GB/s and up to 335,000IOPS,complemented by the high density of 60 disks installed in a 4U module. DDN’sSATAssure technology helps to detect hidden data corruption while reading thedata, to recover it without affecting performance. Active-active controllers andredundancy of other components assure high system availability.Software shells:• NAS Scaler – a highly scalable multiprotocol NAS-solution, designed for the needsof quickly growing cloud service providers, for unstructured and computationalsystems data.• Protocols: NFS, CIFS, FTP, HTTP• xStreamScaler – multicomponent solution for diverse data content in a SANinfrastructure of Linux, Windows, UNIX and MacOS-based platforms, that allowssimultaneous work among SAN and NAS clients, and creation of hierarchal storage.Overall throughput of 50GB/s is sufficient for the support of over 1,000 clientnodes.• GridScaler – a ready solution for the HPC storage market, based on IBM’s GPFSwith a symmetric parallel file system combined with S2A and SFA12000 hardwareplatforms. GRIDScaler supports a wide variety of protocols for LAN, SAN and NASclients, including NFS, CIFS, Linux and Windows platforms, serving up to 2,000clients. GRIDScaler is seamlessly integrated into the leading data managementsystems, including Tivoli and HPSS.• Exascaler – the most scalable and productive data storage platform for HPCsolutions, based on the Lustre file system and S2A and SFA12000 OST modules.An open architecture scales to 20,000 client nodes, providing over 200GB/s ofthroughput and over 10PB of storage capacity.17

NetApp E-series18LSI Engenio was acquired by NetApp in 2011. LSI’s small andmedium sized disk systems are historically used by many computerand storage vendors under their own respective brands.The E-Series product lineup contains medium level storage systemswith various price points, ranging from the value E2600 to highperformance E7900 models. E-Series supports FC, IB, Ethernetinterfaces; its infrastructure can be upgraded by elements withdifferent levels of performance and capacity. Data protection isimplemented both on disk modules and network infrastructure levelsby implementing redundancy of controllers, power supplies and RAIDtechnology. A centralized management system provides diagnostics,setup and control of all system components without taking it offline.NetApp is recommending FastStak and DenseStak solutions, basedon E2600 model, developed for parallel file systems environmentsand for SAN infrastructures.Performance per rackFastStak80GB/s read/60GB/s writeDenseStak42 GB/s readCapacity per rack, PB 0.29 1.8Input/Output operations 800,000 400,000Disks 480 SAS 600 SASScalability, choice of additional unified disk enclosures,single management console, great price performanceratio, and high reliability on both module and networklevels, together with a balanced performance insequential and random access patterns, make thesesolutions a great choice for HPC systems, as well as forSAN infrastructure consolidation.E-Series features• Ability to install various disks (SATA/FC/SAS/SSD)in a single chassis• A wide range of head modules with unified 2U and 4Uchassis which contain 12-60 disks to create solutions withaccent on performance or on capacity• High availability is delivered through redundancy ofcontrollers; power supplies, fans and network connections,and also by a recovery process, automatically engaged aftera failure.• Scalable architecture allows adapting and upgradingexisting storage systems without taking systems offline.• Embedded Engenio Operating System (EOS) supports databackup and protection, including Snapshots, Volume Copyand Remote Mirrors.• SANtricity® Storage Manager Software, - a singlemanagement and diagnostics console with user-friendlyinterface reduces administration-related expensesDisk modules 2U/24/2.5” 4U/60/3.5’’Power per rack, W 8,000 9,500T-Platforms | One-stop source for petascale computing

Open-ЕA tried and proven Open E storage operatingsystem is used by T-Platforms to build storagesolutions for entry-level HPC systems.Open-E DSS V6 operating system provides thehighest performance, security and scalability, whileits cost is significantly below its alternatives.There is an extensive list of advanced functions,supported in Open-E DSS V6, including datafile and volume replication, volume snapshot,automatic failover for iSCSI volumes, Write OnceRead Many (WORM) support and NDMP support.Open-E DSS V6 also supports hardware RAIDcontroller models, provided by leading vendors.Finally, Open-E DSS V6 also allows users tocombine NAS systems with iSCSI SAN systems,and provides built-in virus protection, whileenabling the connection of multiple NetworkInterface Controllers, including Gigabit Ethernet,10 Gigabit Ethernet, Fibre Channel and InfiniBand.Open-E DSS V6 is easily downloaded to anyhardware device for simplified access.Open-E DSS V6 includes NAS and iSCSI functionalityand advanced features such as:• Installable on any media: internal HDD, RAID Array, USB, SATA and IDE DOM,or any other bootable media• Enhanced Graphical User Interface• Symantec Backup Exec backup agent (RALUS) to support versions 11, 12and 12.5• Basic API Configuration• Improved random performance of iSCSI• Windows Server 2008 Clustering Support (persistent reservation SCSI-3)• Hardware RAID Support• Automatic Failover for iSCSI Volumes• Volume and Data (File) Replication• Volume Snapshot• iSCSI Initiator / iSCSI Target• Backup Functionality• NDMP 3.0 Support• WORM (Write Once Read Many) Support• Shadow Copy of OS for System Update• Multiple Network Interface Card (NIC) Support• Network Bonding/Teaming• Task Mechanism with Scheduling• Network UPS Support• Fibre Channel Support• Multi-CPU Support• IP-SEC19

ClustrX HPC Software20The ClustrX product family is a tightlyintegrated cluster software stack for highperformancecomputing (HPC) that utilizesseveral novel approaches, both to serve currentcomputing needs and to scale smoothlyonto future systems. With ClustrX, an entiresupercomputing system is presented as a singlecomputational resource. ClustrX is a completeand scalable single code solution, providingnecessary services for system administratorsand end-users. Using ClustrX, an HPC clustercan be easily installed, configured and managedfrom a single point.ClustrX architecture is designed to providemaximum efficiency for the operation ofclusters of any size, paving the way toexascale computing. It represents the entireHPC system as a single computationalinfrastructure resource, with linear scalabilityachieved by dividing the cluster into twodistinct parts: a management network thatruns OS services and a computing network thatruns user jobs. No change to current systemarchitecture is required; the set of kernelsshipped with each installation can be adaptedto support required heterogeneous hardwareand software environments. And to provideadditional flexibility, the legacy HPC applicationsoftware, created for Red Hat Enterprise Linuxdistributions (including 32-bit packages), runs onClustrX without any modifications.T-Platforms | One-stop source for petascale computing

The ClustrX software family encompassesthe following sub-systems:■■ClustrX.TaskMng – Task and workload manager basedon SLURM.■■ClustrX.CNL – Light-weight Linux-based OS■■ClustrX.Watch – A hierarchical monitoring system thatgathers information from a computational node onhardware and OS health and status. A hierarchicaldesign enables scaling to thousands of nodes withoutperformance degradation. Built-in alarm capabilitiesgive a system administrator an unprecedented levelof flexibility to configure the alarm mechanism basedon the cluster configuration and administrationpolicy. Visualization allows observation of a clusterconfiguration and status of nodes in a convenient GUIinterface■■ClustrX.Safe – An automated emergency shutdownsystem, designed to protect and save hardwarecomponents in case of a critical electrical or thermalevent. Based on SMNP-protocol, ClustrX Safe workswith almost any UPS and Cooling systems, monitorsthe critical parameters and shuts down clusterhardware selectively.■■ClustrX.C-Management – Manages an HPC clusterfrom one integrated GUI. It handles software images,program compilation, cluster health, alarms, workloadmanagement, users, security, etc.ClustrX.TaskMng is an optimized andintegrated solution, based on widely-used SLURMsoftware. TaskMng is a feature-rich, powerful, yeteasy to use resource and task execution manager• TaskMng supports hardware barriers• GUI and Shell queue management• Partitioning configuration• Dynamic user management• Power management on idle• Accounting with LDAP• Hardware barrier supportClustrX.CNL is an ‘optimized for HPC’ Linux OSthat features:• Light-weight kernel stripped of services like WiFi andBluetooth• Support of Ethernet and InfiniBand• Parallel file system support: Lustre, PANFS• PXEboot support via Ethernet and Infiniband or froma local HDD• Syscall minimal latency• Optimized for HPC tasks memory management• TBL trash configured to minimize performancedegradation21

ClustrX.Watch controls an extensive set of hardware andsoftware resources that maintains power supply, cooling andnetwork infrastructure to ensure both the physical safety of thecluster’s devices and the successful completion of computingjobs. To permit the monitoring of their safety, compute nodesrun software agents.The agents are designed to minimize the processor timeconsumption. Some components of the system work with thesupport of SNMP (Simple Network Management Protocol)hence the agents are not needed there.ClustrX Watch is a monitoring system designed to support thepolling, logging and processing of data from 1.5M checkpoints(hardware sensors and software data sources such as CPUloads, kernel parameters, etc.) per second, per one monitoringserver (sufficient to monitor 10-12k of computing nodes).Monitoring servers can be scaled linearly by adding physicalservers to a monitoring farm. The monitoring system providesthe basis to create a close-to-real-time-clusters life supportsystem.Features:2222• Automated cluster deployment, using one console with singleinstallation DVD media• CLI- and GUI-based access• The management subsystem reacts automatically with preciseactions to correct abnormal operation, including automatedemergency equipment power down• Full support of heterogeneous architectures• Monitoring agents for all major Operating Systems with low CPUload• IPMI monitoring and management• SNMP monitoring• Cluster state visualization• 3rd party SW monitoring through open API• Adaptive schemes of task management and cluster controlsT-Platforms | One-stop source for petascale computing

ClustrX.Safe or AESS (Automated Notification/Equipment ShutdownSystem) provides high availability, safety and health of a highperformancecomputing cluster system on the data center engineeringinfrastructure layer.From the AESS standpoint, ClustrX Watch is just another source ofevents and its working logic can be applied to smaller parts such asthe system chassis or racks. However, AESS can also function as astandalone suite.AESS is primarily SNMP-based, polling sensors and dry contacts tocontrol the engineering infrastructure. It supports some water chillers, airconditioning units, UPS systems, rack cabinets and other SNMP-baseddevices with MIB (Management Information Base). AESS can provideflexible, scenario-based automated shutdown and restart routines for theentire cluster and parts of the data center infrastructure.AESS provides event monitoring and control for:■■Data-center-level power & cooling■■Ambient temperature, fire safety and air humidity■■HPC Center and rack-level access control■■AESS self-healthClustrX.C-Management – is a comprehensive management tool thatworks as a single resource for a system administrator, giving access toall ClustrX functionality:2323• Cluster configuration management• Users and group management and policies• Access management• Power management for nodes• Boot manager, including:• Image generation• Image installation• Boot options• License management• Compilation management• C-Management integrates all other sub-systems: TaskMng, Watch, Safe• C-Manager offers DNS, DHCP, LDAP, NTP, NFS and Lustre services support• Supports ClustrX.CNL, RHEL, SuSe, CentOS, Scientific Linux as nodes OS• iKVM support

HPC Services■■Supercomputer-based Modeling and Simulation■■Computing capacity lease■■HPC center management■■Access to specialized application software■■Resources for rendering■■Code parallelization24■■Turn key design of datacentersT-Platforms | One-stop source for petascale computing

Supercomputer-based Modeling andSimulation ServicesHPC Center ManagementT-Platforms provides supercomputer-based modeling andsimulation services, based on industry-common or in-housemethodology, providing customers with on-demand accessto compute resources and necessary application software.Today’s enterprises face a growing need for HighPerformance Computing, but few companies have thenecessary financial, technological and human resourcesto implement such solutions on their own. HPC servicesand supercomputer-based modeling can be provided byT-Platforms as turn-key solutions developed from scratchwith just minimal initial input from the client.Effective computational resources managementrequires having highly paid engineers and systemadministrators on staff. Supercomputer Data CenterManagement services enable customers to reduceoperational expense and support required for suchsystems.T-Platforms specialists provide all complexservice steps required to support heterogeneouscomputational and infrastructure environments.Computing Capacity LeaseAccess to specific application softwareFor companies with a temporary requirement for highperformance computing, or just needing additionalcomputing capacity, the optimal solution is to rent timeon a third party supercomputer.Having extensive knowhow in supercomputer modeling, owncompute capacity, HPC center management experienceand a team of applied science professionals, T-Platformsprovides the following:• Preliminary estimates of total computational time, costof required access and right-sized system configuration• Computational cluster lease• On-demand cluster configuration expansion• Reports on factual compute resource utilizationSupercomputer center resources of T-Platforms partnerscan also be provided as a part of the service.An optimal range of available on-demand software fornumeric modeling and virtual prototyping activities isavailable through T-Platforms:• ANSYS (ANSYS HPC, ANSYS Multiphysics & StructuralMechanics, ANSYS Fluid Dynamics, ANSYS Pre and Postprocessing,ANSYS CAD Integration, Electonic, PolymerProcessing & Glass Forming etc.)• ESI (ProCast and QuickCAST, Systus, Sysmagna, ACE +Suite, Weld Quality, Distortion Engineering)• CD-ADAPCO (STAR-CD/STAR-CCM+)Our professionals assist clients in the selection ofspecialized software and optional features, along withproviding application theory and practice workshopsto empower clients to better utilize their own HPCenvironments.In order to evaluate ANSYS features and applicabilityfor real-world problems, the company is providingcustomers with a 60-day demo trial ANSYS licenseupon completion of ANSYS courses.25

Rendering resourcesCode parallelizationT-Platforms is offering rendering services tocompanies specializing in 3D graphics andanimation. Large and medium sized productionstudios can lease a specific rendering farm toproduce 3D graphics on-site. Access to a 50-node dual-socket Intel Xeon quad-core renderingfarm, based on T-Blade 1.1 systems, is providedeither through secure internet protocol ordelivered fully configured and debugged to thecustomer’s premises.Many high performance computing systems utilize NVIDIA Teslaaccelerators, based on GF1x0 GPUs. TB2-TL, T-Blade V-Class andT-Mini P-Class systems support one or two accelerators in each node.Accelerators help to reach previously unattainable performance forapplications running in dual precision mode. This results in a reductionof node count, runtime, and OpEx related to cooling, power andservice. And yet not that many applications today support GPU-basedaccelerators to enjoy the savings.Typical GPU-based computing implementation issues are:26Available hardware• A single rack cabinet with 12-20 nodes based onthe six-core Intel Xeon 56xx series and AMD 12-coreOpteron 61xx series platform architecture, witha QDR Infiniband interconnect and an Ethernetbasednetwork, complemented by Panasas and LSIEngenio parallel storage systems.• 50-node dual-socket Intel Xeon quad-core renderingfarm, based on T-Blade 1.1 systems and theT-Store ActiveStore parallel file system with 12TB ofstorage or more for centralized data access.• Depending on the complexity and timing ofa specific project, remote access to largerconfigurations can be provided as well.• A substantial time is required for programmers to develop or adaptapplication code• Absence of developer experience for parallel computations• Inconsistencies between CUDA and OpenCL libraries, developed for varioushardware platforms• Performance optimization is limited by the adaption of softwarefor the specific hardware of the systemTo address these issues T-Platforms is offering services to portapplications to massively parallel heterogeneous platforms. Dependingon the system configuration, from 512 to 1024 compute cores ofNVIDIA GF1x0 are present on every node to help achieve a substantialincrease of available computing power on parallel codes.Services:■■Adapting applications for GPU architecture■■Optimization of CUDA programs■■MPU and OpenMP porting to NVIDIA CUDA■■CUDA code development for serial code■■Consulting support■■Support to transfer algorithms to GPU architecture■■Technical support on CUDA and OpenCL-related issues■■Assistance on existing code integration issues, associated with thenew technologies■■System utilities to simplify CUDA and OpenCL programmingT-Platforms | One-stop source for petascale computing

Turn-key design and construction of datacentersT-Platforms has 9 years of experience constructingand bringing very large data centers online.Our design process framework is defined by theEIA/TIA-942 standard, called to establish features andrelation of various data center subsystems for four typesof fault tolerance. Most T-Platform data center designsbelong to the Type III category in which fault tolerance ismaintained automatically, reaching 99.997% of availability.This translates into system downtime equal to one hour peryear. A Type III data center is the most applicable choicefor many HPC customers, both from the typical applicationload standpoint and budgetary considerations.T-Platforms is using two major cooling methods:■■Precision conditioning using Freon to transport the heatto external chillers, cooled by air■■Precision conditioning using water or a glycol solutionto transport the heatT-Platforms also provides strong expertise in hot waterbasedcooling, free cooling, and hot water re-use methodsto provide heating for facilities during the winter season.These methods lead to a significant increase in energyefficiency for most data centers.Turnkey data center deployment includes the followingstages:■■An analysis of customer’s existing infrastructure■■Customer consulting to generate datacenter requirements■■Preliminary layout design■■Development and coordination of technical parameters forall subsystems of a computational datacenter■■Subcontractors selection■■Project and documentation design■■Delivery and installation of equipment■■Test runs■■Certification and licensing■■Organization of staff training■■Monitoring and service department organization27

Appendix.Product Feature MatrixT-BLADE 2 FAMILYT-BLADE V-CLASS FAMILYT-MINI FAMILY28

T-Blade 2 FamilyTB2-XN System TB2-TL System TB2W-TL ConceptEnclosure Form FactorNode levelEnclosureLevel42U racklevelNode levelEnclosureLevel7RU, Fixed42U racklevelNode levelEnclosureLevelNumber of compute nodes (max) N/A 32 192 N/A 16 96 N/A 16 96CPU type Intel Xeon X56xx, 6-core, 95W TDP Intel Xeon L56xx, 4-core, 40W TDPx86 Sockets 2 64 384 2 32 192 2 32 192CPU Cores, max 12 384 2304 8 128 768 8 128 768GPU type not supported NVIDIA TESLA X2070 SXM with 6GB of GDDR5GPU cards #, max N/A 2 32 192 2 32 192Linpack Peak Performance,GFLOPS, x86 DPLinpack Peak Performance,GFLOPS, hybrid (x86+GPU), DP141.6 4531 27187 N/A N/A42U racklevelN/A 1098 17570 105420 1098 17570 105420Performance-per-Watt, peak,x86-only0.41GFLOPS/W N/A N/APerformance-per-Watt, peak,hybrid (x86+GPU)N/A 1.46GFLOPS/W 1.53 GFLOPS/WMemory typeCustom 12GB DDR3 ECC mezzanine module, 3-channelMemory slots 2 32 192 2 32 192Total memory 24GB 768GB 4.6TB 24GB 384GB 2.3TB 24GB 384GB 2.3TBMemory per core 2GB 3GBStorage type Centralized-only Local SSD 80GB (optional)Disks N/A One x 1.8”Total available storage N/A 80GB 1.28TB 7.68TB 1x 80GB 1.28TB 7.68TBChipset One Intel 5500; One ICH10 Two Intel 5520; one ICH10PCI E type PCIe Gen. 2PCI E Expansion slotsNot supportedEthenet PortsOne 100Mb management/One GbE auxilary network per node & two 10GbE stacking ports per enclosureIB Ports 1, QDR 36, QDR 216, QDR 2,QDR 36, QDR 216, QDR 2, QDR 36, QDR 216, QDRIntegrated Ethernet SwitchOne, part of the Management module with two 10GbE stacking portsIntegrated IB SwitchTwo, 36-port, non-blockedIntegrated System ManagementYes, plus iKVM & remote mediaHPC-specific networksFPGA-controlled Global Interrupt and Global Barrier networksCooling type (per enclosure) Front-to-back air cooling, 12 HS 14000RPM fans per enclosure, N+140°C water cooling for compute nodesand two HS 14000RPM fans for switchesPSU Type (per enclosure)Six 2725W PSU’s, HS, N+1 RedundancyPeak power consumption, kW N/A 11 66 N/A 12 72 N/A 11.5 69Electrical Input380VAC, 5-wire, 3-phase, 32A socket19” Rack Cabinet Support (EiA310-D or later)Yes, 1000mm or deeperModified cabinet to supply the water29

T-Blade V-Class FamilyV200S-based systemV200F-based systemNode level System level 42U rack level Node level System level 42U rack level30Enclosure formfactor5RU, fixed railsCompute module type Standard (thin) Double widthCompute module count (max.) NA 10 80 NA 5 40Supported CPU type (max.)Intel® Xeon® E5 2670, 2.6GHz, 8 core/16 threads, TDP 115Wx86 socket/core count 2/16 20/160 160/1280 2/16 10/80 80/640Accelerator type Not supported NVIDIA® Tesla M-seriesAccelerator count (max.) Not supported 2 10 80Peak performance, GFLOPS, DP 294.4 2944 23552 1623 8117 64896Peak performance per watt 0.79 GFLOPS/W, preliminary 1.73 GFLOPS/W, preliminarySupported memory typeReg ECC DDR3 DIMM, 4 channels, 1066, 1333, 1600MHzAmount of DIMM slots 16 160 1280 16 80 640System memory size (max.) 256GB 2.56TB 20.48TB 256GB 1.28TB 10.24TBSystem memory size per core (max.)Storage system16GB (for 8-core CPU-based configurations)Local (on-node), cold swap, SAS 6Gb/s and SATA 6Gb/s controller, 2.5” HDDor SSD (SAS 3Gb/s and SATA 3Gb/s devices are backwards compatible)Amount of disks (max.) 2 20 160 2 10 80Maximum storage space Up to 2TB Up to 20TB Up to 160TB Up to 2TB Up to 10TB Up to 80TBCompute module chipsetPCI Express typePCI E x16 slot countEthernet ports, node levelOptional FDR InfiniBand ports, nodelevelIntegrated Ethernet switchIntegrated management subsystemChassis cooling subsystemPower supplies1 (LP MD2,no GPU support)10 (LP MD2,no GPU support)One Intel® C600 (Patsburg PCH)On-die controller, PCIe Gen.380 (LP MD2,no GPU support)2(GPU support only)10(GPU supportonly)Two external GbE ports (Intel®i350), optional 40GbE support via QSFP port (Mellanox ConnectX® 3 VPI®)80(GPU supportonly)1 10 80 1 5 40Fast Ethernet switch for management purposes onlySystem’s Management Controller with Fast Ethernet switch and two external GbE ports; iKVM and remote media supportFront-to-back air cooling, 3 dual-rotor, hot swap cooling modules with N+1 redundancyThree or four 1600W hot swap PSUs with N+1 redundancyPeak power consumption N/A 4200 33600 N/A 3300 26400Electrical input1) 208-230 VAC, 50-60Hz, 4 х 8А, 1 or 3 phase, IEC С19/C202) Some configurations also support 110-120 VAC, 50-60Hz, 4 х 16A, 1 or 3 phase, IEC C19/C20Weight, without cabling TBD TBD TBD TBD TBD TBD19" rack support EiA 310-D or newer standard compliant rack cabinets, with a minimal depth of 1070mmT-Platforms | One-stop source for petascale computing

V205S-based systemV205F-based systemNode level System level 42U rack level Node level System level 42U rack levelEnclosure formfactor5RU, fixed railsCompute module type Standard (thin) Double widthCompute module count (max.) NA 10 80 NA 5 40Supported CPU type (max.)AMD Opteron 6276, 2.3GHz, 80W, 16 core/16 threads, TDP 115Wx86 socket/core count 2/32 20/320 160/2560 2/32 10/160 80/1280Accelerator type Not supported NVIDIA® Tesla M-seriesAccelerator count (max.) Not supported 1 5 40Peak performance, GFLOPS, DP 294.4 2944 23552 959 4795 38360Peak performance per watt 0.73 GFLOPS/W, preliminary 1.45 GFLOPS/W, preliminarySupported memory typeReg ECC DDR3 DIMM, 4 channels, 1066, 1333, 1600MHzAmount of DIMM slots 16 160 1280 16 80 640System memory size (max.) 256GB 2.56TB 20.48TB 256GB 1.28TB 10.24TBSystem memory size per core (max.)8GB (for 16-core CPU-based configurations)Storage systemLocal (on-node), cold swap, SATA 3Gb/s controller, 2.5” HDD or SSDAmount of disks (max.) 2 20 160 2 10 80Maximum storage space Up to 2TB Up to 20TB Up to 160TB Up to 2TB Up to 10TB Up to 80TBCompute module chipsetOne AMD SR5670; one AMD SP5100PCI Express type PCIe Gen.2PCI E x16 slot countEthernet ports, node levelOptional QDR InfiniBand ports, nodelevelIntegrated Ethernet switchIntegrated management subsystemChassis cooling subsystemPower supplies1 (LP MD2,no GPU support)10 (LP MD2,no GPU support)80 (LP MD2,no GPU support)1(GPU support only)5(GPU supportonly)Two external GbE ports (Intel® 82580DP), optional 10GbE support via QSFP port (Mellanox ConnectX® 2 VPI®)40(GPU supportonly)1 10 80 1 5 40Fast Ethernet switch for management purposes onlySystem’s Management Controller with Fast Ethernet switch and two external GbE ports; iKVM and remote media supportFront-to-back air cooling, 3 dual-rotor, hot swap cooling modules with N+1 redundancyThree or four 1600W hot swap PSUs with N+1 redundancyPeak power consumption N/A 4200 33600 N/A 3300 26400Electrical input1) 208-230 VAC, 50-60Hz, 4 х 8А, 1 or 3 phase, IEC С19/C202) Some configurations also support 110-120 VAC, 50-60Hz, 4 х 16A, 1 or 3 phase, IEC C19/C20Weight, without cabling TBD TBD TBD TBD TBD TBD19" rack support EiA 310-D or newer standard compliant rack cabinets, with a minimal depth of 1070mm31

T-Mini familyT-Mini system configuration,based on P205S nodesT-Mini system configuration,based on P205F nodesEnclosure DescriptionConfiguration Type Deskside Cluster Deskside Cluster / GPU WorkstationEnclosure TypePedestal / Rackmount convertible (option) chassis with integrated system monitoring LCDMax. Compute Nodes 8 dual-socket (8 trays) 4 dual-socket (4 trays)Integrated NetworkingHDD BayCooling ArchitecturePower Architecture (High line / Low Line)Input Power• Option 1: Single Gigabit Ethernet switch with 20 GigE and 2 x 10GigE ports and a single 20-portInfiniBand QDR switch• Option 2: Two ETH switches, providing 40 Gigabit Ethernet ports and four 10GigE ports in total• Option 3: Single Gigabit Ethernet switch with 20 GigE and 2 x 10GigE portsFront bay with up to 16 x 2.5” HDDs or SSDs, with HotPlug option, up to 1TB for each diskN+1 redundant cooling fans in front of enclosure. Front-to-back airflow.Rear bays with two 2725W/1200W modules, N+1, 2N redundant / N+1 redundant90–264VAC (47–63Hz)Dimensions 865 x 356 x 500 (34,0” x 14,0” x 17,5”)32Net Weight TBA TBAOperational Noise Level* TBA TBASustained Power Consumption TBA TBAPersistent Head Node DescriptionMax. ProcessorsChipsetMax. CoresMax. MemoryTwo AMD Opteron 12-core 6100 series or two AMD Opteron 16-core 6200 series (115W TDP MAX)One AMD SR5670 + One SP510032 (when equipped with AMD Opteron 6200 series)128 GB (when configured with MS Windows HPC Server 2008 R2) or up to 256GB, with Windows® Server 2008 R2Enterprise Edition or higherMemory Slots & Type 16 x 1333/1066 MHz DDR3 ECC reg.Supported Graphics GPU CardsSupported Compute GPU CardsSP5100-based RAIDExpansion SlotsOptional LSI HBA RAIDNetworking (Onboard)IPMI Remote ManagementPrimary Operating SystemNVIDIA® Quadro® FX / AMD FireProNVIDIA® Tesla C2050 / C2075 (if used as a workstation)SATA SW RAID 0,1, 10, up to 6 cabled disksUp to 5 PCI-E 2.0 w x16, x8 and x4 signalingSATA/SAS HW RAID 0,1,10,5,50,6,60, w optional BBU; configured for head node, compute nodes or bothIntegrated Dual GigE (Intel 82574) RJ45 ports, optional single QSFP port QDR Mellanox InfiniBand adapterIntegrated IPMI 2.0 + Keyboard Video Mouse (iKVM)MS Windows® HPC Server 2008 R2 or higherT-Platforms | One-stop source for petascale computing

T-Mini system configuration,based on P205S nodesT-Mini system configuration,based on P205F nodesCompute Node DescriptionCompute Node Model P205S P205FCompute Node Form Factor Standard width Double widthMax. ProcessorsTwo AMD Opteron 12-core 6100 series or two AMD Opteron 16-core 6200 series (115W TDP MAX)ChipsetMax. CoresMax. MemoryMemory Slots & TypeNode Hard Disk Drives & Max. Capacity 2 x fixed 2.5” SATA II HDD or SSD drives, up to 1TB each 4 x fixed 2.5" SATA II HDD or SSD drives, up to 1TB eachSP5100-based RAID SATA SW RAID 0,1, 10Expansion SlotsN/ASupported Graphics GPU Cards N/A NVIDIA® Quadro® FX / AMD FirePro (optional)Supported Compute GPU Cards N/A NVIDIA® Tesla C2050 / C2075 (1/node)Networking (Onboard)Dual GigE (Intel® 82574) RJ45 ports, optional single QDR Mellanox® InfiniBand QSFP port33IPMI Remote ManagementPrimary Operating SystemIntegrated IPMI 2.0 + Keyboard Video Mouse (KVM)MS Windows® HPC Server 2008 R2* Sound pressure level depends of CPU wattage. 95W/115W TDP- moderate / normal for Head/S/F nodes, 65W/80W TDP - normal / low for Head/S/F nodes.

© T-Platforms, 2012

T-PlatformsMoscow, RussiaLeninsky Prospect 113/1 Suite B-705Tel.: +7 (495) 956 54 90Fax: +7 (495) 956 54 15info@t-platforms.comtPlatforms GmbHWoehlerstrasse 42, D-30163, Hannover, GermanyTel.: +49 (511) 203 885 40Fax.: +49 (511) 203 885 41T-Platforms, T-Platforms logo, T-Blade, ClustrX TP edition are trademarks or registered trademarks of T-Platforms, JSC. Other brand names andtrademarks are property of their respective owners. This document is for informational purposes only. T-Platforms reserves the right to make changeswithout further notice to any products herein. The content provided is as is and without express or implied warranties of any kind.© 2012