2010 Best Practices Competition IT & Informatics HPC

More documents

Recommendations

Info

Bio‐IT World 2010 Best Practices Awards Nominating Organization name: DataDirect Networks, Inc. Nominating Organization address: 9351 Deering Avenue Nominating Organization city: Chatsworth Nominating Organization state: CA Nominating Organization zip: 91311 Nominating Contact Person: Jeffrey Denworth Nominating Contact Person Title: VP, Marketing Nominating Contact Person Phone: 1‐856‐383‐8849 Nominating Contact Person Email: jeffdenworth@hotmail.com User Organization name: Cornell University Center for Advanced Computing User Organization address: 512 Frank H. T. Rhodes Hall User Organization city: Ithaca User Organization state: NY User Organization zip: 14853 User Organization Contact Person: David A. Lifka, PhD User Organization Contact Person Title: Director, Cornell University Center for Advanced Computing User Organization Contact Person Phone: 607‐254‐8621 User Organization Contact Person Email: lifka@cac.cornell.edu Project Title: Scalable Research Storage Archive Team Leaders name: Team Leaders title: Team Leaders Company: Team Leaders Contact Info: Team Members name: Dr. Jaroslaw Pillardy Team Members title: Sr. Researcher at Cornell’s Computational Biology Service Unit Team Members Company: Cornell University Entry Category: IT & Informatics Abstract Summary: Introduction: The Cornell Center for Advanced Computing (CAC) is a leader in high‐performance computing system, application, and data solutions that enable research success. As an early technology adopter and rapid prototyper, CAC helps researchers accelerate scientific discovery. Located on the Ithaca, New York campus of Cornell University, CAC serves faculty and industry researchers from dozens of disciplines, including biology, behavioral and social sciences, computer science, engineering, geosciences, mathematics, physical sciences, and business. The center operates Linux, Windows, and Mac‐based HPC clusters and the staff provides expertise in HPC systems and storage; application porting, tuning, and optimization; computer programming; database systems; data analysis and workflow management; Web portal design, and visualization.
CAC network connectivity includes the national NSF TeraGrid and New York State Grid. The DataDirect Networks S2A9700 storage system is used as the central storage platform for a number of departments and applications. Initially deployed for backup and archival storage, CAC is increasingly using the S2A9700 as front‐line storage for applications such as genome sequencing. Since CAC provides services to a wide range of Cornell departments and applications, implementing centralized storage platforms is critical in ensuring an efficient, reliable and cost‐effective infrastructure. Cornell researchers were considering buying commodity, off‐the‐shelf storage solutions to locally store their research data. While the cost of such technology appeared initially low – the lack of coordination, data protection and system reliability detracted from the long‐term value of this approach. As research productivity and access to data are directly correlated – the primary focus of the storage solution had to be high reliability and scalability. It was clear that an affordable, centrally managed, highly available research storage system was needed in order to control costs and also to ensure that researchers remained productive. Accommodating a variety of applications and departments would prove a challenge for ordinary storage systems, but the DDN S2A9700 proved capable even beyond the initial scope of the project. Results: The center selected an S2A9700 storage system from DDN with 40TB unformatted capacity in RAID‐ 6 configurations. DDN partnered with Ocarina Networks to provide transparent, content‐aware storage optimization at CAC, reducing the overall capacity need by more than 50 percent. For some Microsoft SQL database applications, a compression rate of up to 82 percent was achieved. DDN storage technology enables massive scalability and capacity optimization through storage collaboration. As compared to other storage technologies in it's class ‐ the S2A9700 features industry leading throughput (at over 2.5GB/s per system), capacity (scalable to hold up to 2.4 Petabytes in a single system) and data center efficiency (DDN systems are the densest in the industry, housing up to 600 hard drives in a single data center rack ‐ also featuring Dynamic MAID power management technology). The combination of the S2A9700 system scale and the data center optimized configuration proved to Cornell that installing and adding capacity could be done very cost‐effectively and the system could scale to meet the Center's evolving storage volume requirements without a forklift upgrade. "We have been very impressed with the performance DDN's S2A9700 delivers," said David A. Lifka, CAC director. "For genomics research ‐ Cornell uses Solexa Sequencers and the DDN storage system is directly connected to the compute cluster, while at the same time continuing to provide backup and archive storage for our other projects and departments." ‐ David A. Lifka, CAC Director Ocarina’s ECOsystem platform uses an innovative approach to data reduction. The ECOsystem first extracts files into raw binary data and applies object boundaries
Page 1 and 2: 2010 Best Practices Competition IT
Page 3 and 4: B. INTRODUCTION/background/objectiv
Page 5 and 6: Published Resources for the Life Sc
Page 21: Results: Purdue selected the free,
Page 29 and 30: Bio‐IT World 2010 Best Practices
Page 43 and 44: 1. User Organization (Organization
Page 45 and 46: Figure 1 Data processing Pipeline (
Page 47 and 48: the cluster grind to a halt. The ke
Page 49 and 50: Figure 2 Saguaro2 supercomputer In
Page 51 and 52: [ROI Expected or Achieved] Figure 4

2010 Best Practices Competition IT & Informatics HPC

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?