2010 Best Practices Competition IT & Informatics HPC

More documents

Recommendations

Info

location and retention of key data. Specifically, IT took the following steps to improve the data management process and accelerate the scientific workflow: • Dedicated NFS storage for raw reads, attached to back-up tape library • Dedicated NFS storage for all results, attached to back-up tape library • Automated backup process for “key” files • User education on how to mount/unmount the storage space • Configured Aspera server to read directly from designated NFS mount points eliminating unnecessary data moves • Weekly cron jobs for monitoring and informing users about storage resource capacity • Automated monitoring of user jobs utilizing the HPC Cluster • Established a SharePoint based web portal to share NextGen project related information These changes had to be synchronized and communicated across multiple scientific divisions as well as the within the IT department. The end result was a more streamlined scientific workflow, improved data management environment and reduced impact on the storage, backup and network infrastructure. Lesson, be flexible in regards to data management procedures and the supporting infrastructure. Rapidly advancing technologies such as NextGen sequencing can render your current methods obsolete and you must be willing to make dramatic changes in response to the needs of the scientific community and the demands of the technology. Benchmarking: Alignment of billions of reads to reference genomes is computationally expensive. An effort was initiated to benchmark sequence alignment tools. TGen’s IT team was actively involved in this process by providing several performance measurement and tuning tools and creating automated scripts for collecting data about computing resource utilization associated with six popular sequence alignment programs. IT used performance measurement tools for cluster computing environments to benchmark the speed, CPU utilization and input-output bandwidth needed for the program. This information is now being used for selecting the best tool for various projects and planning the resource requirements for future NextGen sequencing projects. Lesson, time spent benchmarking can provide significant benefit in terms of reducing the cost and effort associated with the “trial & error” approach to selecting and using complex technology such sequencing alignment tools. [Results] Key Technologies & Supporting Methodologies The TGen High Performance Bio-Computing Center (HPBC) manages a diverse collection of HPC systems, storage and networking resources, including two large supercomputers. The first supercomputer is called Saguaro2, and is a Dell Linux cluster. This system consists of ~4000 Intel x86-64 processor cores, with 2 GB RAM per core. This system has a shared parallel 250 TB (Lustre) file system that allows massive amounts of concurrent input/output operations spread across many compute nodes. This system is very effective at running thousands of concurrent discrete processing jobs, or at running very large parallel processing workloads. This large HPC cluster system is installed at the Arizona State University campus in the Fulton High Performance Computing Initiative (HPCI) center and was funded via NIH grant S10 RR25056-01.
Figure 2 Saguaro2 supercomputer In addition to the Saguaro2 cluster system, TGen also has a large memory Symmetric Multi-Processor (SMP) system available. This system, is an SGI Altix 4700 consisting of 48 Intel IA-64 cores and 576 GB of globally shared memory. The SGI system is well suited for solving memory intensive problems, or algorithms that are not easily parallelized. With the resources available on this system, it can run several concurrent memory intensive jobs, without having a performance penalty inflicted due to the architecture of both the processors and the I/O backplanes on this system. This system was funded via NIH Grant S10 RR023390-01. Updated NextGen sequencing workflow: Learning from the experience and systematically identifying the resource requirements at various stages of the NextGen data analysis and transfer, TGen developed and installed a significantly improved NextGen sequencing data processing pipeline (Figures 3 & 4). The updated data processing pipeline utilizes several customized scripts tailored to the software implementation underlying various data analysis tools have been developed, which improve the effectiveness of using HPC for analyses. By indentifying the critical files at various stages, redundancy of storage has been minimized and policies have been established to delete intermediate files automatically after fixed time. Several compute systems have been dedicated to local data processing, such as annotation and parsing. Involving PIs in the infrastructure design process and educating their research staff has helped significantly in creating a team of proficient and more mindful users of the data processing pipeline.
Page 1 and 2: 2010 Best Practices Competition IT
Page 3 and 4: B. INTRODUCTION/background/objectiv
Page 5 and 6: Published Resources for the Life Sc
Page 21 and 22: Results: Purdue selected the free,
Page 23 and 24: CAC network connectivity includes t
Page 29 and 30: Bio‐IT World 2010 Best Practices
Page 43 and 44: 1. User Organization (Organization
Page 45 and 46: Figure 1 Data processing Pipeline (
Page 47: the cluster grind to a halt. The ke
Page 51 and 52: [ROI Expected or Achieved] Figure 4

2010 Best Practices Competition IT & Informatics HPC

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?