Anke Kamrath - CESM - UCAR

More documents

Recommendations

Info

Using less memory to get same science done Total memory usage for CLM 50x 18
Improving Data Workflow and Management • Today: – We are seeing many challenges with Data Analysis of CMIP5 data • Atomic NCO commands very inefficient to transform the raw <strong>CESM</strong> output into time series. Prevents use of buffering • NCAR GLADE GPFS 2MB Blocks, NETCDF3 10-150kB used – very inefficient. 10-20X too much data being moved. • Overall workflow complex and inefficient, and lacking automation • Work ahead: – Need improvements on all fronts • More efficient data management, scripts, • Thinking carefully about what you store, what should be done during the simulation phase rather than after the fact • Tuning of systems to better support workload • Much more… – Nearly unworkable “today” – won’t survive at all in future. 19
Page 1 and 2: NCAR’s Data-Intensive Supercomput
Page 3 and 4: Yellowstone Supercomputing Environm
Page 5 and 6: GLADE (GLobally Accessible Data Env
Page 7 and 8: Yellowstone Storage - Disk & Tape
Page 9 and 10: Yellowstone Power Efficiency vs Blu
Page 11 and 12: NWSC Installation 11
Page 13 and 14: Yellowstone Environment Lifetime Co
Page 15 and 16: Historical FLOP/Power Efficiency on
Page 17: Inefficient Use of Memory • Repli
Page 21 and 22: Questions 21

Anke Kamrath - CESM - UCAR

Create successful ePaper yourself

Delete template?

Save as template?