TGQR 2010Q2 Report.pdf - Teragridforum.org

More documents

Recommendations

Info

display aggregate information geographically with the ability to drill down to more detailed information as well as aggregate information such as “Job Distribution by Organization.” In our continuing exploration of the data contained in the TeraGrid Central Database (TGCdb), queries are being developed to feed the initial visualizations for the User and Program Officer roles. To date, functionality has been created for converting query results to JSON data structures and using this data to generate charts, images and maps using the various Google Chart APIs. We are addressing performance issues when querying the larger TGCdb tables (i.e., acct.jobs) and are examining ways to mitigate these issues and improve performance in a production system. Several sample data visualizations have been created including reports on usage, CPU consumption vs. Job Size, total jobs run on each resource, and allocation reports. A set of classes encapsulating the functionality for dynamically generating the queries is being built. We have begun to develop an in-house database to hold data ingested from various data sources (initially TGCdb) and support the construction of visualizations via the portal. Scripts have been written to ingest data for a targeted subset of TeraGrid users that will be used for testing purposes. Various methods for drawing charts have been investigated for maximum cross-browser compatibility. We have made contact with Rob Light (PSC) regarding mirroring of the TGCdb. Rob will be assisting us in identifying the tools required to mirror a copy of the TGCdb locally at CCR. • TeraGrid Central Database (TGCdb) Findings/Limitations: In general, the TGCdb records are very good, however in the course of constructing early versions of XDMoD we discovered that there are some areas where data is apparently missing and other areas where there is room for added or improved record storage. Our initial assessment of the TGCdb finds some inconsistency in batch job usage reporting. Two notable examples are that job exit status is not reported and not all computational resources report usage by processor core as well as by node. Indeed, reporting is not always consistent for jobs on an individual resource. We anticipate recommending a uniform taxonomy and more extensive collection of resource utilization statistics. • Monitoring Framework: The Inca User-Level Grid Monitoring tool has been installed and configured locally at CCR and is currently deployed on our 2112 node compute cluster. Efforts are underway to develop a number of example reporter modules that can be used as a basis for deploying custom Application Kernels within Inca. A welldocumented process will be provided for other team members. Exploration of the structure of the Inca Depot is underway with the assistance of Shava Smallen (SDSC). Queries to extract detailed reporter information from the Depot and display it are being developed. A web-based exploration tool for the Inca data depot is being built. This will allow the team to quickly explore the results of the custom application kernels that are being developed. • Application Kernel Development: Initial application kernel requirements have been documented and the kernel classification scheme aligned with the “Collella dwarfs” that describe application processing and data requirements. Deployment will leverage existing Inca reporting infrastructure, and the first (dense matrix) kernel and reporter is ready for testing. Two micro benchmarks: STREAM, which measures processor to memory bandwidth, and BLAS (DGEMM), which measures single-node floating pointing performance in the context of a dense matrix problem, are now running within the local Inca testbed. We will coordinate and collaborate with FutureGrid staff and Inca 104
developers to see what they have already deployed on FutureGrid and leverage their existing codebase for early and easily adopted application kernels. Inca’s current cluster batch wrapper needs to be augmented to facilitate running kernels of varying size (especially important for lightweight versions of the various application kernels), and we will need to develop a meta-scheduling scheme to manage the flow of kernel jobs through the various batch queuing systems in TG/XD (prevent accumulation of jobs, duplication, etc.). • Miscellaneous: TeraGrid accounts have been requested and received for two developers (Charng-Da Lu and Steven Gallo) under the Network/Operations/Security (NOS) allocation by Shava Smallen and Jeff Koerner. These accounts will be used for Inca Monitoring and Application Kernel development and testing. Additionally, a subversion repository has been set up for TAS software management. • Hardware Ordered for XDMoD development: Hardware was purchased to host the development and production environments for XDMoD including a mirror of the TGCdb database. The hardware includes two Dell R710 servers and an Equilogic PS6010XV high performance storage array with 9.6TB of raw storage. • University of Indiana Subcontract: IU tasks include: (a) Identify the current available TG monitoring and resource management service, for example, access interface (API, command line, web service). (b) Assist in the development of the XDMoD (XD Metrics on Demand) portal. (c) Collaboratively design and develop a Custom <strong>Report</strong> Builder that can be incorporated in the XDMoD portal, allowing users to easily generate custom reports in several formats, including PDF and excel. (d) Participate in the development and deployment of application test kernels. (e) Interface with the Futuregrid and TIS (Technology Insertion Service) projects. • University of Michigan Subcontract: The University of Michigan subcontract was finalized. • The TAS Technical Advisory Committee: This committee will meet regularly to provide feedback and guidance on the TAS program. The current committee members are listed below. We plan on adding a representative from the XD resource providers once the XD award has been established. The first meeting will tentatively be held in November 2010. TAS Technical Advisory Committee: 1. Dr. Jerzy Bernholc (Atomistic QM, Physics, NC State) Email: bernholc@ncsu.edu; http://chips.ncsu.edu/~bernholc/ 2. Dr. P.K. Yeung (CFD, Ge<strong>org</strong>ia Tech) Email: pk.yeung@ae.gatech.edu; http://soliton.ae.gatech.edu/people/pyeung/index.html 105
Page 1 and 2:
NSF Extensible Terascale Facility T
Page 3 and 4:
Working Group Leaders Accounting Ad
Page 5 and 6:
6.7 RP Operations: User Facing Proj
Page 7 and 8:
1 Overview The TeraGrid is an open
Page 9 and 10:
Further details can be found §6.2.
Page 11 and 12:
2.1.10 2.1.11 2.1.12 2.1.13 2.1.14
Page 13 and 14:
Figure 2-3. Illustration of the thr
Page 15 and 16:
mapped out pathways from the bisabo
Page 17 and 18:
The Center for Analysis and Predict
Page 19 and 20:
density distribution. ADF/cofilin b
Page 21 and 22:
Ranger at TACC, they simulated the
Page 23 and 24:
Presentations at gateway telecons f
Page 25 and 26:
USGS National Map infrastructure an
Page 27 and 28:
Figure 4-4 Effect of adaptive optic
Page 29 and 30:
4.2.0/. The NCSA team will support
Page 31 and 32:
significantly improve the user expe
Page 33 and 34:
petascale applications and tools, a
Page 35 and 36:
Srinivasa Jampani, University Rao D
Page 37 and 38:
efforts: Advanced Support for TeraG
Page 39 and 40:
CMU Global Kinetic Simulations of t
Page 41 and 42:
Heat Transfer Simulation of Suspens
Page 43 and 44:
VanBriesen, CMU Villous Motility as
Page 45 and 46:
• Subversion source management ha
Page 47 and 48:
may be necessary to use a different
Page 49 and 50:
Prof. Cheatham’s ptraj MD analysi
Page 51 and 52:
PI: Choi (U. Nevada, Las Vegas, Ast
Page 53 and 54: Amanda Hughes, a PhD student of Dr.
Page 55 and 56: Yang Wang (PSC). Initial contact wa
Page 57 and 58: community. User feedback received,
Page 59 and 60: 6 User Facing Projects and Core Ser
Page 61 and 62: Table 6-1. Q2 2010 Most Visited TGU
Page 63 and 64: “replacement” model, in which t
Page 65 and 66: The TGUP responded to 106,097 portl
Page 67 and 68: Most new documentation came as a di
Page 69 and 70: 6.8 Information Production and Qual
Page 71 and 72: initial testing of NFS4 as a gatewa
Page 73 and 74: (pnetCDF and netCDF4) was completed
Page 75 and 76: provide a path to becoming Longhorn
Page 77 and 78: Figure 8-2: Total transfer (GB) per
Page 79 and 80: 8.5.1 Data Movement (GridFTP) Stati
Page 81 and 82: Figure 8-5: Total number of GridFTP
Page 83 and 84: 8.5.2 Job Submission (GRAM) Statist
Page 85 and 86: Figure 8-9: GRAM-submitted jobs as
Page 87 and 88: As part of the NIGMS award, NRBSC a
Page 89 and 90: The second issue related to adding
Page 91 and 92: Figure 8-9. TeraGrid Computational
Page 93 and 94: Figure 8-11. Top 20 TeraGrid PIs, Q
Page 95 and 96: Institution Users Colorado Sch of M
Page 97 and 98: Institution Users CSU-San Bernardin
Page 99 and 100: objects that pass Futuregrid testin
Page 101 and 102: 9.4.1 Technology Audit Services Pro
Page 103: user management is complete. Local
Page 107 and 108: GIG place and orient new data. I pa
Page 109 and 110: north of UT Austin’s main campus)
Page 111 and 112: • Invite potential users of Gordo
Page 113 and 114: Modules Under Development Advanced
Page 115 and 116: Purdue RP has recruited two undergr
Page 117 and 118: eak-out session to the regional STE
Page 119 and 120: to coordinate press for all related
Page 121 and 122: ER-RP Reports for Q210: NCSA TeraGr
Page 123 and 124: The ORNL RP continues to collaborat
Page 125 and 126: NCAR Type Title Location Date(s) Ho
Page 127 and 128: Lab, TN Workshop Computational Thin
Page 129 and 130: tion Industry Technology Incubator
Page 131 and 132: Students Part 2 W Proteins in Actio
Page 133 and 134: Q2 ‘09 638 216 Q3 '09 504 403 Q4
Page 135 and 136: Factory Berkeley, Berkeley Lab (LBL
Page 137 and 138: performance analysis of parallel pr
Page 139 and 140: them with a thorough overview of th
Page 141 and 142: MIDAS Network software development
Page 143 and 144: 12. Wang, S. 2010. "A Cyber-GIS Fra
Page 145 and 146: 36. Xue, M., M. Tong, and G. Zhang,
Page 147 and 148: press, 2010 (arXiv:0909.0011) TeraG
Page 149 and 150: 121. Rast, M.P., Talk, “Precision
Page 151 and 152: 171. Brown, B.P.,Talk, “Solar rot
Page 153 and 154: 214. Cui, Y., Chourasia, A., Moore,
Page 155 and 156:
253. Le, P.M., and D.V. Papavassili
Page 157 and 158:
291. Liu, M.; Li, T.; Amegayibor, F
Page 159 and 160:
338. Xu, J., M. A. Sharpe, L. Qin,
Page 161 and 162:
387. Galan, J. F.; Brown, J.; Wildi
Page 163 and 164:
2008 ACM/IEEE Conference on Superco
Page 165 and 166:
461. Spinks, P.Q., Thomson, R.C., L
Page 167 and 168:
MCA06N063 498. "Emerging Photolumin
Page 169 and 170:
543. Zhu, J., Zhu, J., Negri, A., P
Page 171 and 172:
Physics AST030030 583. N. F. Lourei
Page 173 and 174:
634. M. S. Pindzola, J. A. Ludlow,
Page 175 and 176:
683. C. Reisswig et al., Gravitatio
Page 177 and 178:
726. E. Schnetter, P. Diener, N. Do
Page 179 and 180:
SI.Pkg 2.0.4.6.1 M Extension Quarte
Page 181 and 182:
Work Package Dependencies Planned Q
Page 183 and 184:
Work Package Dependencies Planned Q
Page 185 and 186:
TeraGrid AUS Project Plan Work Pack
Page 187 and 188:
TeraGrid AUS Project Plan Work Pack
Page 189 and 190:
Work Package Dependencies Planned R
Page 191 and 192:
Work Package Dependencies Planned R
Page 193 and 194:
TeraGrid DV PY5 Project Plan (as of
Page 195 and 196:
TeraGrid NOS Project Plan Work Pack
Page 197 and 198:
Albert[85%], Bennett[45%], ongoing
Page 199 and 200:
Project-ID WBS O P M Work Package D
Page 201 and 202:
Page 203:
show all

TGQR 2010Q2 Report.pdf - Teragridforum.org

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?