Project Management Plan - Marine Geoscience Data System

Project Management Plan
Marine Geoscience Data System
&
Geoinformatics for Geochemistry Program
Version 1.7p
August 22, 2006
Lamont-Doherty Earth Observatory of Columbia University
1

1. Introduction................................................................................................................ 3
2. Geoinformatics Projects at LDEO............................................................................. 3
2.1. Vision statement ........................................................................................................... 3
2.2. General Management Structure ................................................................................. 5
2.3. Advisory Structure....................................................................................................... 5
2.4. Stakeholders ................................................................................................................. 6
2.5. Synergies .......................................................................................................................6
3. PEP for the Marine Geoscience Data System........................................................... 9
3.1. Project Definition ......................................................................................................... 9
3.2. Project Management.................................................................................................. 12
3.2.1. Organizational Structure....................................................................................................... 12
3.2.2. Roles and Responsibilities.................................................................................................... 13
3.2.3. Work Tasks........................................................................................................................... 15
3.2.4. System Development............................................................................................................ 16
3.2.5. Operations and Sustaining Engineering................................................................................ 16
3.2.6. Education.............................................................................................................................. 17
3.2.7. Reporting .............................................................................................................................. 18
3.2.8. Metrics of Success................................................................................................................ 18
3.2.9. Risk and Contingency Management..................................................................................... 19
3.2.10. Schedule/Milestones ........................................................................................................ 21
4. Appendix................................................................................................................... 23
4.1. System Architecture and Infrastructure for the Marine Geoscience Data System
(MGDS).................................................................................................................................... 23
2

1. Introduction
Federal agencies including the NSF, NASA, NOAA, and USGS fund a range of data
management and Geoinformatics activities for the Earth and Ocean Sciences which are
carried out at the Lamont-Doherty Earth Observatory (LDEO), the Center for
International Earth Science Information Network (CIESIN) and the International
Research Institute for Climate Change (IRI). LDEO, CIESIN and IRI are units of the
Earth Institute at Columbia University, co-located on Columbia’s Lamont Campus in
Palisades, NY. Projects funded by the NSF include several mature activities with a long
history of funding; the Sediment Core Repository which houses sediment cores from the
global oceans: and the ODP/IODP Borehole Group which serves down-hole logging data
in support of the ODP and IODP programs.
Recently the NSF has funded a suite of new activities to build and manage digital data
collections for Marine Geology & Geophysics, Geochemistry, and the broader
Geosciences. Though funded as independent activities with different duration and
termination dates and at different stages of maturity, these projects widely overlap in their
objectives, technical requirements, equipment, and personnel resources with significant
synergies among them.
This document describes strategic and management approaches, organizational
structures and timelines under which these NSF-funded cyberinfrastructure projects will
be executed. This is intended to be a dynamic document, reviewed and revised an
annually in discussion with the Advisory Committee and the National Science
Foundation. Revisions will reflect changes in the scope and type of projects included, in
the organizational structure, and in the funding. New projects will be integrated into this
plan as appropriate.
2. Geoinformatics Projects at LDEO
2.1. Vision statement
“It is exceedingly rare that fundamentally new approaches to research and education
arise. Information technology has ushered in such a fundamental change. Digital data
collections are at the heart of this change.” (Long-lived Data Collections, Report of the
National Science Board to the NSF, 2005)
3

Geoinformatics projects at LDEO are focused on building and maintaining digital data
collections that are essential resources for the Geosciences to maximize accessibility and
thus the application of scientific data in research and education. These data collections
aid data discovery, access to data, and data analysis, support cross disciplinary
approaches in science by facilitating data integration across disciplinary, spatial, and
temporal boundaries, and to advance the “principle of open access” to data and to
samples that were acquired with public funding.
Most of our systems can be viewed as Resource or Community Data Collections
according to the classification of digital data collections in the NSB report. Resource
Data Collections “serve a specific science and engineering community… They typically
conform to community standards, where such standards exist. Often these digital
collections can play key roles in bringing communities together to develop appropriate
standards where a need exists. In many cases community database collections migrate to
reference collections”. Projects in the MGDS and the GfG program have taken
leadership roles in defining standards for cruise metadata and geochemical data,
respectively, working closely with the appropriate communities through their Advisory
Committees, and various outreach activities.
Management of digital data for the Geosciences has a long tradition at LDEO that dates
back into the 1960s, and which led to a rich database of underway geophysical data and
subsequently to an early experiment in geographical database access (W. Menke’s
Geographical Database Browser XGB). In the mid to late 1990s, the RIDGE Multibeam
Synthesis project and the PetDB database for geochemical data of ocean floor igneous
and metamorphic rocks were developed and started to serve scientific data over the web
to a broad audience. The new projects described in this Project Execution Plan, build
upon and extend these earlier efforts. Most of our projects were proposed based on
recommendations from community workshops that outlined objectives, scope, and
functionality of data management systems that would best serve science and education
(Michael et al. 1991, Smith et al., 2001, Shipley et al., 1999, Cervato et al. 2004).
The significant growth in data management activities at LDEO, which is based on the
success of the initial efforts, was intentionally pursued to be able to establish a critical
mass of professionals who can take on the wide range of tasks necessary to design,
develop, and maintain digital data collections. Data management activities at LDEO have
grown incrementally, but have now reached a level where a well-defined management
plan is required that documents the approaches used to accomplish design and
development tasks, and a sustained operation of the data collections. This plan also
exposes the significant synergies among the individual projects and project clusters that
have been critical for the efficiency with which the systems are built and operated. These
synergies will be further exploited in the future through improved integration of our
operations, management and advisory structure.
Our primary goal for the future is to ensure the highest level of quality and long-term
sustainability of our systems, data products, and of the services they provide to the
community. In order to achieve this goal, we need to maintain a level of funding that will
allow us to support our operations with the superior expert teams that we have gathered,
and to continue infusion of relevant new technologies. A potential decrease of resources
due to termination of projects or reduction of funding will need to be balanced by new
4

endeavors that complement the existing capabilities. We will primarily focus on
expanding collaborations with other Geoinformatics efforts to leverage ongoing
developments and further contribute to establishing a cyberinfrastructure for Geoscience
research and education.
2.2. General Management Structure
The activities are managed in two thematically and organizationally distinct clusters,
the Marine Geoscience Data System (MGDS) and the Geoinformatics for Geochemistry
(GfG) system. The GfG projects are executed by a joint team from LDEO and CIESIN
with management responsibilities for the IT team at CIESIN, while all IT and science
directive activities for the MGDS are executed by a team from LDEO.
Figure 1 General
Management and
Advisory Structure for
the MGDS and GfG
In order to simplify management of the activities in the future, we envision the projects
within the two clusters to merge into two larger efforts. We plan to achieve this by
combining operation, maintenance, and further development of the individual systems in
upcoming renewal proposals.
2.3. Advisory Structure
All digital data collections and information systems built and maintained under this
Project Execution Plan are community resources for the broad Geosciences that manage,
archive, and serve scientific data to better benefit research and education. As such, they
need to address user concerns, and respond to broad scientific and educational needs. The
recent NSB report on Long-lived Digital Data Collections emphasizes the obligation of
data managers to maintain an “open and effective communication with the served
community” and to “gain the trust of the community that the collection serves”.
5

To establish the needed community input and oversight, and to ensure efficient
communication with the community and other stakeholders, advisory committees have
been established for individual projects and groups of projects with broad disciplinary,
institutional, and agency representation: An Advisory Committee was set up in 2004 for
the projects funded out of the MG&G program at NSF (all efforts combined under the
MGDS plus PetDB). This committee met twice, in 2004 and 2005, to discuss progress of
the projects and provide advice for ongoing and future development efforts. The
EarthChem project set up an Advisory Committee in 2005 that advises on tools,
functions, and data sets that may be desired to best benefit the user community, and to
define and assist with implementation of data policies and metadata standards. The
EarthChem AC met in December 2005 for the first time. The R2K and MARGINS data
management systems also report to their respective program steering committees on a
regular basis.
We propose to establish a single Advisory Committee for all data management projects
executed under this plan to enhance integration of the disciplinary data sets, their goals,
strategies. This Advisory Committee will be composed of a broad range of disciplinary
scientists representing marine and terrestrial geology, geophysics, and geochemistry,
Geoinformatics, and Information Technology practice.
2.4. Stakeholders
The following list outlines our current view of the stakeholders in this effort:
• Data Providers (system operators and scientists) require that data contribution be as
easy and as efficient as possible
• Data Users (researchers, educators, and the public) – need metadata and effective
search tools
• Researchers need to be able to access, visualize, and download data for further
analysis and the ability to limit access to some data
• Operating institutions (including vessel operators) need to be able to find data they
collected and to be recognized for their efforts.
• Oceanographic institutions have vested interest with respect to ownership and
recognition
• Shipboard technical support personnel who maintain, operate and document the data
acquisition equipment
• Mission scientists who have report-writing responsibilities
• Foreign collaborators with data submission and distribution requirements
• Educators use data and derived products for a broad range of activities
• Federal agencies including NSF, USGS, MMS, and NOAA who use and who
contribute data
• General public who pay the bills.
2.5. Synergies
The construction and maintenance of the MGDS and GfG data systems benefit
substantially from the natural synergies that exist among the individual projects as well as
the two project clusters. Objectives, science applications, and user communities overlap
6

widely, as do the technical requirements related to data and metadata types, tools, and
interoperability, offering extensive opportunities for collaboration and integration that
positively impact the execution of the projects and, more importantly, the quality of the
products. The synergies allow us to share design approaches, experiences, expertise,
personnel, and management tools to continuously support each other’s operation, ensure
compatibility, accelerate progress, and promote the broadest application of our data
collections. Figure 1 highlights the most prominent synergies among the projects.
Examples of synergies among MGDS and GfG include the application of GeoMapApp
(MGDS data visualization tool) as a map interface and plotting tool for the geochemical
data systems, the use of sample metadata schemes developed by PetDB for the MGDS
metadata catalog, seamless data exchange between PetDB and the MGDS, and the use of
MGDS cruise metadata for SESAR sample profiles. We will further augment shared
activities by integrating our advisory structure, and building a joint web site that will
serve as a common portal to all projects.
Figure 2 Synergies between projects in the MGDS and the GfG program
Additional benefits arise from close interaction with the broader data and sample
management activities at LDEO and CIESIN that include the IODP logging services data
management, the Socioeconomic Data and Applications Center, SEDAC, one of the
7

distributed active archive centers (DAACs) as part of NASA’s Earth Observing System,
the IRI Data Library, and the LDEO core repository.
8

3. PEP for the Marine Geoscience Data System
3.1. Project Definition
The projects grouped within the MGDS include the Antarctic Multibeam and
Geophysical Data Synthesis, the Ridge2000 Data Management System (DMS) the
MARGINS DMS, Seismic Reflection Field DMS and the Legacy project. Although these
individual projects serve different sections of the geoscience community, the underlying
goals of each effort are fully complementary. Thus we have evolved these projects as a
single unified data system. The overall scope of these projects is four fold 1) to serve as a
primary sensor database for underway geophysical data, multibeam sonar and seismic
reflection data from the R/V Palmer, Gould, Ewing, and Langseth 2) to develop a
comprehensive metadata catalog and data repository including derived data products for
Ridge2000 and MARGINS programs and for the wider spectrum of future and historical
marine geoscience programs 3) to provide a synthesis of global ocean bathymetry at full
spatial resolution of the available data, 4) to provide tools for data access tailored to
science needs. The projects within the MGDS are being developed as a single data
system with a common backend database and common front-end tools for access.
Because of the integrated nature of these efforts, developments funded under each project
have been leveraged to the benefit of all the others, providing significant economies of
scale. Development of the integrated data system has been underway since 2003 in
collaboration with researchers at other academic institutions (Table 3.1). The core data
system architecture is established and public access to project components has been
available since 2003/2004. The data system currently (Aug 2006) provides access to
~1.9TeraBytes of data, corresponding to over 196,000 individual data objects, and
associated with 1429 cruises dating back to the 1970’s. Data system use has been tracked
since public access began. During 2006 we track ~500-1500 data download sessions per
month and total monthly downloads of 160,000-510,000 individual data files.
Table 3.1 MGDS Projects and Collaborators
Project
Seismic
Reflection DMS
Lead
Institution
UTIG (T.
Shipley)
Collaborator
LDEO
Non-LDEO
Responsibilities
Development and operation of
Seismic Reflection Processed
DMS
RODES LDEO WHOI (T. Shank)* Scientific guidance for
biological data, biological data
compilation
MARGINS
DMS
LDEO TAMU (D. Becker)* Linkages with ODP database
LEGACY LDEO NGDC (Chris Fox), University
of New Hampshire (L. Mayer)
No cost collaborations to
implement multibeam data
exchange (NGDC) and develop
multibeam QC metrics (UNH)
9

R2K/MARGINS LDEO
renewal
(proposed Feb
2006)
* Subcontract
SIO (E. Simms)*
WHOI (V. Ferrini, S. Lerner)*
Vent Image Data Bank – image
compilation
NDSF post-processing Alvin/J2
nav, metadata delivery
1) Geophysical Sensor Database for R/V Ewing, Langseth, Palmer and Gould
Unlike other kinds of data which can be preserved and accessed via publications,
digital data derived from sensors require a dedicated repository to ensure long-term data
access and to provide needed functions of data migration to new digital media, and data
documentation to facilitate re-use. The MGDS serves the marine geoscience community
working on problems within the global oceans including the Southern Ocean to preserve
key marine geoscience digital datasets collected from the RV Ewing, Langseth, Palmer
and Gould. Geophysical sensor data which fall within the scope of the MGDS include
multi-channel seismic (MCS) reflection, multibeam sonar and underway geophysics
(gravity, magnetics) data. Acquisition costs alone for a typical multi-channel seismic or
polar ocean expedition are close to $1M per cruise but prior to our effort, no central
repository for these data existed and secondary reuse of these data has been minimal.
Data have resided with scientist originators and re-use has required prior knowledge of a
data set and direct contact with a PI who may or may not be able to read tapes and
transfer the data. The MGDS undertakes data validation and quality assessment (QA) for
these data, assumes responsibility for adequately documenting these data using
community defined standards, and providing open public data access, secure storage and
long-term data preservation.
Under the AMBS, all multibeam sonar data from the Palmer and underway
geophysical data from the Palmer and Gould have now been aggregated and are served.
Under the SRDMS, we have established direct transfer of MCS reflection field data from
the Ewing for programs during the final two years of operation and will implement a
similar procedure for upcoming programs of the Langseth. Older MCS field data from the
Ewing and processed MCS data, which reside with LDEO PIs are being recovered from
tape and incorporated into the data system. Under the LEGACY project, all multibeam
sonar data from the Langseth will be processed, validated, and archived.
2) Metadata Catalog and Data Repository for R2K, MARGINS, historical and future
MG&G programs.
The scope of the Metadata Catalog effort covers R2K and MARGINS funded field
expeditions, future MG&G programs outside these program areas, as well as historical
expeditions and data sets, which can be accessed from the community and adequately
documented. The primary goal for the Metadata Catalog component is to enable scientists
to discover what data were collected, by who, where, when, and the location where the
data reside. For the geophysical sensor data listed above, as well as for submitted data for
which no designated national repository exists, data are served locally from our
repository. For data for which a designated national repository does exist, the MGDS
links to these distributed data resources, rather than duplicating data holdings. An
important function of this component is to serve as a repository for derived data products,
10

e.g. gridded data sets, velocity models, tomography solutions, GIS project files etc, which
are of high value to the science user community for future re-use. The metadata catalog
effort provides essential proxy functions for the science user community with respect to
defining vocabularies and ontologies for marine geoscience data and establishing
protocols for data exchange.
3) Synthesis of seafloor bathymetry
This component is primarily a function of the AMBS and LEGACY projects, and
involves synthesis of publicly available multibeam bathymetry data from the ocean floor
into an easy to access multi-resolution gridded global digital elevation model (DEM).
Multibeam bathymetry data are unique among the marine geophysical data types in their
relevance for a broad range of scientific investigations as well as for non-academic uses.
For example, they provide fundamental characterization of the physical environment for
studies ranging from ocean bottom circulation to biological studies, as well as serving as
primary base maps for multidisciplinary programs. Detailed bathymetry maps are also
relevant for applications beyond academic research including management of marine
fisheries and other coastal resources as well as marine navigation (e.g. Jan 2005 collision
of the USS San Francisco into an uncharted seamount off Guam). At present, specialist
expertise is needed to access and manipulate multibeam bathymetry data, which is
typically available only as individual survey areas. The only existing broad access and
global compilations of seafloor bathymetry do not include multibeam sonar data at their
full spatial resolution (e.g. 2x2 minute compilation of Smith and Sandwell, 1997). The
MGDS provides synthesis of expedition based multibeam datasets at their full spatial
resolution data for non-specialist use by maintaining a continually updated dynamic
gridded global compilation.
4) Tools for data access tailored to science users needs.
In parallel with the data catalog and archiving functions, the MGDS works to provide
tools for data access and interactive visualization tailored to science user needs. The goal
of this component is to lower the barriers to data access and enable users to explore data
holdings without requiring a specialist understanding of the underlying data structures.
Our approach has been to develop a range of options for searching and accessing data. To
enable specialist users to find particular data sets of interest or to quickly see what data
have been collected in a region, we have developed a server-side text-based keyword
search interface integrated with a server-side mapping tool (MapServer). To enable visual
exploration for both specialist and non-specialist users, we have developed a domainaware
client side application, GeoMapApp, which permits dynamic interaction with a
variety of marine geoscience data including the multi-resolution global DEM. Open
Geospatial Consortium compliant Web Services are being developed to enable MGDS
data holdings to be accessed by other visualization tools.
Long Term Goals
The projects within the MGDS cluster are being developed as a community data
resource as defined by NSB report 0540 on Long-Lived Digital Data Collections. The
11

long-term goal of the MGDS is to establish core databases for marine geoscience data
and serve as an active data system for these data. Related goals are to:
• Enable marine geoscience data to be discovered and reused by a diverse
community for present and future use.
• Develop more than a data resource for marine geoscientist specialists by
establishing links to other geoinformatics activities, and pursuing developments in
interoperability.
• Compile global datasets to facilitate global syntheses.
Requirements arising from these goals drive the development of the data system and
include the need to:
• Handle diverse and large multidisciplinary data types (e.g. seismic, sonar,
geological, fluid, biological, rock, and sediment samples, temperature, photo
imagery)
• Provide access for both specialist and non-specialist users
• Take advantage of new emerging technologies
• Respond to evolving user needs
3.2. Project Management
3.2.1. Organizational Structure
One of the strengths of the MGDS has been the capacity to gather adequate resources
to build an expert team with the diverse expertise needed to handle the variety of MGDS
activities. This includes science experts and science community liaisons who can guide
the database development; data scientists and database and application programmers with
the essential domain expertise to contribute to database design; and data specialists who
gather and format data and metadata for input to the database. Any single project of the
MGDS would be unable to support a team with this needed range of expertise. Figure 3.1
illustrates the organizational structure for the MGDS with the teams identified that
perform tasks within six functional areas.
12

Figure 3.1 Organizational Structure of the Marine Geoscience Data System
As noted in section 3.1, several of the projects within the MGDS cluster include
collaborations with investigators at other institutions (Table 3.1) who participate in
Scientific Directorate activities as well as providing specific database components.
3.2.2. Roles and Responsibilities
The roles along with primary responsibilities of the MGDS team members are outlined
below.
Program Director: Oversees and coordinates all project activities; provides scientific
guidance for data system design and priorities; manages and oversees resources;
responsible for reporting, interfaces with NSF program managers for project components.
coordinates activities with collaborating partners; scientific community liaison.
Senior Scientist: Guides interface development (GeoMapApp); contributes to data
system design and priorities; scientific community liaison; coordinates activities with
collaborating partners; supervises undergraduate and graduate research experiences
associated with data system.
Science Advisors: Contributes to data system design and priorities including metadata
requirements.
13

Senior Engineer: Contributes to data system design and priorities, with focus on
system engineering; coordinates activities with collaborating partners; directs multibeam
QA effort.
Applications Programmers: Responsible for development and maintenance of
applications for data visualization and mapping (GeoMapApp); web services for
implementing interoperability.
Database Developer and Programmers: Responsible for database design and
implementation; web search tool for data access; aid data managers in development of
data loading and validation procedures.
Project Data Managers: Responsible for data solicitation, formatting, QC and entry;
work with database programmers to develop procedures for data loading and validation;
provides supervision for data specialists who work with team on data entry; science user
support and liaison.
System Analyst: Administration of servers including upgrades and monitoring;
software maintenance; system backup. Ensures system reliability, performance and
security.
Data Specialists: Aids in data and metadata entry including reformatting and compiling
metadata from legacy sources; recovers legacy data from tape; user support.
14

3.2.3. Work Tasks
Activities of the MGDS are grouped into tasks related to the development of the shared
infrastructure as well as project specific tasks. The overarching activities that serve all
projects represent the synergies gained by building a unified data system to serve the
various science needs addressed by the projects within the MGDS. Significant cost
savings are achieved by having all of these elements developed once rather than by
multiple groups. Table 3.2 shows general Work Tasks of the MGDS with shared
infrastructure and project specific tasks indicated, as well as primary staff members
responsible for each component
Work Task
Staff
Management
Program Management & Directive • Carbotte, Ryan
Program Administration& Coordination Assistance
Taylor (hire expected
• November 2006)
System Engineering & Development
Database design, development, and review • Arko, OHara +Team
Web site development • sub contract
Search interface development & deployment • Arko, OHara +Team
Interoperability development (web services, controlled • Arko, Melkonian
Map Interfaces (GeoMapApp, GoogleEarth,
Melkonian, Ryan, Arko
MapServer)
•
Data loading procedures/scripts OHara, Goodwillie
MB quality assessment metrics development Chayes
System Operation
Database administration • Arko
Data ingest Goodwillie, OHara
System maintenance, security, backups
Arko, Chayes +LDEO
• network support
Web site maintenance • OHara, Goodwillie
Data stewardship (long-term archiving, metadata
Arko, OHara
documentation)
•
Data Development
Data compilation/solicitation: Legacy + Modern
Carbotte (R2K), Goodwillie
(Margins/Legacy), OHara
(AMBS), Alsop (SRFDMS),
Weissel, Barone
15

Data and metadata entry & quality control
User support
Goodwillie
(R2K/Margins/Legacy),
OHara (AMBS), Alsop
(SRFDMS),
Weissel
Goodwillie
(R2K/Margins/Legacy),
OHara (AMBS), Alsop
(SRFDMS),
Weissel (RT) +Team
Outreach & Community Relations, Education
Publications & presentations
Carbotte, Ryan, Goodwillie,
Arko, Chayes
Workshops Team
Exhibits • Team
Education
Ryan, Kastens
Table 3.2: Work Tasks for the MGDS program.
Legend: • = overarching activities; = project-specific tasks; = partly overarching
Development of specific components of the shared infrastructure has been funded
through the individual projects of the MGDS as reflected in the combined schedule for
the system shown in section 3.10. The project schedule shows work tasks grouped by the
project funding the task, rather than by the broad grouping shown above.
3.2.4. System Development
System development for the MGDS has been guided by community input regarding
science needs as well as new technological developments. The core architecture of the
MGDS is designed to follow recommendations of community workshop reports (Smith et
al., 2001, Shipley et al., 2000) as well as requirements defined in R2K and MARGINS
data policy documents. The MGDS is based on open, well documented, standards and
implemented almost entirely with open source free software. Outreach and reporting to
the science user community has been an integral part of system development (see section
3.2.7) along with participation via meetings and workshops with geoinformatics activities
in other areas (www.marine-geo.org/meetings). Our most closely aligned activity is with
the Marine Metadata Initiative (MMI), a community-based effort lead by John Graybeal
at MBARI to develop standard vocabularies and ontologies for metadata across the
spectrum of marine science activities. Our involvement in this effort ensures that our
metadata developments are coordinated with broader community efforts.
3.2.5. Operations and Sustaining Engineering
System operations include periodic review and refinement of database design, data
solicitation procedures (digital cruise metadata forms), as well as data ingestion,
16

validation and archiving procedures. We currently operate under version 89 of our
database schema. Our digital cruise metadata forms have been through two cycles of
revision based on user feedback and will continue to be assessed on an annual basis.
Periodic review of search interfaces and GeoMapApp are conducted based on user
feedback provided through advisory and science steering committee feedback. We are
scheduled for a major new release of our text-based search interface, Data Link, in spring
2006. Updates to GeoMapApp functionality have been provided on a roughly quarterly
basis.
Project goals, tasks, timelines, critical paths and milestones are planned with FastTrack
Schedule 9® software chosen because it runs in both our network Windows and
Macintosh environments.
Document management is handled using Plone®, a web-based content management
system with a full audit trail that lets the team enter, view, and edit documents. The
MGDS currently uses Plone to manage documents describing data ingestion procedures,
dynamic task lists, as well as documents associated with team meetings (agendas, action
items),
Request Tracker® is employed for handling all user feedback and inquiries and also
provides an audit trail of all user communications. Each user request is logged via
Request Tracker along with subsequent correspondence, comments and solutions.
Routine security checks and updates are done in collaboration with the network support
group at Lamont. Software updates to our core services are applied after testing on a nonproduction
system.
3.2.6. Education
The MGDS contributes to the goal of educating a new science workforce with an
understanding of data stewardship by involving student interns in project activities. Our
approach is to develop science research projects for these interns which include data
compilation and entry into the database, Since project inception, graduate students and
undergraduate summer interns as well as local high school students have been involved in
many aspects of data harvesting and programming (including the Java code for
GeoMapApp). Two of these individuals have presented their work at AGU and GSA
meetings (Ryan, W B, Muhlenkamp et al., 2003; Schon et al., 2004). The experience of
these individuals contributes to the education of a new generation that understands the
importance of metadata as well as data contribution as part of the scientific research
process.
The R2KDMS includes a dedicated Education Component, which has proceeded in
parallel with development of the data system. This component has been lead by Kim
Kastens and involved collaboration with Tamara Ledley of the DLESE Data Services
Group to host a workshop focused on obtaining input from education users into the
design of data system tools. This workshop was held at Lamont on July 21-22, 2004 and
is documented on the web at http://swiki.dlese.org/DataSvcsWkshp-04/140. Development
of data-rich activities for K-16 education was supported by this component and has
resulted in the following curriculum activities: a module on the Dynamics and
Geomorphology of Mid Ocean Ridges by Jeff Thomas of Teachers College
17

(http://serc.carleton.edu/dev/eet/rodes_6/index.html), a Mid Ocean Ridge Basalt example
using PetDB and Excel by Matt Smith and Mike Perfit of the University of Florida and a
Calcium Carbonate exercise based on GeoMapApp assembled by Bill Ryan and Peter
DeMenocal of LDEO/Columbia University. Additional examples include an evaluation
of earth science awareness developed by Sandra Swenson, a graduate student at Teachers
College, working with 8 th , 10 th and 12th grade Earth Science students, spring 2005
3.2.7. Reporting
An external Advisory Committee which covers all projects within the MGDS cluster
has been established to provide advice and oversight from the scientific community
(http://www.marine-geo.org/advisory.html). This committee meets annually to provide the
MGDS team with advice on priorities, directions and science community needs as well as
advice to NSF regarding functioning of the DMS.
Reporting to the Ridge2000 and MARGINS science user communities occurs twice
yearly with biannual updates provided at the steering committee meetings of both of
these programs. This frequency of reporting has kept the data system closely connected
to the science user community and has contributed to the success of data submission to
the MGDS. It also ensures system development in response to needs and priorities
defined by these science communities.
The MGDS has operated a booth on the Exhibition Hall for the past two fall AGU
meetings and will continue this activity in future years. This activity contributes
significantly to advertising the data system and engaging new users.
Annual reports are written for NSF for each project. We favor moving to a single
annual report to NSF for all projects of the MGDS. This annual report could also serve as
a summary document for the AC and record of annual progress. NSF oversight of the
MGDS could include facility site visits coincident with annual Advisory Committee
meetings.
3.2.8. Metrics of Success
The success of the MGDS is monitored by tracking web site and GeoMapApp use, as
well as growth of the data holdings. The number of site visits and volumes of file
downloads are tracked on a monthly basis using the Webalizer site use statistics package.
For GeoMapApp usage, we track number of new and returning users, and volumes
downloaded. User feedback was directly solicited at our annual booth on the AGU
Exhibition Hall in 2005 with a user logbook. Unsolicited user feedback received via
email queries and comments are logged using Request Tracker. Other methods to solicit
user feedback including email surveys and web site questionnaires will also be
implemented in the future.
Given the nature of the data system as a data catalog as well as a repository of sensor
data, growth of the data system is monitored through a variety of metrics, including
number of data objects within the DMS, number of field programs cataloged, as well as
growth in data file volumes. Each data object cataloged within the MGDS is tied to the
relevant project that funded the data ingestion so that growth in data holdings can be
tracked via MGDS project.
18

An additional metric of success will be the willingness of the user community to
submit data to the MGDS. Since development of our digital metadata forms we have
observed a growth in number of users who submit metadata forms without direct contact
and solicitation from us. As the data system grows more useful to the user community we
hope data submission may become more automatic and timely with less direct solicitation
required.
Citations in publications are the standard measure of scientific impact, and will be
tracked as an additional measure of success. However, the nature of many of the data
services provided by the MGDS, are unlikely to lead to citations, and publications are
likely to reflect only a small percentage of DMS impact. For example, use of the system
to find the whereabouts of data collected within a region of interest as part of a pre-cruise
or proposal-planning effort would not lead to a citation. Downloaded bathymetry data
used as a base map for a study are rarely cited. Given the typical publication cycle of a
manuscript, citations also become a more useful measure of impact of a data system
through time.
3.2.9. Risk and Contingency Management
This section outlines our existing strategy for dealing with some possible risks
including physical damage, malicious hacking, personnel changes, etc.
1. Risk of physical destruction of facility by fire, flood or other catastrophic event.
Mitigation: Risk of physical destruction is mitigated by maintenance of off-site data
backups and off-site redundant server as described in the MGDS Infrastructure that is
described in Appendix 5.1. In the event of physical destruction of our primary servers (or
loss of physical access to the building) we would switch service to the off-campus
system, which we are in the process of installing on the Morningside Campus of
Columbia. We would then acquire new hardware and re-establish our servers in a
different (temporary) location on our campus. In the event of a campus-wide destruction,
we would rely on our Morningside location. In either case, we would restore from our
off-campus backups that are stored by Iron Mountain.
2. Network vandalism.
Mitigation: In the event of malicious network vandalism, we assume that such an attack
would hit both our primary servers at Lamont and the Morningside campus. Such an
attack would most likely not damage the hardware so recovery would be relatively quick.
This would take us off-line for a period on the order of 12-24 hours while we understood
the breach, protected ourselves from a future instance and restore from our backups.
3. Loss of key personnel. Due to limited funding the data system operates with minimal
redundancy in expertise.
Mitigation: The data system is operated with overlap in expertise in essential areas of
database programming (O’Hara and Arko), applications (Arko, Melkonian, and new
personnel), and science directive (Carbotte, Ryan). Project leadership, reporting and
19

liaison with the science community have been shared within the team ensuring that loss
of one individual can be absorbed. Adequate documentation of system components
including database structure, ingest procedures, and interfaces ensures that DMS
operations can be transitioned to new personnel as required.
4. The short-term and uncertain nature of the funding under which the MGDS has
evolved means that the longevity and security of the projects is a risk. Projects are
competed through peer review and are subject to the interests and priorities of the science
community regarding need for open data access and data preservation. In a tight funding
climate, needs to ensure data preservation and data access may be perceived as in conflict
with the need to fund new scientific research. Projects require highly expert personnel
with training in both science and IT domains. Such people are difficult to find and keep
in an uncertain and short term funding climate.
Mitigation: The MGDS aims to ensure that the data system provides high quality
service to build the community support needed to secure ongoing funding under renewal
grant periods. In the event of a short-term gap in funding, bridging support can be
requested from the Observatory. The MGDS Advisory Committee (currently configured
to include PetDB and SedDB) voiced their support for future funding of the DMS under a
Cooperative Agreement (see recommendations from October 2005, www.marinegeo.org/advisory).
More secure long-term funding would contribute significantly to
ensuring that high quality personnel can be trained and retained.
In the event that all NSF funding for the MGDS ceases, operation of the data system
would continue as long as servers remain operational. Addition of new data, user support,
and further development of functionality would cease.
5. Enforcement of data policies for data submission fall outside the purview of the
MGDS and data submission requires cooperation of the science community. Data
submission requirements are defined by NSF and community data policies, and in the
case of the R2K and MARGINS communities, include required submission to the
MGDS. However, enforcement of these submission requirements fall outside the purview
of the DMS and data contribution can only be solicited from the community on a
voluntary basis. This leads to partial and in some cases difficult to obtain data
submission, and a large time investment in solicitation.
Mitigation: Develop a multi-prong approach to improving data submission in the longterm
that includes direct solicitation of scientists, as well as working towards new
automatic procedures for data documentation at sea and direct data transfer from the ship
operators. Automatic transfer of data from the Palmer by Raytheon Polar Services
personnel has been established through the directive of our OPP Program Manager.
Direct transfer of seismic and multibeam data from the Langseth will be established in
support of the SRDMS and LEGACY through our direct link to LDEO ship operations.
The R2K and MARGINS DMS projects have dedicated science communities that support
the data effort and can provide community pressure. The legacy data components of
MGDS projects are focused on data sets within Lamont holdings as well as other public
centers (NGDC) where data access is known. Mitigation also requires more rigorous NSF
enforcement of data submission requirements as well as engagement of NSF Program
20

Managers for Ship Operations in devising multi-prong data submission procedures that
include both scientists and improved data documentation at sea.
6. Project scope is large and project activities associated with data documentation and
validation and all aspects of legacy data recovery are manpower intensive.
Mitigation: Target activities of maximum benefit to science community as determined
through the proposal review process and via an advisory structure that participates in
establishing priorities.
3.2.10. Schedule/Milestones
Figure 3.3 provides a combined schedule for all project tasks for the MGDS for the
duration of current funding (until September 2010). Note that the current R2K and
MARGINS DMS grants expire in summer of 2006 and most project activities for these
grants are completed. A renewal proposal for a combined R2K and MARGINS DMS was
submitted to the February 15 2006 MG&G deadline and is currently pending.
21

4. Appendix
4.1. System Architecture and Infrastructure for the Marine Geoscience
Data System (MGDS)
The diagram below illustrates the core architecture on which MGDS systems are
developed, tested, and operated. It consists of backend Platforms (Web and Database),
connecting Services, and end-user Applications. This infrastructure serves both
production and development.
Hardware
The production servers are Intel/Linux (Fedora Core), which yield excellent priceperformance
and benefits from the open-source developer community. The development
23

servers are a mix of Linux, Solaris, Mac OS X, and Windows systems, which allow us to
test applications for cross-platform compatibility. The complete list is shown below:
Purpose Operating System Manufacturer/Model
Production #1 Linux Dell Precision 8300
Production #2 Linux Dell Precision 8300
Production #3 Linux Dell Precision 8300
Development Linux Dell Precision 450N
Development Solaris Sun Blade 100
Development Mac OS X Apple iMac G5
Development Windows Dell OptiPlex GX280
A master copy of all MGDS data objects is stored in the LDEO Mass Store system, an
enterprise-class “deep archive” consisting of a quad-dual-core Sun Fire V890 server and
ADIC Scalar i2000 tape library. The system holds 2 TB of online disk cache and ~80 TB
of near-line LTO tape storage, replicated to secure, climate-controlled sites in two
different buildings on campus via redundant gigabit switches to the campus backbone.
A wide range of magnetic/optical drives are available to transfer data packages, directly
attached to the MGDS cluster or adjoining subnet, including DLT (SuperDLT 320 and
DLT-4000/7000), IBM 3480/3490, 8mm tape (Exabyte 82xx, 85xx, and 87xx), 4mm tape
(DDS-2, 3, and 4), removable disk (DynaMO 2300, Zip100 and 250, Jaz 1- and 2-GB)
and optical disc (8x DVD+/-RW and 40x CD-RW).
Software
A wide range of open-source and commercial software supports MGDS development
and operation. The complete list is shown below:
Package/Version
Apache HTTP 2.2
Apache Tomcat 5.5
PHP 5.1
PostgreSQL 8.1
PostGIS 1.1
MapServer 4.8
Eclipse 3.1
Subversion 1.4
Plone 2.1
Request Tracker 3.6
NetWorker 7.3
Purpose
Web server
Java servlet engine
Web script language
Relational database
Database geospatial extensions
Web GIS
Integrated development environment
Version control
Content management system
User feedback
Backup/recovery
MGDS development also utilitizes CU/LDEO site licenses for commercial scientific
software packages including ArcGIS (ESRI), ENVI (RSI), and MATLAB (MathWorks).
24

Networking and Security
The MGDS server cluster is connected to the Lamont campus gigabit fiber backbone,
which is in turn connected to the commodity Internet via redundant land (to the
Morningside campus in New York City and Internet 2) and microwave (to an alternate
ISP routed through Boston) T3 links.
The entire Lamont campus network is protected by a Juniper NetScreen-208
(www.juniper.net) enterprise firewall. External access to MGDS production machines,
such as user logins and incoming file transfers, is segregated or disabled.
Automatic backups are performed nightly on all DMS systems (both production and
development). EMC NetWorker (www.legato.com) is used for general filesystem
backups, and built-in (application-specific) export functions are used for the PostgreSQL
relational database system and Plone content management system. All hardware systems
are covered by extended vendor warranties with on-site service. All software systems are
reviewed and updated as needed on a quarterly basis.
Contingency
An agreement has been reached with the Columbia University Information Technology
(CUIT) department to install a remotely managed backup server at the CUIT Computer
Center on the Morningside campus in New York City. A contract has been signed with
Iron Mountain, Inc., to provide secure long-term digital media storage at their Data Vault
facility in New Paltz, NY. Transfers to the Vault occur monthly by courier.
25