download as a pdf - Southern California Coastal Ocean Observing ...

Part III. Appendix 3: Data Archive and AccessThe IOOS DMAC methods of transporting and discovering data and metadata will evolve duringits early years. They will eventually set the foundation for increased data usage through “data mining,”which is currently a research endeavor focused on accessing data and automatically searchingout suitably described patterns in the largest data sets.Data latency is a requirement that links the users’ needs to the archiving costs. For IOOS, accesslatency is defined as the time between the earliest primary observation (not counting ancillarydata) in a data file and the availability of that file to users. For example, a field of monthly meanSST has a minimum latency of one month, whereas broadcast satellite data and buoy observationsused in operational modeling could have a latency of only minutes. Affordability is a factor herebecause low latencies are expensive. Requirements for low latency often come with requirementsfor high availability, which together imply around-the-clock staffing and special redundancies inhardware. For IOOS data users, latency requirements need to be assessed and suitably defined inthe metadata.Unrestricted access is a first principle for non-commercial IOOS data sets. Restricting access goesagainst this principle and is not encouraged. A policy on this issue will have to be established whenIOOS begins. There are circumstances where access may have to be temporarily restricted, however,typically beginning when the data are collected. Such circumstances include:• Proprietary embargo—Data are available only for sale from commercial companies (e.g., theinitial two-week embargo on SeaWiFS data),• National security—Data are available only for defense purposes,• Calibration and validation—Data are available only to the science team while they calibrate orvalidate instruments, data, or models,• Non-commercial use only—Data are available for government applications and academic research,but not for resale.These periods are envisioned to be temporary. Cost and efficiency make it useful to enter data intothe archives during the restricted period, however, while they are still fresh. Any archive center thatsupports temporary restrictions must be able to authenticate and properly authorize users so as toshield the data from general public use. The opportunities for restricted access, data security, andmetadata and data discovery support offered by the IOOS Archive System are an asset, previouslyunavailable, for the research science community.No archive system is complete without user services and use metrics. On-line documentation andknowledgeable staff will provide assistance and advice on both access and content. Additionalbackground information will be available through references and citations in the metadata. Broad210