download as a pdf - Southern California Coastal Ocean Observing ...

Part III. Appendix 2: Data Transportusing ftp, very thick client(s), and a great deal of metadata to describe how the data are organizedand what they mean. The level of effort required to develop clients for such a system as well as toserve the data, however, makes such a design quite impractical:• The client would have to handle both format conversions and structural reorganization of thedata, as well as transformation of the data to a consistent set of units.• The data server would have to develop an extensive amount of metadata that describe the detailsof the organization of the data in addition to the data semantics.At the opposite extreme is the requirement that all data be stored using the same data model. Thisis even less practical, because:• Archives of historic data will be vital to IOOS and these are not stored in a consistent formatwith consistent metadata descriptions.• IOOS does not control all real-time data streams that are of potential interest to the IOOS community.Requisite Data Transport System Capability #2: It must be capable of providing accessto data via a variety of client programs, and it must translate from the format in which thedata are stored to the format required by the client program.It is quite clear that at least for the foreseeable future, users of the system will access data from thesystem via a variety of applications and interfaces; the data must be made available to the user’s clientapplication in the format desired by the client.Requisite Data Transport System Capability #3: It must be capable of delivering dataof a given data type in a structurally (syntactically) consistent form across all data sets in the system.It is important to stress that format consistency does not imply consistency in the structural organizationof the data delivered. For example, a sea surface temperature archive at one site might consistof a number of two-dimensional (longitude, latitude) files, one per time step, while another sitemight present similar data as one three-dimensional (longitude, latitude, time) file. The fact thatthe data are delivered directly to the application package in a consistent format substantially reducesthe complexity of client-side applications as well as the metadata needed to describe the data,but the lack of structural uniformity is still a substantial burden on the clients. For example, inthe multi-file 2-d case cited above, the client would have to deal with an inventory system of some145