download as a pdf - Southern California Coastal Ocean Observing ...

Part III. Appendix 2: Data TransportFormat translationIn theory, format translation could be the province of an all-knowing, mediating site capable ofaccepting data in the native format of any registered archive and transforming it into any registeredapplication’s format. Such a design is both delicate and complex. It would be subject to overloading,if it broke the data transport system would stop, and each added input or output format wouldincrease algorithmic complexity exponentially (number of translation methods equals number ofarchive formats times number of application formats).An alternative approach is to suppose the existence of a mediating format capable of representingall input and output formats and to distribute the responsibility of being able to translate fromnative to mediating format, and vice versa, to every server and client site respectively. This is thetack taken by OPeNDAP and used with success in the NVODS community, and by envisioning“machine-to-machine interoperability” rather than “machine-to-mediating-server-to-machineinteroperability,” this IOOS team endorses this second format translation topology.The idea underlying OPeNDAP’s treatment of numerical data formatting is not that a single mediatingformat can express all server and client formats, but that a single grammar can be devisedthat describes and stipulates a universe of mediating formats, at least one of which can be translatedto and from the formats used by any server-client combination. This grammar is modeled onthat used to declare programmatic datatypes and variables in such modern computer languagesas C++. As it happens to be true that a depth-first tree traversal of any instance of this grammarproduces an unambiguous, unique interpretation, then fidelity of transmission can be attained. Itcan be argued that the grammar is both terser and more familiar to those accustomed to lookingat program source code than other grammars such as XML, although an instance of OPeNDAP’sgrammar could be contained within (or referenced by) an XML-encoded document.Presently, OPeNDAP does not enforce any uniformity upon transmitted data other than using itspredefined basic datatypes to represent values. How the values are organized is left open, and canbe any well-formed instance of the grammar, conveyed as a Data Description Structure (DDS). Asprecisely equivalent information can be stored in different formats, and commonly is, OPeNDAPwill transmit these equivalent data sets somewhat differently, too. A time series of observations oftwo variables collected at a fixed location would be transmitted as a sequence of pairs of variablevalues or as two sequences of one variable value each, depending upon how the data were archived.Work remains to be done on whether, where, and how to specify that such data sets be transmittedidentically. The principal motive for such an effort would presumably be to minimize the complex-180