download as a pdf - Southern California Coastal Ocean Observing ...

Part III. Appendix 2: Data Transportby subscription (push 4 ) while retrospective data are generally requested (pull 5 ) in an as-neededmode. In addition, access to data for special events, such as hurricanes and floods, generally imposesa greater burden on real-time data sources than on retrospective data sources at the time of theevent. The fully developed data access system will likely be composed of a broadcast capability suchas the LDE developed by Unidata together with a pull technology such as OPeNDAP. For routinereal-time data flows, the push technology is likely to dominate, although data pull will also likelyplay a role. This will depend on the access approach chosen by the data user. For routine access toretrospective archives, the pull technology will dominate. For access during special events, it willlikely be a combination of the two forms of access that will be result. Specifically, recipients of thedata pushed to the community as part of the routine data delivery system may become the serversof choice for access by many because of the relatively lighter load associated with these servers intimes of heavy demand.There is a refinement of “pull” called “informed pull,” whose implementation probably lies in thefuture but which should be considered seriously by the IOOS data transport system. In this scheme,servers send metadata describing characteristics of newly available data, and clients decide whetheror not to initiate a “pull’ transaction, presumably on the basis of whether or not the new datapromises to fill some previously recognized need.Mediated facilities: a form of distributed computingData restructuring, aggregation, and manipulation will be absolutely essential aspects of IOOS datatransport. Such facilities could be implemented at only selected server sites yet be available to theentire community of IOOS Data Transport clients. For instance, a client might have data sets organizedaccording to one data model and wish for some reason to have them expressed according toanother informationally equivalent data model instead, that is, to have its data set “restructured.”This client should be able to send a request to a Restructuring Server asking that the server obtainthe data set from some site, restructure it, and return the result to the client. Such mediating serverscan be constructed to provide an arbitrary array of operations. COLA’s GDS (http://cola8.iges.org:9090/index.html) is an example of this.This report will recommend funding for a working group to address the issue in detail as soon aspossible as part of the design implementation program.3By real time we mean “shortly” after the data have been collected, not actual access to the sensor data stream.4“Push” refers to data being sent from a server site to a client site. The send is initiated by the server as opposed to data being requestedby the client. The client would previously have registered a subscription to the service, but, like a daily newspaper, packagesof data would arrive at the client either regularly or as they became available without any further action on the client’s part.5“Pull” is the process we are all used to using: we request specific data, the request is filled, and the transaction ends.156