download as a pdf - Southern California Coastal Ocean Observing ...

Part III. Appendix 2: Data TransportSRBThe Storage Resource Broker is a generic data management infrastructure that supports digital entities.It treats each digital entity as an atom from the viewpoint of a logical name space. However,the SRB does support all of the traditional latency management functions required for wide areanetworks, including:• partial file read/write• streaming• caching• replication• staging• aggregation of digital entities into containers• aggregation of metadata into XML files• aggregation of I/O commands into remote proxiesThe aggregation of I/O commands allows the user to specify a series of operations that requireknowledge of the encoding format/data model. The operations are performed at the remote storagesystem to minimize the number of I/O commands that are sent over the network.Similarly, the aggregation of metadata into XML files is done through the application of templatesat the remote storage system. The templates specify how a digital entity can be parsed to identifyrelevant metadata attributes, which can then be shipped in bulk over the network for ingestion intoa database.Since the SRB deals with digital entities, it is possible to register files, directories, databases, URLs,SQL command strings, etc. into the logical name space. One can manipulate the organizationof the logical name space independently of the physical names used on the actual storage systems.This freedom makes it possible to assemble a collection that spans multiple administrationdomains/sites/storage systems.The SRB can be used as generic collection management software, independent of the structure ofthe digital entities. The challenge comes in when latency management functions are invoked thatrequire knowledge of the structure of the digital entity. Hence, the desire to encapsulate the knowledgethat is required to manipulate a digital entity as a remote proxy or template that can be movedto the remote storage system for application.178