MARKLOGIC SERVER

storage media but still query those documents as a single unit. For queries to work, the
super- and sub-databases must have the same configuration. Querying multiple data
sources at once and then aggregating the results is also known as federated search.
For example, let's say that you have a Database A of active documents on fast,
expensive storage and a Database B of archival documents on slower, lower-cost
storage. You can create a super-database and associate it with Database A and Database
B. (The super-database itself can also contain forests of documents, although that is not
recommended since those forests cannot be queried independently of the associated
sub-databases.) With this arrangement, you can query just the high-value documents
in Database A, just the archival documents in Database B, or the entire corpus by
querying the super-database.
Query
“cat OR dog”
SUPER DATABASE
DATABASE A
DATABASE B
F
F
F F F
F
F
F
F
F
Figure 25: A super-database lets you query multiple databases as a single unit.
There are limitations on super-databases. You cannot perform updates on documents
that reside in a sub-database via a super-database, and you must perform the updates
directly on the sub-database where the documents reside. You also cannot have a superdatabase
with more than one level of sub-databases beneath it. This is to avoid circular
references—i.e., two databases referencing one another.
HDFS AND AMAZON S3
A key to tiered storage is the ability to store less-active data on lower-cost storage media.
Two low-cost options are Hadoop Distributed File System (HDFS) and Amazon S3
(Simple Storage Service).
HDFS is the storage part of Hadoop. Because HDFS is open source and runs on
commodity hardware, it can store data less expensively than traditional shared storage
solutions such as SAN (storage area network) and NAS (network-attached storage).
Once you configure MarkLogic to use HDFS, you can specify HDFS as a storage
directory for a database using the "hdfs:" prefix.
Amazon S3 is cloud-based storage service offered through Amazon Web Services. S3
customers can interact with documents stored on the service through various interfaces,
117