MARKLOGIC SERVER

Fields are another way to pinpoint what parts of your documents get indexed. They
enable you to include (or exclude) different XML elements or JSON properties as
a single indexable unit. For example, you can combine and
elements (but exclude ) as a field. By adding a range index
on the field, you can perform range operations on those combined values. Or maybe
you have documents that define last names in different ways, with and
elements. By creating a range-indexed field for those variations, you can
order your search results across these documents by last name even though the markup
varies. See the "Advanced Topics" chapter for more on fields.
LEXICONS
When configuring a database, you have the options to configure a "URI lexicon" and
"collection lexicon." These are range indexes in disguise. The URI lexicon tracks the
document URIs held in the system, making it possible to quickly extract the URIs
matching a query without touching the disk. The collection lexicon tracks the collection
URIs, letting you do the same with them. Internally, they're both just like any other
range index, with the value being the URI. The lexicon retrieval calls can accept a
passed-in cts:query object constraint, making it easy and efficient to find the distinct
URIs or collections for documents matching a query.
Why do we need a range index on these things? Isn't this stuff in the indexes? Not by
default. Remember that term list lookup keys are hashes, so while it's possible with the
Universal Index to find all documents in a collection (hash the collection name to find
the term list and look at the document IDs), it's not efficient to find all collections (hashes
are one-way). The lexicons can calculate the document and collection URIs the same
way regular range indexes extract values from within documents.
It's also possible to configure a "word lexicon" to track the individual words in a
database (or limited to a certain element). For memory efficiency the word lexicon
is kept as a flat list of words with no associated document IDs, lest a common word
require a large number of sequential entries in the list. You can still pass a cts:query
object constraint to the word lexicon retrieval call to retrieve only words in documents
matching the query. To impose that constraint, MarkLogic pulls each word from the
lexicon and checks it, via a quick term list lookup, to see if any document IDs from its
term list are in the document IDs matching the query. If so, the word is returned; if
not, it's on to the next word. Word lexicons can help with wildcard queries, which will
be discussed later.
DATA MANAGEMENT
In the next section, we'll take a look at how MarkLogic manages data on disk and
handles concurrent reads and writes.
33