MARKLOGIC SERVER

CHAPTER 2
CORE TOPICS
INDEXING TEXT AND STRUCTURE FOR FAST SEARCH
Now that we've covered what MarkLogic is, let's dig into how it works, starting with its
unique indexing model. MarkLogic indexes the words, phrases, structural elements,
and metadata in database documents so it can search those documents efficiently.
Collectively, this indexed information is known as the Universal Index. (MarkLogic uses
other types of indexes as well—for example, range indexes, reverse indexes, and triple
indexes. We'll cover those later.)
INDEXING WORDS
Let's begin with a thought experiment. Imagine that I give you printouts of 10
documents. I tell you that I'm going to provide you with a word, and you'll tell me
which documents contain the word. What will you do to prepare? If you think like a
search engine, you'll create a list of all words that appear across all the documents, and
for each word, make a list of which documents contain that word. This is called an
inverted index; inverted because instead of documents having words, it's words having
document identifiers. Each entry in the inverted index is called a term list. A term is just
a technical name for something like a word. Regardless of which word I give you, you
can quickly give me the associated documents by finding the right term list. This is how
MarkLogic resolves simple word queries.
Now let's imagine that I ask you for documents that contain two different words. That's
not hard, you can use the same data structure. Find all document IDs with the first
word and all document IDs with the second, and intersect the lists (see Figure 1). That
results in the set of documents with both words.
It works the same with negative queries. If I ask for documents containing one word but
excluding those with another, you can use the indexes to find all document IDs with the
first word and all document IDs with the second word, then do a list subtraction.
12