MARKLOGIC SERVER

INDEXING LONGER PHRASES
What happens if instead of a two-word phrase I give you a three- or four-word phrase?
Again, what would you do on paper? You can choose to rely solely on filtering, you can
use position calculation, or you can create a term list entry for all three- and
four-word phrases.
It goes beyond the point of diminishing returns to try to maintain a term list for all
three- and four-word phrases, so that's not actually an option in MarkLogic. The fast
phrase searches option only tracks two-word phrases. However, the two-word phrase can
still be of some help with longer phrases though. You can use it to find documents that
have the first and second words together, the second and third words together, and the
third and fourth words together. Documents that satisfy those three constraints are more
likely candidates than documents that just have all four words at unknown locations.
If word positions is enabled, will MarkLogic use fast phrase searches as well? Yes, because the
position calculations require time and memory proportional to the number of terms
being examined MarkLogic uses fast phrase searches to reduce the number of documents,
and thus terms, that need to be processed. You'll notice that MarkLogic doesn't require
monolithic indexes. Instead, it depends on lots of smaller indexes, turned on and off
depending on your needs, cooperating to resolve queries. The usual routine is to look at
the query, decide what indexes can help, and use the indexes in cooperation to narrow
the results down to a set of candidate documents. MarkLogic can then optionally
filter the documents to confirm that the documents actually match, eliminating false
positives. The more index options enabled, the tighter the candidate result set can be.
You may be able to tune your index settings so that false positives never occur (or occur
so rarely that they're tolerable), in which case, you don't need filtering.
INDEXING STRUCTURE
Everything up to this point is pretty much standard search engine behavior. (Well,
except that traditional search engines, because they don't hold the source data, can't
actually do the filtering and always return the results unfiltered from their indexes.) Let's
now look at how MarkLogic goes beyond simple search to index document structure.
Say I ask you to find XML documents that have a element within them. What
would you do? If you're MarkLogic, you'd create a term list for element , the
same way you do for a word. You'd have a term list for each element, and no matter
what element name gets requested you can deliver a fast answer. (It works similarly for
JSON documents. See "Indexing JSON" below for details.)
Of course, not many queries ask for documents containing just a named element. So
let's say the question is more complex. Let's try to find XML documents matching the
XPath /book/metadata/title and for each, return the title node. That is, we want
documents having a root element, with a child and a
15