Apache Solr Reference Guide Covering Apache Solr 6.0

By looking at the start and end positions for each term, we can see that the only thing this field type does is
tokenize text on whitespace. Notice in this image that the term "Running" has a start position of 0 and an end
position of 7, while "an" has a start position of 8 and an end position of 10, and "Analyzer" starts at 11 and ends
at 19. If the whitespace between the terms was also included, the count would be 21; since it is 19, we know that
whitespace has been removed from this query.
Note also that the indexed terms and the query terms are still very different. "Running" doesn't match "run",
"Analyzer" doesn't match "analyzer" (to a computer), and obviously "an" and "my" are totally different words. If
our objective is to allow queries like " run my analyzer" to match indexed text like " Running an
Analyzer" then we will evidently need to pick a different field type with index & query time text analysis that
does more processing of the inputs.
In particular we will want:
Case insensitivity, so "Analyzer" and "analyzer" match.
Stemming, so words like "Run" and "Running" are considered equivalent terms.
Stop Word Pruning, so small words like "an" and "my" don't affect the query.
For our next attempt, let's try the " text_general" field type:
With the verbose output enabled, we can see how each stage of our new analyzers modify the tokens they
receive before passing them on to the next stage. As we scroll down to the final output, we can see that we do
start to get a match on "analyzer" from each input string, thanks to the "LCF" stage -- which if you hover over
with your mouse, you'll see is the " LowerCaseFilter":
Apache Solr Reference Guide 6.0
176