The Annoyance Filter.pdf - Fourmilab

§178 ANNOYANCE-FILTER TOKEN PARSER 153
}
return true ;
}
continue;
This code is used in section 174.
179. If the user has so requested, we can assemble tokens into phrases in a given length range. The
default minimum and maximum length phrase is 1 word, which causes individual tokens to be returned
as they are parsed. When the maximum is greater than one word, consecutive tokens (but never crossing
a reset or setSource boundary) are assembled into phrases and output as pseudo-tokens of each length
from the minimum to maximum length phrase.
Here we examine the phrase length parameters, report any erroneous specifications, and determine
whether phrase assembly is required at all.
〈 Check phrase assembly parameters and activate if required 179 〉 ≡
assemblePhrases = false ;
if ((phraseMin ≠ 1) ∨ (phraseMax ≠ 1)) {
if ((phraseMin ≥ 1) ∧ (phraseMax ≥ phraseMin )) {
if ((phraseLimit > 0) ∧ (phraseLimit < ((phraseMax ∗ 2) − 1))) {
cerr ≪ "Invalid␣−−phraselimit␣setting.␣␣Too␣small␣for␣specified␣−−phrasemax." ≪
endl ;
}
else {
assemblePhrases = true ;
}
}
else {
cerr ≪ "Invalid␣−−phrasemin/max␣parameters.␣␣Must␣be␣1␣