The Annoyance Filter.pdf - Fourmilab

154 TOKEN PARSER ANNOYANCE-FILTER §181
181. If we’re assembling phrases, we take each token parsed (which has already been stored into the
dictionaryWord argument d in canonical form) and place it on the phraseQueue queue, removing
the element at the tail if the queue is longer than phraseMax . Then, if the queue contains phraseMin
elements or more, iterate over the range of phrase lengths we wish to generate, creating phrases and
storing them onto pendingPhrases for subsequent return.
〈 Class implementations 11 〉 +≡
void tokenParser ::assembleAllPhrases (dictionaryWord &d)
{
phraseQueue .push back (d.text );
if (phraseQueue .size ( ) > phraseMax ) {
phraseQueue .pop front ( );
assert(phraseQueue .size ( ) ≡ phraseMax );
}
for (unsigned int p = phraseMin ; p ≤ phraseMax ; p++) {
if (p ≤ phraseQueue .size ( )) {
deque〈string〉::const reverse iterator wp = phraseQueue .rbegin ( );
}
}
}
string phrase = "";
for (unsigned int i = 0; i < p; i++) {
phrase = (∗wp) + ((phrase ≡ "") ? "" : "␣") + phrase ;
wp ++;
}
if ((phraseLimit ≡ 0) ∨ (phrase .length ( ) ≤ phraseLimit )) {
pendingPhrases .push back (phrase );
}