The Annoyance Filter.pdf - Fourmilab

§185 ANNOYANCE-FILTER CLASSIFY MESSAGE 157
185. The classifyThis method reads the next message from the mail folder and returns the probability
that it is junk. If the end of the mail folder is encountered −1 is returned.
〈 Class implementations 11 〉 +≡
double classifyMessage ::classifyThis (bool createTranscript )
{
dictionaryWord dw ;
double junkProb = −1;
}
if (createTranscript ∨ (transcriptFilename ≠ "")) {
mf ⃗ setTranscriptList (&messageTranscript );
if (Annotate (’p’) ∨ Annotate (’d’)) {
saveParserDiagnostics = true ;
}
}
〈 Build set of unique tokens in message 187 〉;
〈 Classify message tokens by probability of significance 188 〉;
〈 Compute probability message is junk from most significant tokens 189 〉;
if (tp.getSaveMessage ( )) {
〈 Add classification diagnostics to parser diagnostics queue 190 〉;
ofstream mdump(pDiagFilename .c str ( ));
tp.writeMessageQueue (mdump);
mdump.close ( );
}
if (createTranscript ∨ (transcriptFilename ≠ "")) {
〈 Add annotation to message transcript 191 〉;
if (transcriptFilename ≠ "") {
mf ⃗ writeMessageTranscript (transcriptFilename );
}
}
return junkProb;
186. Just one more thing. . . . We need to define an absolute value function for floating point
quantities. Make it so.
〈 Class definitions 10 〉 +≡
#ifdef OLDWAY
double abs (double x)
{
return (x < 0) ? (−(x)) : x;
}
#endif
187. Read the next message from the mail folder and build the set utokens of unique tokens in the
message. set insertion automatically discards tokens which appear more than once.
〈 Build set of unique tokens in message 187 〉 ≡
set〈string〉 utokens ;
while (tp.nextToken (dw )) {
utokens .insert (dw .get ( ));
}
This code is used in section 185.