The Annoyance Filter.pdf - Fourmilab

§144 ANNOYANCE-FILTER MAIL FOLDER 125
144. A clever junk mail author might try to evade filtering based on the header items we include in
the −−transcript by including his own, on the assumption that a downstream filter would not detect
the multiple items and filter on the first one it found. To prevent this, and to make it more convenient
when feeding transcripts back through the program (for testing the effects of different settings or for
training on new messages), we detect header lines which begin with our Xfile sentinel and completely
delete them from the transcript. The isSpoofedHeader flag causes continuation lines, if any, to be deleted
as well. (At this writing we never use continuations of our header items, but better safe than sorry.)
〈 Check for lines with our sentinel already present in the header 144 〉 ≡
bool isSpoofedHeader = false ;
if (inHeader ) {
string sc = s, scx = Xfile ;
}
stringCanonicalise (sc);
stringCanonicalise (scx );
scx += ’−’;
if (sc.substr (0, scx .length ( )) ≡ scx ) {
if (tlist ≠ Λ) {
tlist ⃗ pop back ( );
}
isSpoofedHeader = true ;
}
This code is cited in section 256.
This code is used in section 143.
145. When processing mail folders in bulk, as when generating a dictionary, we want to identify parser
diagnostics with the message which they refer to. While processing the header, we save the Message−ID
tag, which which reportParserDiagnostic prefixes the message in its −−verbose mode output. Messages
which lack a Message−ID header item must be identified from the “From␣” line. RFC 2822 specifies
that Message−ID should be present, but is an optional field.
〈 Save Message-ID for diagnostics 145 〉 ≡
{
string arg ;
}
if (inHeader ∧ compareHeaderField (s, "message−id", arg )) {
messageID = arg ;
lastMessageID = "";
}
This code is used in section 141.