The Annoyance Filter.pdf - Fourmilab

218 DEVELOPMENT LOG ANNOYANCE-FILTER §256
Added a −−pop3server option to specify the server and optionally, port (which defaults to 110 if not
given) to which the POP3 proxy server will connect. This must be the last option (a warning is given
if it isn’t), and causes the server to immediately begin operation. I removed the server test code from
the −−jig and physically moved it to a subsection within the “POP3 proxy server” section, following
the class definition.
2002 October 31
Disabled the −−jig, since there’s nothing in it at the moment.
Added proper conditional setting of POP3_PROXY_SERVER based on the capabilities sensed by autoconf
and fixed one compile problem if the proxy server is disabled. At the moment, we assume that if socket
and signal are defined, everything else we’ll need will also be defined
2002 November 1
Cleaned up POP3 proxy code and added documentation of the related command line options. I still
need to add a main document section on how to install and operate a proxy server.
2002 November 2
We weren’t activating the byte stream parser for spoofed mail worm attachments which trick Microsoft
Outlook into executing an attachment through the incredibly subtle strategem of declaring the attachment
as an innocuous file type such as audio or image, but with an extension which denotes an
executable file. Brain-dead Outlook decides whether to block or confirm executable content based upon
the former, but then actually executes the file based upon the latter. Can you say “duh”?
Well, thanks to this particular piece of Redmond rot, tens of millions of these worms continue to
pollute the net since, even though the hole has been plugged, millions of the bottom-feeders who use
such software continue to use unpatched versions and/or run machines which are already infected and
actively propagating the worm.
All right, enough polemic. What this means for annoyance−filter is that when we see an attachment
with a Content−Type which usually denotes something we’re not interested in parsing, but then discover
its file name is one of the suspicious executable Microsoft file types, we need to feed it through the byte
stream parser just as if it were tagged with an “application” file type. Doing so will extract the
inevitable embedded strings, which will act as a signature for subsequent encounters with the same or
similar worm. (SourceForge bug 631503, reported by Neil Darlow.)
Improved diagnostics for parser errors by saving the “From␣ line and Message−ID (if any) from the
header and then labeling any parser diagnostics written to standard error with the −−verbose option
with them. The labels are written only before the first diagnostic for each message in a folder, and
diagnostics are now indented to better diatinguish them from the labels.
Diagnostics from MBCSdecoder objects were written to standard error without any identification
of the message in which they occurred. I added the ability to link an MBCSdecoder to its parent
mailFolder with the new setMailFolder method. If linked, diagnostics from the decoder are emitted via
the reportDecoderDiagnostic method of the linked folder, permitting them to be labeled with the message
identification as described in the previous paragraph. It’s still possible to use an MBCSdecoder
without linking it to a mailFolder—if the link is Λ, diagnostics are sent to standard error as before.
Improved diagnostics from the various MBCSdecoder classes. All reports of invalid two-byte sequences
now report both hexadecimal bytes, and other invalid value diagnostics report the offending hexadecimal
value.
Added the ability to search for a literal substring as well as a regular expression in utilities/splitmail.pl.
If the search target begins with “+” (which is invalid in a regular expression), the balance of the pattern
is searched for with case-insensitive comparison. Since so many of the message headers you’re likely