The Annoyance Filter.pdf - Fourmilab

§256 ANNOYANCE-FILTER DEVELOPMENT LOG 219
to be looking for contain regular expression meta-characters, it’s a lot more convenient to specify an
explicit target than remember what they all are and quote them.
Corrected the diagnostic for an unknown character set in a header line string to say “Header line” rather
than the obsolete and misleading “Subject line” it used to say.
Added “us−ascii” to the list of character sets which require no multi-byte decoding or interpretation
when they appear in header line quoted strings. Junk mail sometimes encodes even ASCII subject lines
(and sometimes other headers) as Base64 or Quoted-Printable to hide the text from naïve filters.
Added a script to build under Cygwin, makew32.sh. Attempting to link in our own copies of getopt.c
and getopt1.c runs afoul of the Cygwin linker (why?), so I removed them from the compiles and link
done by this script.
Building on Cygwin failed because the library I was using didn’t define in_addr_t. I’d seen this earlier
on Solaris, but had inadvertently added a new reference since I’d last tested there. I changed the
offending reference (in a static_cast of all places), to our cop-out type u_int32_t, which autoconf
guarantees will always be there. With that fix, the program built and worked on Cygwin, including
POP3 proxying!
The check for non-white space following a soft line break in a Quoted-Printable MIME part failed for a
POP3 proxy message containing CR/LF line terminators. I broadened the definition of white space in
〈 Character is white space 62 〉 to include carriage return.
2002 November 3
Scribbled a first cut README.WIN file to be included in the Win32 executable archive which explains the
issues involving the included Cygwin DLL. I modified Makefile.in to include this file, the DLL, and
COPYING.GNU (the GPL) in the Win32 archive.
Tested the Win32 archive on a Cygwin-free machine. Seems to work OK, including POP3 proxy from
another machine on the LAN.
Verified that POP3 proxy on a Cygwin-free machine running Windows 98 works with the version of
Outlook furnished with that system, which can be configured to retrieve messages from ”localhost” on
our default port of 9110. Note, however, that one must first configure the account (defaulting to port
110), then edit the properties of the account, using the “Advanced” tab to specify the POP3 port of
9110.
Messages embedded within other messages with the Content−Type specification of message/rfc822 did
not have their own MIME parts correctly decoded because mailFolder failed to scan the header of the
embedded message for its own Content−Type and boundary specifications. Fixed. This should get rid
of the previously mysterious long gibberish strings which decoded out of forwarded messages with image
and other binary attachments. The strings were due to the Base64 decoder not being activated for the
embedded message’s attachments.
2002 November 5
Implemented the first cut of fast dictionary support. Having created a dictionary in memory, you can
export it to a file in fast dictionary format with the −−fwrite option. The −−fread option loads such a
dictionary and, if loaded, it takes precedence over a regular dictionary. This permits fast classification
of messages without all the overhead of creating a full-fledged in-memory dictionary.
Added memory-mapping of the fast dictionary when HAVE_MMAP is defined. In the interest of code
commonality, the header fields are read from an istrstream bound to the memory mapped block, but
access to the hash and word tables are pure pointer-whack.
Fixed a typo in configure.in which caused a harmless but ugly warning when running the script.