The Annoyance Filter.pdf - Fourmilab

224 DEVELOPMENT LOG ANNOYANCE-FILTER §256
are not supported, as they were a disastrously poor idea—you can generally treat them as usual UNIX
folders. By default, folders are parsed using UNIX semantics. A new −−bsdfolder option marks the
following −−mail or −−junk folder as following BSD rules. Note that you must specify −−bsdfolder
before each BSD-style folder; it is not modal. This is a change in default behaviour: folders were
previously parsed using BSD rules, while UNIX is now the default.
The very large case statement which processes command line options ran afoul of CWEAVE’s maximum
token per scrap capacity limit. I added a cweb/cweave−bigger.ch file to increase the limit to 5000
tokens (from 2000), and modified cweb/Makefile to apply the change file when building CWEAVE. I
probably ought to break the option processing case into one piece for each option, but as there’s little
or nothing to be said about each one, that really wouldn’t improve the readability of the code.
2003 February 20
Completed the implementation of −−autoprune. This new option permits you to specify a memory
size, in bytes, at which a dictionary to which words are being added with the −−mail or −−junk
options will be automatically be pruned by discarding all words which appear only once. A new
dictionaryWord ::estimateMemoryRequirement method estimates the memory occupied by an inmemory
word, and this is used to compute the total dictionary size. dictionary ::purge has been
extended to accept an optional argument which, if nonzero, causes the pruning of the dictionary to be
based on the number of occurrences of the word rather than our ability to compute its probability.
If the user sets −−autoprune too low, we can fall into a trashing situation when the non-unique words
in the dictionary exceed the pruning threshold. To keep this from happening, whenever the dictionary
size after an automatic prune exceeds 90% of the −−autoprune threshold, the threshold is increased by
25%.
2003 February 21
Modified the makew32.sh script to build with gcc 3.x rather than 2.x. Note that this means the
source should be ./configured for a gcc 3.x build before creating winarch to transport to the Cygwin
machine.
When building on Cygwin with gcc 3, getopt.h managed to get included twice for some reason. I
changed the condition around our local copy to __GETOPT_H__ to agree with the symbol in the library
include to prevent this from happening.
Updated the cygwin.dll included in the Win32 executable distribution to the January 24, 2003 version
we’re currently using on Ovni.
Release 1.0.
2003 June 24
As reported by and fixed by Wolfgang Schnerring, utilities/splitmail.pl had an assignment statement
in the dispose_of_message subroutine which was missing the dollar sign before the variable
name. I integrated his fix. Thank you!
2003 August 27
A pdfTextExtractor was not restartable—once instantiated, it could only be used once; calling close
and then re-initialising with the parent applicationStringParser class setMailFolder left the extractor
at end of file. This required fixes both in pdfTextExtractor, where the close method failed to reset
initialised to false , and in applicationStringParser, whose close method did not reset the eof and
error flags.
2003 August 28