The Annoyance Filter.pdf - Fourmilab

§19 ANNOYANCE-FILTER DICTIONARY 25
19. Dictionary.
A dictionary is a collection of dictionaryWord objects, organised for rapid look-up. For convenience and
efficiency, we derive dictionary from the STL map container, thereby making all of its core functionality
accessible to the user. It would be more efficient and cleaner to use a set, but objects in a set cannot be
modified; values in a map can.
〈 Class definitions 10 〉 +≡
class dictionary : public map〈string, dictionaryWord〉 {
public:
unsigned int memoryRequired ;
void add (dictionaryWord w, dictionaryWord ::mailCategory category ); void
include (dictionaryWord &w) ;
void exportCSV (ostream &os = cout );
void importCSV (istream &is = cin );
void computeJunkProbability (unsigned int nMailMessages , unsigned int nJunkMessages , double
mailBias = 2, unsigned int minOccurrences = 5);
void purge (unsigned int occurrences = 0);
void resetCat (dictionaryWord ::mailCategory category );
void printStatistics (ostream &os = cout ) const;
#ifdef HAVE_PLOT_UTILITIES
void plotProbabilityHistogram (string fileName , unsigned int nBins = 20) const;
#endif
void exportToBinaryFile (ostream &os );
void importFromBinaryFile (istream &is );
unsigned int estimateMemoryRequirement (void) const
{
return memoryRequired ;
}
dictionary( )
: memoryRequired (0) { } } ;
20. The add method looks up a dictionaryWord in the dictionary. If the word is already present, its
number of occurrences in the given category is incremented. Otherwise, the word is added to the dictionary
with the occurrence count for the category initialised to 1.
〈 Class implementations 11 〉 +≡
void dictionary ::add (dictionaryWord w, dictionaryWord ::mailCategory category )
{
dictionary ::iterator p;
if ((p = find (w.get ( ))) ≠ end ( )) {
p ⃗ second .add (category );
}
else {
insert (make pair (w.get ( ), w)).first ⃗ second .add (category );
memoryRequired += w.estimateMemoryRequirement ( );
}
}