ePrism User Guide - EdgeWave

STA (Statistical Token Analysis)
STA (Statistical Token Analysis)
STA is a sophisticated method of identifying spam based on statistical analysis of mail content.
Simple text matches can lead to false positives because a word or phrase can have many meanings
depending on the context. STA provides a way to accurately measure how likely any particular
message is to be spam without having to specify every word and phrase.
STA achieves this by deriving a measure of a word or phrase contributing to the likelihood of a
message being spam. This is based on the relative frequency of words and phrases in a large
number of spam messages. From this analysis, it creates a table of "discriminators" (words
associated with spam) and associated measures of how likely a message is spam.
When a new incoming message is received, STA analyzes the message, extracts the discriminators
(words and phrases), finds their measures from the table, and aggregates these measures to
produce a spam metric for the message.
STA uses three sources of data to build its run-time database:
• The initial tables supplied by St. Bernard based on analysis of known spam.
• Tables derived from an analysis of local legitimate mail. This is referred to as "local learning" or
"training".
• Mail identified as "bulk" by DCC is also analyzed to provide an example of local spam.
How STA Works
Consider the following simple message:
---------------------------------------------------------------
Subject: Get rich quick!!!!
Click on http://getrichquick.com to earn millions!!!!!
----------------------------------------------------------------
STA will break the message down into the following tokens:
Get
rich
quick!!!
Click
on
http://getrichquick.com
to
123