SPAM Reduction

Background

The U.S. Naval Cryptologic Veterans Association strongly discourages production and forwarding of unwanted email solicitations. As a result, the Postmaster for usncva.org has activated SpamAssassin (tm).

SpamAssassin is an extensible email filter which is used to identiy spam.

Using its rule base, it uses a wide range of advanced heuristic and statistical analysis tests on mail headers and body text to identify "spam", also known as unsolicited bulk email.

Spam Identification Tactics

The spam-identification tactics used include:

• header analysis: spammers use a number of tricks to mask their identities, fool you into thinking they've sent a valid mail, or fool you into thinking you must have subscribed at some stage. SpamAssassin tries to spot these.
• text analysis: again, spam mails often have a characteristic style (to put it politely), and some characteristic disclaimers and CYA text. SpamAssassin can spot these, too.
• blacklists: SpamAssassin supports many useful existing blacklists, such as mail-abuse.org, ordb.org, SURBL, and others.
• learning classifier: SpamAssassin uses a Bayesian-like form of probability-analysis classification, so that a user can train it to recognise mails similar to a training set.
• distributed hash databases: Vipul's Razor is a collaborative spam-tracking database, which works by taking a signature of spam messages. Since spam typically operates by sending an identical message to hundreds of people, Razor short-circuits this by allowing the first person to receive a spam to add it to the database -- at which point everyone else will automatically block it.

Filtering with an email client

Once identified, the mail is tagged as spam for later filtering using the user's own mail user-agent application (i.e., Microsoft Outlook or Outlook Express, Eudora, etc.). Identified email is tagged by modifying the subject line of the email.

*****SPAM***** Citizens Bank REMINDER: PLEASE UPDATE YOUR DATA

The "*****SPAM*****" part of the subject line is added by SpamAssassin. Users can setup a filter to move emails that have subject lines starting with *****SPAM***** into a temporary or deletable folder within their email client.

SpamAssassin typically differentiates successfully between spam and non-spam in between 95% and 99% of cases, depending on what kind of mail you get. I do not recommend automatically deleting email tagged as SPAM. You could be throwing out valid email. I review all messages redirected by my email client prior to manually deleting the real spam.

Last Modified: Sunday, 14-Nov-2004 08:50:03 EST