6. Mail Filtering

6.1. .maildelivery

This is used with the MH family of mail readers. It is not real sophisticated, but it's useful enough if your needs are limited.

6.2. procmail

procmail is included with all distributions of Linux that I know of.

6.3. Ifile

This works in conjunction with MH/EXMH, providing a filter that examines your message refiling patterns in order to determine where future mail ought to go.

• knowledge_base.perl

  Searches thru your mail folders, counting the frequencies of words and messages. Infrequently used words are discarded, and the statistical results are dropped into a file, .idata

• irefile

  Use irefile when a message is misclassified. It does two things:

  • Moves the message to the right folder

  • Dribbles frequency statistics into .idata.queue

• data_collector

  Updates .idata by adding in frequency information from .idata.queue

• ifilter

  Reads the message, computes a "relevance value" for each folder based on the statistics in .idata, and files the message in the folder with the best "relevance value."

  Relevance is based on the log function so that high occurances of common words (like the or and) are appropriately discounted.

  Text search systems commonly use "stop lists," removing these common words because they just make queries messy. Since Ifile uses the entire text of the incoming message as a "query key," even members of a typical "stop list" can usefully contribute to the result.

6.4. Other Email Filtering...

• The BlackMail Anti Spam Mailer Daemon

• Spam and Junk Email: Against the Onslaught

• Tagged Message Delivery Agent (TMDA) Homepage

  TDMA takes the approach of only accepting mail from those willing to jump through the hoop of a quasi-authentication process.

  All mail coming from non-authenticated sources gets queued, and a message is sent back asking them to authenticate themselves. Spammers certainly won't bother; those that do go onto the "authenticated" list, and their messages will be passed straight on through.

• zoe

  The "email client" equivalent to Google; written in Java , targeted primarily at MacOS-X, with some ability to run on other Java-supported platforms.

• Sieve Home Page

• Sieve: A Mail Filtering Language - RFC 3028

• Mailfilter

  Mailfilter allows the gentle user to apply a set of rules to the headers of mail in their POP3 mailbox, deleting messages that match particular rules. It can function in a " rule-based" as well as in a " score-based" manner.

  It is not nearly as good a classifier as (say) Ifile , but has the merit that it can expunge mail before you download it. I found this extremely useful in the wake of 2003's Swen-32 email virus that was pummelling my mailboxes with exorbitant quantities of email viruses. My systems may be immune to the viruses, but that doesn't mean I want to consume the bandwidth to download the messages.

• maildrop - mail delivery agent with filtering abilities - part of Courier

• Mailagent

• SpamAssassin

6.5. Email Filtering using Ifile

ifile uses a statistical approach called Naive Bayesian Filtering directed (at this point) at filtering mail into MH mail folders.

Ifile was designed for use in conjunction with MH for email. I have written Perl scripts that go through a news spool and pass news messages on to ifilter. I'm still in the experimental stages on this; at present it doesn't handle outgoing news very well. But it does provide fairly good message classification. Various forms of "spam" get quite accurately classified as such, so that it takes only a quick browse to "nuke the spam." I still have to complete the circle, turning at least portions of this back into news. (Maybe it's time to try out GNUS. )

6.6. Other Email Filtering Links

• Searching and Filtering

• Freely Available Information Filtering Systems

• The Spam Letters

  A set of amusing responses to commonly-received spam.

• Scam-O-Rama