Spam filter
Spam is mainly unwanted commercials that are sent via e-mail to a large number of recipients. The senders use various techniques to cover the fact that what they are sending is spam. Sender and recipient can easily be forged. Spam is a hassle you don't want in your mailbox.
Stockholm university (SU) or DSV are not allowed to delete e-mail sent to the departments employees since it may be legitimate e-mail which may have to be recorded. Employees should put all spam-mail in a specific folder and every once in a while have a look if any real e-mails by accident have been classified as spam.
E-mail to students of DSV are spam filtered centrally at DSV's e-mail server (though only e-mail with seven points or higher, see below).
Spam headers
Most e-mail clients have a feature for filtering e-mail. One may want e-mail from a specific sender to be placed in a certain folder. This feature can be used to put most of the spam one receives in a certain folder. This is possible since all incoming e-mail is spam controlled by the e-mail servers (mx.su.se) of Stockholm University, and are tagged with a special so called header line which indicates that it is spam. The header line looks like:
X-Spam-Flag: YES
Aside from the above mentioned X-Spam-Flag there is another header line, X-Spam-Level, which indicates how many spam points the e-mail has been given. SU currently sets X-Spam-Flag to YES if it has received seven points or more. If the level is 5, the flag looks like this:
X-Spam-Level: *****
Some e-mail filters can also check this header line which means that it is possible to choose the spam level yourself. Normally 5 points is enough to classify the e-mail as spam. Information about how this is done is available at the above mentioned SU web page.
Spam filtering e-mail with procmail
The program procmail makes it possible to have the incoming e-mail sorted in folders depending on what is written in various fields in the e-mail. This feature may then be used for sorting all spam tagged e-mail to a specific spam folder so you don't have to see them among legitimate e-mail.
The advantage of procmail is that the e-mail is filtered when it comes to the e-mail server so that your e-mail client won't have to do it every time you log on to read your e-mail. Besides, some e-mail clients do not have support for filtering, such as for instance DSV's webmail.
This is how to do it.
- Make sure all incoming e-mail is handled by procmail by putting a line like below in a file with the name .forward (note the dot at the beginning of the file name) in the root of your Unix home directory.
"|procmail -t #username"
Note that " signs should enclose the line. username should be changed for your user name. - Ensure that there is a directory named "mail" in the root of your Unix home directory. If it does not exist it may be created by executing the Unix command mkdir mail when you are in your home directory.
- Have a file .procmailrc (note here as well the dot in the beginning of the file name) in your Unix root home folder that makes spam tagged e-mail filtered to the file ~/mail/spam. The .procmailrc file should look like this:
MAILDIR=$HOME/mail DEFAULT=$MAIL LOGFILE=$MAILDIR/procmail.log LOCKFILE=$HOME/.lockmail :0: * ^X-Spam-Flag: YES spam
-
If you want to use the vacation program to send automatic replies while you are on vacation, or similar, add the following two lines to your .procmailrc file after the spam filtering above:
:0 c | /usr/bin/vacation your_login_name
Note that the .forward file in this case should not be updated with start of vacation. That is, do not use the vacation program to turn it on.
- If you also want your non-spam tagged e-mail to be sent to another e-mail address, for example your FirstClass mailbox "user-name@fc.dsv.su.se", add the following rule definition at the end of your .procmailrc file:
:0: ! user-name@fc.dsv.su.se
Note that the files that are created must be in Unix format. That is, every line is ended with just a LF character (Line Feed). If the files are created on MS Windows every line is usually ended with the two characters CR LF (Carriage Return followed by Line Feed). To convert a Windows file to Unix format one can give the command flip -u file_name when logged in to a Linux computer, such as the Linux server triton.dsv.su.se.
Spam filtering with SpamProbe
If you are not happy with the spam filtering done by SU's central spam tagging with SpamAssassin it is possible to do a refined filtering with SpamProbe. This software uses so called Bayesian filtering and can be tought which e-mail is spam and which is not. Below are instructions for using SpamProbe.
- Log in to one of DSV's Linux computer, for example the Linux server triton.dsv.su.se.
- Create a directory for SpamProbes database with the command mkdir .spamprobe
- Use the program spamprobe to teach it what is spam and what is not. If you have used spam filtering with SpamAssassin you should have spam e-mail saved in for instance mail/spam. Real e-mail may be saved in different folders, such as mail/work, mail/research, mail/private and so forth. Now give the command:
spamprobe good mail/work spamprobe good mail/research
and so forth
spamprobe spam mail/spam
and similar for other folders with spam. - Next, update your .procmailrc file. Add the filtering with SpamProbe after the SpamAssassin filtering with X-Spam-Level but before any sorting of e-mail. A .procmailrc file could look like:
MAILDIR=$HOME/mail DEFAULT=$MAIL LOGFILE=$MAILDIR/procmail.log :0: * ^X-Spam-Level: \*\*\*\*\* spam :0 SCORE=|spamprobe train :0 wf |formail -I "X-SpamProbe: $SCORE" :0 a: * ^X-SpamProbe: SPAM spamprobe :0: * ^List-Post: <mailto:debian@lists.debian.org> debian
What SpamProbe considers to be spam will be placed in the folder spamprobe in this example. One should occasionally check so that real e-mail doesn't end up there. In this case you should move them to a specific folder with legitimate e-mail which you feed to SpamProbe with spamprobe good ....
Spam filtering with Eudora
This is how you filter spam tagged e-mail with the e-mail client Eudora when downloading them from the e-mail server:
- Start Eudora
- Click on Tools
- Choose Filters
- Click on New
- Enter "X-Spam-Flag:" in the Header field
- Choose "contains" and then enter "YES"
- Next, choose what should happen with the filtered e-mails. Choose for instance "Transfer to:" and then "Trash"
- Close the Filter window
Links with more information
- spamassassin.org, Stockholm University uses the software SpamAssassin to spam tag incoming e-mail.
- spamprobe.sourceforge.net
- DSV's web pages (in Swedish) about filtering e-mail with Procmail and more
