building a better spam trap

I've spent a lot of time over the past couple of months trying out some new (and some not so new) anti-spam techniques. Note that this article assumes some familiarity with sendmail m4 macros; see $CFDIR/README for background and all sorts of details on these, where $CFDIR is one of:
  • /etc/mail/cf on Solaris 10
  • /usr/lib/mail on Solaris 7, 8 or 9
  • the cf sub-directory of the sendmail distribution for people "rolling their own"

These techniques are in the form of FEATURE and HACK m4 macros (the difference being that the former are provided and blessed by sendmail.org / Solaris whereas the latter are not, though a HACK may evolve into a FEATURE in a future release). For a HACK, one would use

        HACK(`hack-name')dnl
in one's .mc file, likewise
        FEATURE(`feature-name')dnl
When installing hacks, one must create $CFDIR/hack (if it does not already exist) and place hack-name.m4 in that directory. Note that the sendmail distribution comes with such a sub-directory but Solaris does not.

Also, to explain some terms used below: the access list is enabled by the FEATURE(`access_db') macro; details on this are in $CFDIR/README, both in its sub-section in the FEATURES section, and in the ANTI-SPAM CONFIGURATION CONTROL section. And FEATURE(`delay_checks') is strongly recommended, as it is needed to enable the overrule by an OK entry in the access list that I mention in a few places; this feature is also described in its subsection in the FEATURES section, as well as in the "Delay all checks" sub-section of the ANTI-SPAM CONFIGURATION CONTROL section.

Anyway, onto the details. In the order I started deploying them:

  • The first is HACK(`block _bad_helo'), written by Neil Rickert, a professor in the Computer Science Department at Northern Illinois University and a volunteer sendmail.org contributor. SMTP clients are supposed to send the client FQHN (fully qualified host name) as the HELO/EHLO parameter, but many broken clients send the server FQHN (or IP address) instead, or something without a ".". This rejects any such transmissions. The upside is that I have found it to block a good amount of spam, with no false positives for me. A couple of users of my personal domain have had some small number of false positives with it, though. And the down side is that it cannot be overruled by an OK entry in the access list. Bart has had a lot of troubles with this rule; apparently old versions of Netscape and early version of Mac's Mail.App got this wrong.
  • The second is a regular feature: DNS-based black-lists. I use (note: line wrapped for readability)
      FEATURE(`enhdnsbl', `bl.spamcop.net',
            `"Spam blocked see: http://spamcop.net/bl.shtml?"$&{client_addr}',
            `t')dnl
      
    while Bart uses
      FEATURE(`dnsbl', `sbl-xbl.spamhaus.org')dnl
      
    Both have proven extremely effective with very few false positives, and this feature, using whichever list, has the added virtue of allowing an OK override in the access list.
  • The last is HACK(`require_rdns'), also written by Neil Rickert. I enhanced Neil's original version so that it would allow an OK override in the access list. This enhanced version has been unbelievably effective, in sheer numbers, while also so far amazingly accurate (I estimated a false positive rate of 0.5% after a few weeks, but I think that may have gone even lower since I white-listed a few sites). As the name suggests, it requires that the SMTP client's IP address reverse map to some name, and also that the name forward map to the same address. (An IP address can have multiple A records; this merely requires that the original IP address is one of them.) This is the single most effective anti-spam rule I have ever deployed.

Overall, spam getting thru my personal domain's mail server to my users (including myself, my wife, my siblings, our mom, etc.) has dropped about 90% since I started using these techniques, despite the ever-increasing spam trends on the rest of the Internet.

Comments:

I'm \*very\* interested in the last of these two - I run sendmail to handle the email for my personal domain, and although the Thunderbird adaptive spam filtering works quite well, I'd prefer to cut the spam off earlier. Any chance of an Idiot's Guide of exactly how to set this on on S10? I've read the comments in require_rdns.m4 and I'm not quite sure what 'access table entries' are, or what FEATURE(`delay_checks') does, and why I would want it. Thanks!

Posted by Alan Burlison on January 26, 2005 at 07:13 PM PST #

Off topic, I just implemented bogofilter with tie-ins to my MUA (mutt) and my MDA (procmail). W0rks pretty darn cool.

Posted by sbank on January 30, 2005 at 11:12 PM PST #

Why don't you use Postfix?

Posted by guest on April 19, 2005 at 12:05 AM PDT #

Hi,

The require_rdns.m4 file seems to have gone AWOL!

Rgds

Damon

Posted by Damon Hart-Davis on September 01, 2006 at 10:53 PM PDT #

Just found something odd, when you have: HACK(`dnswl',`nlwhitelist.dnsbl.bit.nl')dnl HACK(require_rdns)dnl My thoughts would be whenever an ip is listed in the whitelist rbl, it should be excluded from the require_dns hack. Atm whitelisted ips without rdns are also being blocked. Any ideas?

Posted by Gerwin on October 09, 2006 at 07:57 PM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

jbeck

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today