more spam filtering ideas

[richb at pioneer.ci.net (Rich Braun)]

Tom Metro <blu at vl.com> wrote:
> A description of spam filtering techniques used by an ISP that claims to
>   have "the most advanced Spam and Virus Filter on the Planet!"
>
> ...he's using Exim...

Interesting article.  Too bad he doesn't post any of samples of his work: but
his descriptions are on-target, at least if you're operating as a small ISP or
hosting provider (as opposed to my simple home setup) such that you have
multiple IP addresses.  In fact you could probably do everything he does if
you simply make an arrangement with a friend to run backup MX for one another
(caveat--you'd have to add some rules or state-machine logic to deal with the
fact that the two machines might not be up simultaneously).

When I was writing my rules I thought about implementing the concept that this
ISP (Marc) calls the Penalty Box, which he apparently uses as a replacement
for rather than a supplement to greylisting.  I decided that a 45-minute
greylist interval was good enough, not that big of an inconvenience (delay on
incoming mail from new correspondents) and in fact greylisting would work
almost as well if you cranked down the delay to 5 or 10 minutes.

Sender callback verification is another interesting technique that he brings
up.  I am not sure what percentage of spam would get trapped by it--my stats
are already so good that it seems unnecessary.

One thing I do that he doesn't is reject all messages listed at spamhaus:  I
do not pass from exim to spamassassin, senders get told to go away.  The list
there is good enough, IMHO, that if anyone tried to send me a message from a
listed server, I don't mind having them see a "try sending from some other
email address" rejection message from me.

In the time that I have been running exim (several months), I have not seen
much growth in the two spam-control techniques backed by the big guys:  SPF
and DomainKeys.  SPF is used by a much larger number of ISPs than DomainKeys
(which is currently run only by Yahoo/Google) but its popularity is not yet
spreading.

The most-useful technique that I use which Marc doesn't mention is rejecting
all messages which lack a Message-ID header.  I also use Razor2 to reject
messages rather than pass them to SpamAssassin.

One other interesting thing to note--growth in spam seems to have stalled for
half a year.  Either my email address is getting culled from spammers' lists
(doubtful) or growth in the industry has finally slowed down.  Stats:  I get
about 400 to 500 spam attempts per day; over the past 80 days since 8/1, of
the ones that were not rejected, 216 were put into the quarantine folder and
146 were passed to my inbox--of the latter, about 45 were untagged (overall:
better than 99.8% accuracy).  I have found no false-positives except for
newsletters and opt-in marketing stuff, and none of my correspondents have
reported problems reaching me (at least, since after that initial week or so
of tweaking).  The server is a $300 home-built econobox which is running more
than 99% idle (in fact it is much more idle than before when I was running
SpamAssassin alone without exim rejecting most junk).

Unlike the material in use at junkemailfilter.com, mine is freely available
for use by anyone:  look for my prior postings in the BLU archives.  I will
post more if asked.

-rich