mail loops with dspam/exim4

[sethg-Dp9fwfP21SfQT0dZR+AlfA at public.gmane.org (Seth Gordon)]

My personal mail server is a Debian VPS that uses exim4 as its MTA and 
dspam as the spam filter.  We use the dovecot antispam plugin 
(http://johannes.sipsolutions.net/Projects/dovecot-antispam) so that 
fixing a misclassified message is as easy as moving it in or out of our 
?auto-spam? folder.  A script runs at 1:00 am to expunge old messages 
from that folder.  We use Mozilla Thunderbird to read mail.

Every once in a while, I see the performance on the machine grind to a 
halt, with the load as high as 20, and a massive number of exim 
processes.  Running mailq shows a large number of messages in the system 
whose sender and recipient are both dspam-Dp9fwfP21SeXj1p+fO2waQ at public.gmane.org  I?ve tried 
shutting down the exim4 server and cleaning these out by hand, but 
generally by the time I run ?exim4 -Mrm <message-ID>?, the message in 
question has already been delivered and a new attempt made.  Eventually, 
the same message would get cycled through enough times that exim4 would 
detect that there was a mail loop and give up.

A representative sample from /var/log/exim4/mainlog, when this problem 
is rearing its ugly head, is like this:

> 2010-06-27 03:27:42 1OSmHD-0002TB-6m => dspam <dspam-Dp9fwfP21SfQT0dZR+AlfA at public.gmane.org> R=spamcheck_director T=spamcheck
> 2010-06-27 03:27:42 1OSmHD-0002TB-6m Completed
> 2010-06-27 03:27:42 1OSmHD-0002TA-6c => dspam <dspam-Dp9fwfP21SfQT0dZR+AlfA at public.gmane.org> R=spamcheck_director T=spamcheck
> 2010-06-27 03:27:42 1OSmHF-0002TZ-Ge <= dspam-Dp9fwfP21SfQT0dZR+AlfA at public.gmane.org U=dspam P=local S=32580 id=E1OSmGa-0002OC-1r at localhost
> 2010-06-27 03:27:42 1OSmHF-0002Ta-Go <= dspam-Dp9fwfP21SfQT0dZR+AlfA at public.gmane.org U=dspam P=local S=19017 id=E1OSmGb-0002OJ-Px at localhost
> 2010-06-27 03:27:42 1OSmHD-0002TA-6c Completed
> 2010-06-27 03:27:42 1OSmHF-0002TZ-Ge ** dspam-Dp9fwfP21SfQT0dZR+AlfA at public.gmane.org: Too many "Received" headers - suspected mail loop
> 2010-06-27 03:27:42 1OSmHE-0002TO-HF => dspam <dspam-Dp9fwfP21SfQT0dZR+AlfA at public.gmane.org> R=spamcheck_director T=spamcheck
> 2010-06-27 03:27:42 1OSmHE-0002TO-HF Completed
> 2010-06-27 03:27:43 1OSmHE-0002TP-I4 => dspam <dspam-Dp9fwfP21SfQT0dZR+AlfA at public.gmane.org> R=spamcheck_director T=spamcheck
> 2010-06-27 03:27:43 1OSmHE-0002TP-I4 Completed
> 2010-06-27 03:27:43 1OSmHG-0002Tm-K5 <= dspam-Dp9fwfP21SfQT0dZR+AlfA at public.gmane.org U=dspam P=local S=19249 id=E1OSmGb-0002OJ-Px at localhost
> 2010-06-27 03:27:43 1OSmHG-0002Tm-K5 ** dspam-Dp9fwfP21SfQT0dZR+AlfA at public.gmane.org: Too many "Received" headers - suspected mail loop
> 2010-06-27 03:27:43 1OSmHG-0002Ti-JQ <= <> R=1OSmHF-0002TZ-Ge U=Debian-exim P=local S=33336
> 2010-06-27 03:27:43 1OSmHF-0002Ta-Go => dspam <dspam-Dp9fwfP21SfQT0dZR+AlfA at public.gmane.org> R=spamcheck_director T=spamcheck
> 2010-06-27 03:27:43 1OSmHF-0002Ta-Go Completed
> 2010-06-27 03:27:43 1OSmHH-0002Tp-Ap <= <> R=1OSmHG-0002Tm-K5 U=Debian-exim P=local S=20005
> 2010-06-27 03:27:43 1OSmHF-0002TZ-Ge Completed
> 2010-06-27 03:27:44 1OSmHH-0002Tu-BQ <= dspam-Dp9fwfP21SfQT0dZR+AlfA at public.gmane.org U=dspam P=local S=33568 id=E1OSmHG-0002Ti-JQ at localhost
> 2010-06-27 03:27:44 1OSmHG-0002Tm-K5 Completed

When the dust cleared from all this, my wife?s ?auto-spam? folder 
(never, as far as I?ve noticed, mine) could have thousands of messages 
in it, because certain spam messages appeared about a dozen times over. 
  My wife has complained that moving misclassified messages into 
?auto-spam? is often painfully slow, and that dspam seems to be doing a 
lousy recognition job (e.g., a lot of messages containing That Word 
Beginning With V are being passed through as legit), and I suspect that 
these spurious copies are screwing up dspam?s statistics-gathering 
operation; I also suspect that the multiple copies and the mail loops 
have the same cause, although for all I know they are two separate 
problems and I just notice them at the same time.

I?ve tried switching dspam from the hash-based to the mysql backend, and 
various other configuration changes, and, well, I?m tired of just 
panicking every time the problem becomes noticeable and twiddling the 
first thing that comes to mind and hoping that it makes everything better.

I have noticed that the router/250_dspam_spamcheck_director file, 
provided with the dspam debian package, looks like this:

> # DSPAM
> spamcheck_director:
>   driver = accept
>   check_local_user
>   condition = "${if and {\
>                          {!def:h_X-DSPAM-Result:}\
>                  }{1}{0}}"
>   headers_add = "X-DSPAM-Check: by $primary_hostname on $tod_full"
>   transport = spamcheck
>   no_verify

...and the sample dspam_router on 
http://dspamwiki.expass.de/Distribution_Specific/DSPAMOnDebianHOWTO is 
more complicated.  Should I be adding ?{!def:h_X-DSPAM-Check:}? to that 
condition statement above?

Or is there something else I should be trying?