802 copies of dccm running

Vernon Schryver vjs@calcite.rhyolite.com
Mon Jan 1 16:32:37 UTC 2007


> From: Gary Mills 

> It occurred to me that even though the RBL lookups would be extremely
> quick, `dccm' would have to convert many hostnames into IP addresses
> before looking them up in the RBL. 

`dccm` deals with the host names one at a time, converting each to an
IP address and then checking the IP address in an RBL.  Assuming you
run a local caching DNS server (including negative caching), duplicate
host names will be fast.  While email contains many different URLs,
there are not many in the grand scheme of tyical DNS caching and serving.


>                                     The hostname lookup could be very
> slow and would time out. 

`dccm` limits the time spent for all DNSBL work for a single message.
You can change that limit with -Bset:msg-secs=XX


>                           So, I did a triage by setting `no-body' and
> `no-MX' within DNSBL_ARGS in dcc_conf.  The difference was dramatic!
> The number of `dccm' processes stabilized at 5.  It does go higher
> when the traffic increases, but it settles back to 5. 

If you are only checking the SMTP client IP address, then that is
probably best done by sendmail with FEATURE(`enhdnsbl'...) and applying
/var/dcc/libexec/hackmc to the mc/cf file so that dccm will know about
sendmail's DNSBL hit.

>                                                        Now I have to
> wait for a busy e-mail day to see how high it goes.  I'm only using
> the XBL blocklist just now, so really only the SMTP peer address needs
> to be checked.

On othe contrary, Spamhaus and I both recommend checking URLs in mail
bodies against the XBL.  I see plenty of XBL hits on URLs in spam bodies.
I understand that the CBL component of the XBL needs perhaps half a day
to detect a new trojan.  Checking URLs against the SBL component of the
XBL helps during that gap,


> No, but I did run `truss' on some of the DNS helper processes.  There
> were many lines like this:
>
> poll(0xFFBFEAD8, 2, 60000)      (sleeping...)
> poll(0xFFBFEAD8, 2, 60000)                      = 1
> recvfrom(37, 0xFFBFEEA8, 1936, 0, 0xFFBFECF8, 0xFFBFECF4) Err#11 EAGAIN

I suspect that and all other idle helpers were awakened by a request from
the main `dccm` process for a DNS resolution.  Some other helper won
the race to recvfrom(), so this one went back to sleep.

Perhaps I should switch to a blocking socket and use a timer to reap
excess helpers.  That waking up to lose the race and go back to sleep
is wasteful.  The next version of dccm will reap excess helpers much
more agressively, which would also help.


> poll() behavior.  The poll() system call is used internally by the
> select() function.  File descriptor 37 is a UDP socket which, I

In some circles in decades past, a selling point of poll() over select()
was that only one of the waiting processes needed to be awakened and
avoiding that wasteful rush.


Vernon Schryver    vjs@rhyolite.com



More information about the DCC mailing list

Contact vjs@rhyolite.com by mail or use the form.