DCC - Postfix MTA mail hub implementation question

Vernon Schryver vjs@calcite.rhyolite.com
Fri Aug 23 16:53:03 UTC 2002

> From: Matt Armstrong <matt@lickey.com>

> ...
> >The worrisome cost of such designs is that every mail message requires
> >at least one fork() and exec() for dccproc and for all I know, more
> >for however postfix gets the result from dccproc
> >
> For postfix the dominant cost is not fork/exec but management of the 
> queue files.

Depending on the operating system and file system, fork/exec can
cost more than creating a file or two.

>              The postfix content filtering scheme involves the mail 
> totally leaving the postfix universe (it is effectively "delivered" to 
> the content filter), so the queue file is deleted. The content filter 
> reinjects the message, so a new queue file is created and synced to 
> disk. So the performance will be roughly cut in half given that disk I/O 
> is the bottleneck for SMTP servers.

Not all SMTP servers are limited by disk I/O.  However, that filtering
tactic does sound like a good way to double whatever costs are the limit.

> ...
> >       daemon and the result on output, wand the bodies delimited
> >       probably by a leading byte count.  (RFC 821 escaping sounds like
> >       more trouble for all concerned.)
> >
> I think the SMTP protocol itself is sufficiently powerful. 

SMTP "transparency" (see section 4.5.2 of RFC 821) is certainly
sufficiently powerful to handle anything that arrives via SMTP,
particularly when you "just send 8 bits".  The trouble is that
scanning for "." is expensive for both sender and receiver.

>                                                            A permanent 
> DCC daemon would run as a postfix content filter. Postfix would 
> "deliver" the message via SMTP to the DCC daemon. The daemon would do 
> its thing, possibly rejecting the message with a 5XX SMTP error or 
> possibly a 4XX SMTP error (if, say, the DCC server isn't answering). If 
> the message pases the DCC thresholds, the daemon would reinject the 
> modified message back into postfix via SMTP on a different port.

Oh, you mean literally use SMTP?  Well, I'm not enthused about the
notion of writing an MTA.  Not only do I wonder if would save many
CPU, disk, or other resources, the effort seems redundant.  One could
use sendmail+dccm as the "DCC daemon."  

It wouldn't hurt enough to matter to use a restricted form of Rcpt_To
and Mail_From commands to convey most of the envelope (i.e. no "optional
<mail-parameters> [that] are associated with negotiated SMTP service

An additional problem is that the SMTP protocol is not sufficent,
because there is no way to represent the IP address of the real SMTP
client.  (Parsing a locally generated Received line doesn't count.)

> SMTP command pipelining basically eliminates concerns about the 
> efficiency of the protocol, especially when you consider that the 
> dominant cost for postfix is disk I/O.

That makes assumptions about costs that where true, argue against
doing anything for postfix+dccproc.

>                                        Consider that many people run 
> every message through a virus scanner already and many people have 
> plenty of capacity on their SMTP servers. 

That is a compelling argument against doing anything for postfix+dccproc.

>                                           These folks would find this 
> approach doable. Those that have a huge message volume already would 
> have to add more hardware, though they may still find postfix+DCC 
> content filter faster than sendmail+dccm (some claim that postfix is so 
> much faster than sendmail that a 2x performance hit would still be 
> acceptable).

I'm very skeptical of claims about the speed (or security) of the
free alternatives to sendmail.  There are alternatives to sendmail
that I understand to be faster, but they are not free.

> Something like this has the best chance of getting accepted upstream 
> because it has no chance of compromising postfix's security model. In 
> fact it requires no changes to postfix so upstream's acceptance is moot.
> The other option is to patch postfix to support a DCC specific protocol, 
> but that has little chance of getting accepted upstream as it isn't a 
> generic solution to the "I want to process, possibly modify, possibly 
> reject all messages that pass through my server with algorithm Foo" problem.

It also has little chance of getting accepted "downstream," if I
understand what is meant in this context by "upstream."  The things
that might almost be considered sendmail patches in the public DCC
source (the FEATURE() files, the hackmc script, and the site configuration
file)) are at or beyond what I consider reasonable.

Would any of the other cases that use dccproc be able to use a reasonable
protocol, and if so, what sort?  
Again, I think SMTP is not reasonable.  For that matter, the sendmail
milter protocol is not at all what I would have proposed.

Vernon Schryver    vjs@rhyolite.com

More information about the DCC mailing list

Contact vjs@rhyolite.com by mail or use the form.