DCC - Postfix MTA mail hub implementation question

Vernon Schryver vjs@calcite.rhyolite.com
Fri Aug 23 19:00:02 UTC 2002

> From: Matt Armstrong <matt@lickey.com>

> >>For postfix the dominant cost is not fork/exec but management of the queue files
> >>
> >Depending on the operating system and file system, fork/exec can
> >cost more than creating a file or two.
> >
> I think when you factor in the fact that the queue files are synced 
> (fsync()) to disk the disk I/O overshadows fork/exec on anything but the 
> most lame OS.

What's that about a fact that queue files are fsync()'ed?  It sounds
reasonable and prudent, but I wouldn't go so far as to call it a fact
about all MTAs.  If you're saying that postfix does it, and therefore
postfix is limited by disc I/O, then I can't argue.

> ...
> I have a hard time imagining a server not limited by either network 
> bandwidth or disk bandwidth. Everything else they do is orders of 
> magnitude faster, including the odd fork/exec.

Consider the computers that have files systems that can read or write
many GBytes/second of UNIX files and that have OC-768 network links,
and then add the CPU costs of TLS, SMIME, PGP, or other encryption
and virus or other content filtering....well, OC-768 might be a tad
rare just yet, but the rest of my point stands.
It may be that postfix is disk bound on all existing installations,
but the world contains more than postfix or PCs.

> ...
> >Oh, you mean literally use SMTP?  Well, I'm not enthused about the
> >notion of writing an MTA.  Not only do I wonder if would save many
> >CPU, disk, or other resources, the effort seems redundant.  One could
> >use sendmail+dccm as the "DCC daemon."
> >
> The content filter daemon is not an MTA. It doesn't have to interface 
> with the disk at all if keeping the whole message in RAM is acceptable. 

I don't think dealing with disk files is the crux of even a skeleton
of an MTA.

> And it need only implement a subset of SMTP. I think it would be 
> surprisingly easy to do (yes, passing the IP addr is a problem).

You would need to implement that subset of SMTP that Postfix uses when
it pass mail to a filter.  If that subset is small now, how do you
ensure that it stays small?  What keeps the Postfix maintainers
from using some SMTP or even ESMTP bell or whistle next week?
Such problems are solvable, if only by implementing enough of SMTP,
but to what profit?  If Postfix installations can afford to run
SMTP twice for every message, then as you say, they can probably
can afford to use fork/exec instead of SMTP to a filter.

Besides, I'm not considering converting the DCC to a Postfix adjunct.  
What other MTAs would want to use a filter whose interfaces is SMTP
in and out?

> ...
> I think there is a spectrum of "customers" that would be completely 
> satisfied with an SMTP based solution (with Received: header parsing for 
> source IP address and all). This might even be the majoraty of SMTP 
> servers on the net -- those with relatively low volume; those with 
> enough resources to double their server capacity; those whose network 
> bandwith will be maxed out before their SMTP server is.

I don't understand that, unless you mean the "majority of SMTP
servers using postfix" instad of the "majority of SMTP servers."
I doubt the majority of SMTP servers (as measured by traffic) on the
net that can use the DCC without significant changes might usefully
double themselves to use the DCC.

An MTA that deals with filtering by "punting" everything to a second
MTA that filters and then forwards back to the first is not really
interested in filtering.  That may be an execellent design trade-off,
but let's call it what it is.

> But you're right, the top end probably requires a more efficient solution.

What is "top end" in this context?  "High volume SMTP servers,"
"upper level processing" in some sense such as rejecting before saying
"2xx" to the DATA command, convincing the Postfix maintainers to accept
patches to Postfix, or something else?

Vernon Schryver    vjs@rhyolite.com

More information about the DCC mailing list

Contact vjs@rhyolite.com by mail or use the form.