DCC - Postfix MTA mail hub implementation question

Matt Armstrong matt@lickey.com
Fri Aug 23 18:27:39 UTC 2002


Vernon Schryver wrote:

>>For postfix the dominant cost is not fork/exec but management of the queue files
>>
>Depending on the operating system and file system, fork/exec can
>cost more than creating a file or two.
>
I think when you factor in the fact that the queue files are synced 
(fsync()) to disk the disk I/O overshadows fork/exec on anything but the 
most lame OS.

>>The postfix content filtering scheme involves the mail 
>>totally leaving the postfix universe (it is effectively "delivered" to 
>>the content filter), so the queue file is deleted. The content filter 
>>reinjects the message, so a new queue file is created and synced to 
>>disk. So the performance will be roughly cut in half given that disk I/O 
>>is the bottleneck for SMTP server.
>>
>Not all SMTP servers are limited by disk I/O.  However, that filtering
>tactic does sound like a good way to double whatever costs are the limit.
>
I have a hard time imagining a server not limited by either network 
bandwidth or disk bandwidth. Everything else they do is orders of 
magnitude faster, including the odd fork/exec.

>>...
>>    
>>
>>>      daemon and the result on output, wand the bodies delimited
>>>      probably by a leading byte count.  (RFC 821 escaping sounds like
>>>      more trouble for all concerned.)
>>>      
>>>
>>I think the SMTP protocol itself is sufficiently powerful. 
>>    
>>
>
>SMTP "transparency" (see section 4.5.2 of RFC 821) is certainly
>sufficiently powerful to handle anything that arrives via SMTP,
>particularly when you "just send 8 bits".  The trouble is that
>scanning for "." is expensive for both sender and receiver.
>  
>
>>A permanent 
>>DCC daemon would run as a postfix content filter. Postfix would 
>>"deliver" the message via SMTP to the DCC daemon. The daemon would do 
>>its thing, possibly rejecting the message with a 5XX SMTP error or 
>>possibly a 4XX SMTP error (if, say, the DCC server isn't answering). If 
>>the message pases the DCC thresholds, the daemon would reinject the 
>>modified message back into postfix via SMTP on a different port.
>>    
>>
>
>Oh, you mean literally use SMTP?  Well, I'm not enthused about the
>notion of writing an MTA.  Not only do I wonder if would save many
>CPU, disk, or other resources, the effort seems redundant.  One could
>use sendmail+dccm as the "DCC daemon."
>
The content filter daemon is not an MTA. It doesn't have to interface 
with the disk at all if keeping the whole message in RAM is acceptable. 
And it need only implement a subset of SMTP. I think it would be 
surprisingly easy to do (yes, passing the IP addr is a problem).

>It also has little chance of getting accepted "downstream," if I
>understand what is meant in this context by "upstream."  The things
>that might almost be considered sendmail patches in the public DCC
>source (the FEATURE() files, the hackmc script, and the site configuration
>file)) are at or beyond what I consider reasonable.
>
>Would any of the other cases that use dccproc be able to use a reasonable
>protocol, and if so, what sort?  
>Again, I think SMTP is not reasonable.  For that matter, the sendmail
>milter protocol is not at all what I would have proposed.
>
I think there is a spectrum of "customers" that would be completely 
satisfied with an SMTP based solution (with Received: header parsing for 
source IP address and all). This might even be the majoraty of SMTP 
servers on the net -- those with relatively low volume; those with 
enough resources to double their server capacity; those whose network 
bandwith will be maxed out before their SMTP server is.

But you're right, the top end probably requires a more efficient solution.






More information about the DCC mailing list

Contact vjs@rhyolite.com by mail or use the form.