Greylisting with multiple MX servers

Michael Mansour mic@npgx.com.au
Sat Apr 7 21:41:44 UTC 2007


Hi Vernon,

> > From: "Michael Mansour"
> 
> > Because it's only spam checking it's basically just upping the spam score a
> > bit.
> 
> The correct, although less popular way to use the DCC is with local
> thresholds defining "bulk."  If a mail message must be checked more than
> once for DCC target counts, the second and subsequent checks should
> be done with -Q so that a message that is not bulk does not become bulk
> and so get branded as unsolicited bulk email or spam.
> 
> Instead of running with -Q, you could whitelist your exterior MX servers
> by with mxdcc lines in whiteclnt files, provided you are using the

Yes, I've added mxdcc lines in whiteclnt on main1.

> current version of the DCC source instead somethink like the more 
> than 2 year old version that Debian is still pushing.

I use the DCC RPM from ATrpms:

DCC-1.3.50-16.el4.at.x86_64

on "main1" (64bit) and:

DCC-1.3.50-16.el4.at

on the MX servers (32bit).

The 64bit version currently has a bug in the package (no dccm compiled in)
which I've reported to Axel:

http://bugzilla.atrpms.net/show_bug.cgi?id=1169

he recommends for now to use the 32bit version on the 64bit platform,
something I'll do soon.

So it's my understanding that as long as I'm using the mxdcc in whiteclnt on
main1, then I won't need -Q.

> Local systems running browsers as MUAs that cannot be trusted to not
> become infected with a trojan proxy and that cannot tolerate 4yz
> rejections of greylisting should be whitelisted with "submit" lines,
> but that's an unrelated story.

I'm wondering what "Local systems" means in the context of being a provider. I
take inbound mail, deliver it to main1 and clients pickup via pop, imap, etc.
In some cases the MX servers deliver to smtp servers for client sites, while
all I do is scan messages for them.

> > I've now disabled dccifd and disabled the plugin in SA, added an SA rule to
> > see the header and score it accordingly.
> 
> Again, I thought the SpamAssassin plug-in already includes a regular
> expression that detects X-DCC headers.

Yeah it may, but just thought it is better to be safe than sorry :)

> > Test emails from hotmail seem to fail quickly (hotmail doesn't seem to wait
> > the default 270s, multiple test emails sent from there fail regularly), while
> > yahoo goes through most of the time. gmail takes a while and seems to not come
> > through and not produce an error.
> >
> > Yahoo seems to produce the best error reports out of the three.
> 
> I hope you mean "least bad error reports," because none of the three
> you report are useful.  You need an honest copy of the SMTP response
> from your SMTP servers, not a supposed distillation.

Exactly what I mean :)

> > During my tests today, I'd send an email from all three to:
> >
> > testuser@example.com
> >
> > and hotmail pops back with "undeliverable", yahoo with "User unknown" and
> > gmail with "Recipient address rejected: User unknown in recipient maps table
> > (in reply to RCPT TO command)"
> 
> None of those messages are really useful.  You need a copy of SMTP response
> string that your sendmail processes are sending.
> 
> Many years ago Hotmail refused to pass the SMTP responses to users, but
> then they mostly fixed that.

Ok, I dug a bit and found that they have "options" which can enable full
headers, which I did. I then viewed the "email message source" and found:

Final-Recipient: rfc822;testuser@example.com
Action: failed
Status: 5.7.1
Diagnostic-Code: smtp;554 5.7.1 <testuser@example.com>: Recipient address
rejected: User unknown in recipient maps table (in reply to RCPT TO command)

basically the same error that google and yahoo gave.

> > Looking at the yahoo error more closely, the test message I sent to:
> > To: testuser@example.com
> > was changed to:
> > To: testuser@realmail3servername.realdomainname.com
> > which is why it bounced back with a "User unknown" and why gmail complained
> > about the same thing.
> >
> > This is extremely strange. With dcc greylisting off, the MX servers work as
> > expected by accepting the mail, doing the recipient checking with main1 via
> > another sendmail milter, relaying the mail to main1 via mailertable entries.
> 
> Are you using greylisting on your interior SMTP server?  Have you checked

Hmm.. I did have greylisting enabled in dcc_conf on the interior server but it
wasn't working because the dccm wasn't available in the 64bit RPM package from
ATrpms (although dccifd was working and SA was using the plugin).

I've reconfigured dcc_conf to disable any greylisting and dccifd and the SA
plugin, I've also removed the 64bit DCC package and installed the 32bit DCC
package so dccm is present now.

Either way, I really should have checked this more thoroughly before getting
those MX servers on-line.

> the Received headers to ensure that what you are seeing is not a bounce
> from your mail2 or mail3 reporting that your mail1 did not like the address?

 From the above bounce, it seems that the problem is recipient checking on the
local MX server.

> I think you said you are using dccifd on your interior mail1.
> Have you whitelisted your mail2 and mail3 systems for mail1 with 
> lines like these in whiteclnt on mail1?  mxdcc  ip mail2 mxdcc  ip mail3

Yes, that was done before.

> As the `man dcc` man page tries to say, "mxdcc" whitelisting should
> be used to tell your interior SMTP servers about mail that has 
> already been DCC checked.
> 
> > > It is nice to have 2 greylist servers for larger mail systems, so 
> > > that one is available when the other is down for maintenance or running
> > > dbclean.  The best way to do that is to 
> > >   - define a new DNS name such as dccgrey.example.com with A RRs
> > >     for the IP addresses of the servers.
> >
> > Hmm... but wouldn't an RR request fail if the greylist server was down?
> 
> Today, it is generally a bad idea to have SMTP MX secondaries as
> backups.  Multiple SMTP servers are necessary if one can't handle all
> of the mail you receive, but old fashioned MX secondaries cause a lot
> more trouble today than they prevent.  15 or 20 years ago you needed
> MX secondaries to receive and queue mail when your primary was down,
> because you could not rely on distant SMTP clients to do their own
> queuing and retrying.  Today all mail sendesr except many spammers do
> good jobs of retrying--sometimes too good as with common configurations
> of sendmail that retry once every 1 or 2 seconds.  Even some spammers
> retry reliably.
> 
> DNS differs from SMTP.  You must have DNS secondaries and they must
> be dispersed both geographically and topologically.  If not, you will
> lose mail as well as HTTP hits and other things that often to translate
> to money.

Yes, I have both MX and DNS secondaries both locally in my country (Australia)
and geographically internationally (two distant locations in the USA). I've
had this setup for some years now and when things have gone haywire locally,
the USA servers has at those times, come to the rescue.

> So to answer the question, a DNS RR request must never fail and not
> merely for the DCC clients.  (Never mind that the DCC clients only
> re-check the DNS resolutions of DCC servers every couple of hours and
> I think they do the right thing if DNS does fail.)

Ok.

>  ...........
> 
> ] From: "Michael Mansour"
> 
> ] I tell the MX servers, via the sendmail relay-domains and the mailertable
> ] file, to use the main1 smtp server to relay any mail they receive. 
> I can't see ] any other reason why hotmail, yahoo, gmail would all 
> have a bounce back of ] "recipient address rejected: user unknown in 
> recipient map table" if the ] sendmail delivery rules were being 
> followed. ] ] It seems what is happening is that the message is 
> embargoed, then let's say it ] passes, the greylist doesn't attempt 
> to follow the sendmail rules at ] delivering the message to "main1" 
> but instead tries to find the recipient on ] the MX server it's run 
> on. This is why the message gets bounced with a ] recipient map problem.
> 
> I cannot see how turning on DCC greylisting could cause that.
> 
> Please try using "mxdcc" lines in your whiteclnt files.

Yep, that was done before. I'll try and do some more tests today with sendmail
milter logging at a high value to see if I can see anything that would cause
this problem in the maillogs.

> ] When using multiple MX, the sequence of delivery attempts can 
> become ] complicated, and if the sending MTA does not try all the MX 
> servers, the ] delivery can be much longer. milter-greylist has a 
> multi-MX sync feature, that ] enable the database to be shared by 
> all the MX servers. ] *** ] where the developer seems to have dealt 
> with this problem effectively.
> 
> I do not see how that is relevant.  It seems to me to be saying what
> I said before.  If you do not synchronize the greylisting databases
> of your MX servers (as most greylisting implementations do not), then
> greylisting embargos can be inflated N times, where N is the total
> number of MX servers.

In reading the http://hcpnet.free.fr/milter-greylist/README, he explains a
little more:

***
8 Working with multiple MXs
 ===========================

When running several MXs, the client should try each server after
its message gets refused, thus causing greylist entries creation 
on each MX. Things should work, but with two minor problems:

* Some stupid clients don't try all the available MXs. In that 
  situation, it could take some time before the message gets in,
  as the client might try a different MX each time and wait for 
  several hours between the retries.

* After a messages is accepted, its entry is removed for one MX, 
  but not the others. Stale entries remain until being flushed
  because of a timeout. If a message with the same {IP, from, rcpt}
  gets in on an MX with a stale entry, it will be accepted 
  immediately, and the X-Greylist header will report it had been
  delayed for some time.

In order to address these issues, milter-greylist is now able to
sync the greylist among different MXs. This can be configured in
the greylist.conf file, by adding one line per peer MX,  
like this:
peer 192.0.2.17
peer 192.0.2.18

If you have firewalls between your MXs, you should enable TCP 
connections in both directions between random unprivileged 
source ports and destination port 5252.
***

Is this the same as flooding?

I originally understood it as meaning if an smtp client connection came in on
one MX, then that connection would be sync'ed with the other MX so if the smtp
client then tried the next MX, the MX would pass the message.

But re-reading it, that doesn't seem to be the case. It seems to just be
saying that if a triplet is in the greylist database, then that is synced with
other MX'es, and if removed from the database, then the removal is also
synced. That may be exactly what flooding does?

Thanks.

Michael.

> Vernon Schryver    vjs@rhyolite.com
> _______________________________________________
> DCC mailing list      DCC@rhyolite.com
> http://www.rhyolite.com/mailman/listinfo/dcc
------- End of Original Message -------




More information about the DCC mailing list

Contact vjs@rhyolite.com by mail or use the form.