Current versions of this list can be found among the https://www.rhyolite.com/dcc/ web pages and their mirror at https://www.dcc-servers.net/dcc/.
personalizations?
Resource temporarily unavailable?
thread_create() failed: 11, try again? or
pthread_create(): Cannot allocate memory?
too many simultaneous mail messages?
spam traps?
See the main DCC man page as well as the DCC web page and its mirror.
Organizations that do not qualify for the free license are welcome to inquire about licenses for the commercial version by email to sales@rhyolite.com or via the form. The commercial version supports DCC Reputations.
Please note that organizations that do not qualify for the free DCC license have never been allowed to use the public DCC servers.
Please do not try to use ancient versions of DCC software dating from early 2005 and redistributed by third parties including some Linux packagers. Those versions do not detect bulk mail as well as more recent versions. Installations using those old versions also have problems using the public DCC servers that often make it necessary to add their IP addresses to the blacklist that protects the public DCC servers. Even worse, all known Linux redistributions of DCC software have been changed in ways that break things, including the /var/dcc/libexec/updatedcc shell script that could otherwise be used to fetch, configure, compile, install, and restart a current version.
When installing DCC software, please consider the installation instructions in the INSTALL.html file included with the source or in the on line source trees.
The FreeBSD packages are not too far out of date and include a working version of the /var/dcc/libexec/updatedcc shell script that fetches, configures, compiles, installs, and restarts a current version.
As far as known in 2009, all DCC RPMs offered by Linux distributors are based on DCC software from 2005 and should not be used. They cause problems for the public DCC servers.
personalizations?
personalizations and
hash busters.
Bulk messages are usually logged by DCC clients. On systems receiving a lot of mail, the mechanisms for automatically creating new log directories every minute, day, or hour can keep any single log directory from becoming too large. See the dccm and dccproc man pages.
About 1.4 GBytes/day are exchanged between each pair of DCC servers. Each server has 3 or 4 peers. The resulting database is about 3 GBytes with the default expiration parameters.. However, while dbclean is deleting old checksums, there are three copies of the database. The DCC clients and server do not need many CPU cycles, but the daily executions of dbclean on a system with a DCC server require a computer with at least 2 or 3 GBytes of RAM. In 2006, a DCC server prefers more than 4 GBytes of RAM and can use 6 GBytes. 12 to 18 GBytes of disk space are also needed.
The memory found by dccd or dbclean and how can be seen by starting them with -d in DCCD_ARGS or DBCLEAN_ARGS /var/dcc/dcc_conf file in the DCC home directory. They try to use half of physical memory if there is less than 2 GBytes, all except 512 MByte between 2 GBytes and 4 GBytes, and all but 1 GByte if there are more than 4 GBytes.
DCC servers used by clients handling 100,000 or more messages per day need to be larger. Each additional 100,000 messages/day need about 100 MBytes of disk space and system memory, given the default expiration used by dbclean.
When normally installed by the included Makefiles, DCC clients are configured to use the public DCC servers without any additional configuration except opening firewalls to port UDP 6277.
Mail systems that process more than 100,000 mail messages per day need local DCC servers connected to the global network of DCC servers. The public DCC servers include denial of service defenses which ignore requests in excess of about 240,000 per day per client.
It is wrong to resell the CPU cycles, network bandwidth, disk space, and, most important, human system administration work of the public DCC servers. Vendors of "anti-spam appliances" or similar that do not steal from the operators of the public DCC servers have always run their own DCC servers.
If the DCC sendmail interface or milter program, dccm, crashes, the default parameters in misc/dcc.m4 for the sendmail.cf Xdcc line tell sendmail to wait only about 30 seconds before giving up and delivering the mail.
The DCC client code keeps track of the speeds of the servers it knows about, and uses the fastest or closest. Every hour or so it re-resolves A records and checks the speeds of the servers it is not using. When the current server stops working or gets significantly slower, the client code switches to a better server.
In tests thesevalues for -p and -o in the /var/dcc/dcc_conf file:
DCCIFD_ENABLE=on
DCCIFD_ARGS="-p 127.0.0.1,10025,127.0.0.1/32 -o 127.0.0.1,10026 ..."
worked with these lines in /etc/postfix/master.cf
smtp inet n - n - - smtpd
-o smtpd_proxy_filter=127.0.0.1:10025
127.0.0.1:10026 inet n - n - - smtpd
-o smtpd_authorized_xforward_hosts=127.0.0.0/8
-o smtpd_client_restrictions=
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o smtpd_data_restrictions=
-o mynetworks=127.0.0.0/8
-o receive_override_options=no_unknown_recipient_checks
I think the warnings in the Postfix documentation about the costs of Before-Queue filters apply more to computationally expensive filters such as SpamAssassin than dccifd.
However, those mailing list messages talked about using dccproc before dccifd was available. Dccproc is suitable only for low mail volumes.
If the dccifd socket is present, SpamAssassin prefers to talk to dccifd instead of running dccproc.
Some commercial virus and spam filters include DCC clients that query public DCC servers or DCC servers operated by the filter vendor and that "flood" or exchange bulk mail checksums with public servers. Reputable manufacturers of such devices operate their own DCC servers connected to global network of DCC servers instead of stealing and then selling the CPU cycles, network bandwidth, disk space, and, most important, human system administration efforts of the public DCC servers.
Bharat Mediratta has developed DeepSix for people using mail user agents on UNIX boxes connected remote servers such as corporate Exchange servers. See his project on Sourceforge as well as his announcement in the DCC mailing list.
Also see the DCC installation instructions.
After firewalls, the most common cause of problems while trying to use the public DCC servers is sending too many requests. The DCC server daemon, dccd, includes defenses against denial of service or DoS attacks. Those defenses include progressively delaying responses and eventually ignoring requests. The ancient version of the DCC client software included in some Linux redistributions tries so hard to reach the fastest server that it can trigger those DoS defenses.
If the command `cdcc info` says no DCC servers are answering, you may need to adjust your firewall. Also consider the other reasons why the public DCC servers can ignore requests.
If you run a DCC server, open incoming connections to local TCP port 6277 from your flooding peers, and outgoing connections to TCP port 6277 on your flooding peers. Also open UDP port 6277 to your DCC clients (systems using dccproc, dccifd, or dccm to filter mail) and to dcc-status.rhyolite.com for the DCC server status web page.
See also the discussion of Cisco ACLs at https://www.dcc-servers.net/dcc/firewall.html.
TCP and UDP port 6276 is by the greylisting mechanism in dccifd and dccm very much like port 6277. TCP 6276 is used for flooding greylist data among your greylist database servers, if you have more than one. UDP port is used by dccm and dccifd for database operations similar to DCC operations.
To delete old log files, run the /var/dcc/libexec/cron-dccd script daily with an entry like misc/crontab in the crontab file for the user that runs dccd or dccd. The DBCLEAN_LOGDAYS parameter in the /var/dcc/libexec/dcc_conf file in the DCC home directory specifies the age of old log files.
fcntl file locking, particularly for the
DCC client map file when it is on
an NFS file system.
Another common case is using an editor such as some versions of vi that locks files on the main or a per-user /var/dcc/whiteclnt file,
thread_create() failed: 11, try again?
or pthread_create(): Cannot allocate memory?
thread_create() failed: 11, try again
or pthread_create(): Cannot allocate memory
error messages from dccm
and dccifd
is a too small limit on the maximum number of processes allowed
the UID running the dccm or dccifd process.
The "maxproc" limit seen with the `limit` or `limits` shell command
should be a dozen or so larger than the sum of
the queue sizes of dccm or dccifd (or both if both are running).
See also the common question and answer about too many simultaneous mail messages.
too many simultaneous mail messages?
Another common limit is the maximum number of file descriptors allowed by the select system call. This limit can be escaped by building the sendmail milter library to use the poll system call.
A nearby server that seems slower than a more distant server will not be chosen. The anonymous user delay set with dccd -u is intended to make a server appear slow to "freeloaders." The "RTT +/-" value that can be used with the cdcc add and cdcc load operations can be used to force DCC clients to prefer or avoid servers except when absolutely necessary.
Redundant paths among DCC servers exchanging or flooding reports of checksums would cause duplicate entries in each server's database without the mechanism that depends on every DCC server having a unique server-ID. With IDS tracing enabled, dccd complains about server-IDs that are not listed in the local /var/dcc/ids file.
The default limits can changed by adding an dccd -R argument can be added to DCCD_ARGS in the /var/dcc/dcc_conf file in the DCC home directory,
See also the dblist -C command.
The DCC source includes proof-of-concept cgi scripts for web pages supporting point-and-click white- and blacklisting. See the /var/dcc/cgi-bin/README file. There is also a demonstration of the cgi scripts.
The /var/dcc/whiteclnt client whitelist file used by dccproc, dccm, and dccifd are generally required. Client whitelists apply only to the stream of mail handled by the DCC client, while server whitelists apply to reports of mail from all DCC clients of the DCC server.
Dccproc is intended for use by individual users with programs such as procmail. Because the global whiteclnt file usually found in the DCC home directory is as likely to be used as a private file, the file name must be explicitly specified with dccproc -w whiteclnt. A perhaps inconvenient implication is programs such as SpamAssassin that switch unpredictably between dccproc and dccifd might get inconsistent results unless they invoke dccproc with the global whiteclnt file.
The global /var/dcc/whiteclnt file and the whitelists specified with dccproc -w are maintained with ordinary text editors.
Per-user whitelists in whiteclnt files specified with DCCM_USERDIRS in the /var/dcc/dcc_conf file are easily maintained with ordinary text editors by the system administrator. However, it is often better to let individual users deal with their own whitelists. The DCC source includes sample CGI scripts in the /var/dcc/cgi-bin directory to let individual end-users monitor their private logs of bulk mail and their individual whitelists. See the /var/dcc/cgi-bin/README file. There is also a demonstration of the cgi scripts.
Changes to the DCC server or dccd whitelist are not effective until after dbclean is run.
Some text editors including versions of vi lock their files. Dccm, dccproc, and dccifd are unable to read whitelist files while they are locked.
#!/bin/sh /usr/local/bin/dccproc -QCw whiteclnt \ -a 127.0.0.1 -f postmaster@example.com <<EOF Message-ID: <1234@example.com> text EOFIf the script produces something like
X-DCC--Metrics: calcite.rhyolite.com; whitelist reported: 0 checksum wlist IP: e475b896 492c60fc efecb432 6e29e3c5 ok env_From: bef98dc1 cc6ea4d7 b8daf07c a2bfbc9e Message-ID: 26573398 2ab927cd 681a89fa e502496dthen you know that SMTP client IP (mail sender) IP address 127.0.0.1 is whitelisted, but the SMTP envelope Mail_From value is not.
To use regular expressions with the DCC, consider procmail. Procmail is included with many UNIX-like systems. See also the Procmail Homepage.
DCC clients can be configured to white- or blacklist using called "substitute" headers. See dccproc -S or dccm -S.
It is also possible to use a sendmail access_db file entries to white- or blacklist based on portions of SMTP envelope and client IP addresses. For example, an access_db file line of "From:example.com OK" can be used to tell dccm to whitelist all mail from SMTP clients in the example.com domain. See the -O argument to the /var/dcc/libexec/hackmc script.
It is generally better to use white- and blacklisting lines in the global /var/dcc/whiteclnt file or a per-user whiteclnt file. See the main DCC man page
The DCC client programs solve this conflict in one of two ways. One is telling the SMTP client that the mail message has been accepted for all recipients and then discarding instead of delivering the message for mailboxes with parameters that make it spam. This solution has the disadvantage of not informing senders of the refusal to deliver the message. The other solution is to temporarily reject recipients with possibly incompatible parameters early in the SMTP transaction with the same SMTP error status number as too many recipients for a single SMTP transaction. This second solution has the advantage of ensuring that senders know when their mail is rejected but the disadvantage of sometimes requiring as many SMTP transactions as there are recipients for a mail message.
Which solution is used is determined by the forced-discard-ok and forced-discard-nok settings in the global and per-user whiteclnt files. Unless all recipients for a mail message agree on the first solution, perhaps by forced-discard-ok in the main /var/dcc/whiteclnt file, the second solution is used.
Another possible reason is that your individual legitimate mail messages have not been marked as spam because their Body or Fuz1 checksum counts are small, but that the IP address or other checksum counts are large. The IP address checksum count, for example, is the total of all reports of addressees for that checksum. That total is independent of the other checksums, and so counts all reports for all messages with that source IP address. A source of legitimate mail that has sent a message that was reported as spam by one of its recipients will often have the totals for the checksums of its IP address, From header, and other values be MANY. This is why it usually does not make sense to reject mail based on what the DCC server reports for the IP address, From header, and other values that are not unique to the message. Only the last Received header line, the Message-ID line, and body checksums can be expected to be unique and sometimes not the Message-ID and Received header lines.
many message-id <>Some Mail Transfer Agents violate section 3.6.4 of RFC 2822 and do not include Message-ID header lines in mail they send, including some combinations of qmail and "sendmail -bs" acting as the originating MTA, and qmail by itself when it is generates a non-delivery message or "bounce." Solutions to this problem include removing that line from your whitelists or adding lines specifying the From or envelope from values of senders of legitimate mail lacking Message-ID header lines.
Another common cause for this problem is implied by the fact that for an env_from whitelist entry to have any effect, dccproc must be able to find the envelope value in the message in a Return-Path header, an old UNIX-style From_ header, or an -f argument. If your mail delivery agent does not add a Return-Path header and you do not use dccproc -f, then dccproc cannot know about white or blacklist entries for envelope return addresses.
Note also that dccproc has no whitelist by default and that dccproc -w must be used.
However, if your sendmail.cf file sets the dcc_notspam macro while processing the envelope, then the message will by whitelisted. This is related to the dcc_isspam macro used by sendmail.cf modified by /var/dcc/libexec/hackmc -R to tell dccm to report blacklisted messages as spam to the DCC server.
To:domain.com DCC:OK
You can apply finer control by adding a third argument to the FEATURE(dcc) macro in your sendmail.mc file as described in misc/dcc.m4. All mail for the domain can use a single "per-user" whiteclnt file, often in the /var/dcc/userdirs/esmtp/example.com, where /var/dcc/userdirs is the default value for DCCM_USERDIRS in the DCC configuration file /var/dcc/dcc_conf. Making /var/dcc/userdirs/esmtp a symbolic link to /var/dcc/userdir/local can be useful.
To whitelist all mail sent from a domain, add a line like the following to the global /var/dcc/whiteclnt file or a per-user whiteclnt file:
OK substitute mail_host example.com
See the DCC man page.
However, in the common mode in which the DCC is used, no checksums of mail are pollution. Checksums of genuinely private mail will have target counts of 1 or a small number, and so will not be flooded by your server to other servers. Strangers will not see your private mail and so will not be able to ask any DCC server about the checksums of your private mail. On the other hand, the DCC functions best by collecting reports of the receipt of bulk mail as soon as possible. That implies that it is generally desirable to send reports of all mail to a DCC server. The DCC flooding protocol does not send checksums with counts below 10 to other servers.
spam traps?
user1 that has never been valid
and is discovered by unsolicited bulk email
advertisers by dictionary attacks or guessing.
It might instead be an address hidden in a web page
or a mailbox of an account that has been disabled for many months.
Any spam trap might receive legitimate mail. For example, a spam trap that differs from an ordinary mailbox by a single character might receive mail intended for the ordinary mailbox. It might be best for a system to reject mail sent to such a trap so that legitimate mail senders know that their messages have gone astray. A mailbox that is a long string of arbitrary letters and digits is much less likely to receive legitimate messages and so might best accept all messages without complaint.
There are several ways to connect
spam trap
mailboxes to DCC:
dccproc -R -tMANY -cCMN,MANY -o/dev/nullwill accept a message on STDIN, look for the IP address of the sender among
Received: SMTP fields,
reports the message to the DCC server as spam and the IP address as the sender,
and exit with the default value of
dccproc -x.
dccif-test -cclnt-IP-addr -oSPAM -O/dev/nullwill do much the same as the dccproc example above.
With sendmail, virtual user mapping can be used to send mail to invalid mailboxes to a single mailbox whose corresponding DCC per-user whiteclnt file contains an option spam-trap-accept or option spam-trap-reject line.
A DCC server in a network of many servers should have at least three flooding peers to ensure that the failure of a single server or network link cannot partition the network. Limiting the number the number of peers of any server to four or perhaps a few more ensures that no single server is critical to the network. To minimize the distances in the network, four peers per server seem necessary.
An organization with more than one server can be viewed as a single server by other organizations, with its servers flooding each other and external peers spread among its servers. This protects the network should the organization suffer large scale problems while protecting the organization from single points of failure.
The cdcc flood list operation displays the current flooding peers of a DCC server. Counts of checksum reports sent and received to and from a single peer can be displayed with cdcc "flood stats ID"
The positions in the local database of outgoing streams of checksums are displayed by the start of dblist -Hv.
This document describes DCC version RHYOLITE_VERSION.