Distributed Checksum Clearinghouse (DCC) Installation

  1. Fetch the Source and Read the License

    The DCC source is available at dcc-servers.net and Rhyolite Software.

    Please do not try to use the ancient, modified versions of DCC software distributed by some Linux packagers. Those versions do not detect bulk mail as well as more recent versions. Installations using those old versions also have problems using the public DCC servers that often make it necessary to add their IP addresses to the blacklist that protects the public DCC servers. Even worse, all known Linux redistributions of DCC software have been changed in ways that break things, including the /var/dcc/libexec/updatedcc shell script that could otherwise be used to fetch, configure, compile, install, and restart a current version.

    The license on the free source is in the source as well as dcc-servers.net. The free license is intended to cover individuals and organizations including Internet service providers using DCC to filter their own mail. Organizations selling anti-spam appliances or managed mail services are not eligible for the free license.

  2. Read the Documentation

    The DCC and other man pages describe the features, operating modes, required data files, and other characteristics of the DCC. Also see the DCC FAQ or list of frequently answered questions.

  3. Build Sendmail

    If the DCC-sendmail interface, dccm, is not used, then skip to the next step.

    Sendmail must have the Mail Filter API or Milter enabled. Some systems such a FreeBSD 4.6 and newer are shipped with Milter enabled and the library installed by default. If your system comes with the Milter interface turn on, then skip to the next step. Otherwise, the Milter interface must be explicitly enabled by adding lines like those in misc/site.config.m4 to your sendmail/devtools/Site/site.config.m4 file or equivalent. Then build sendmail as described in the INSTALL file distributed with sendmail. You must build libmilter separately by something like

    	    cd libmilter
    	    sh ./Build
    

    After sendmail has been rebuilt if necessary it will need to be restarted. That should be done after the next step after misc/dcc.m4 has been created by the ./configure script.

  4. Configure, Build, and Install the DCC Programs

    See the installation considerations in the DCC man page.

    Most DCC files are in a "home directory" such as /var/dcc. DCC programs such as cdcc and dccproc are run by end users and should be installed in a directory such as /usr/local/bin. They must also be set-UID to the UID that can change the DCC data files. DCC programs that do not need to be run by end users are installed by default in the libexec subdirectory of the DCC home directory. See the table of ./configure script and makefile parameters. If necessary, set CFLAGS, LDFLAGS, LIBS or other environment variables listed in the table. Omit any parameters you don't really need to change.

    End users installing only dccproc can install it in their private ~/bin directories and use private directories for their DCC home directories. In this case, the DCC programs that would otherwise need to be set-UID need not be.

    To build dccproc for an individual user, use something like

    	./configure --disable-sys-inst --disable-dccm --homedir=$HOME/dccdir  --bindir=$HOME/bin
    	make install
    

    If you do not intend to use the sendmail interface, dccm, use something like

    	./configure --disable-dccm
    	make install
    
    If you will use dccm, it must be built with the sendmail source and object tree. By default, the makefiles look for native sendmail libraries (e.g. on FreeBSD), an installed "package" (e.g. on FreeBSD), or a directory named sendmail parallel to the DCC source and object tree. Those who regularly build new versions of sendmail may find it convenient to make a symbolic link there to their current sendmail. Otherwise configure the dccm makefile with
    	./configure --with-sendmail=/some/where/sendmail
    	make install
    
    If dccm does not build because it cannot find libmilter, check that libmilter was compiled with sendmail in the previous step.

    To connect the sendmail Milter interface to dccm, copy or symbolically link misc/dcc.m4 to your sendmail/cf/feature directory and add FEATURE(dcc) lines to your sendmail.mc configuration file. It can be useful to modify sendmail.cf by using misc/hackmc. Then rebuild and reinstall your sendmail.cf file, and restart sendmail.

  5. Create Client Configuration Files

    All DCC configuration files are in the DCC home directory, usually /var/dcc. See the dcc, dccm, dccifd, and dccproc man pages for the files each needs. Example files are in the homedir directory in the source.
  6. Create Server Files and Start the Server

    Skip this and the next step if only remote DCC servers will be used. You should use your own, local DCC servers only if your mail system handles more than 100,000 mail messages per day. If your mail system handles fewer than 100,000 mail messages per day, you should use the public DCC servers. The DCC client programs are configuerd to use the public DCC servers by default.

    It is best to use remote servers until the DCC client, dccm, dccifd, or dccproc, is stable. Then

  7. Configure Flooding

    Skip to the next step if only remote DCC servers will be used.

    Flooding requires that every server participating in a network of DCC servers have a unique server-ID. Server-IDs can be obtained by contacting Vernon Schryver at vjs@rhyolite.com by email or via a web form.

    After you have an official server-ID,

    Flooded reports of bulk email contain timestamps that are used for several things including expiring old reports. To accurately detect stale incoming reports, a DCC server needs a clock that is not too inaccurate. For that reason it is good to run an NTP daemon on systems running DCC servers.

  8. Configure Greylisting

    Skip to the next step if greylisting will not be used. Greylist is very effective. See this description.

    Larger sites can use more than one greylist server, with the greylist servers flooding data just like DCC servers.

    To configure greylisting:

    1. Assign greylist client-IDs and server-IDs

      Client-IDs and matching passwords must be used by clients of greylist servers such as dccm and dccifd. The client-IDs must be in the /var/dcc/map file on the client system. Greylist client-IDs and server-IDs must be in the /var/dcc/ids file on the greylist server. When a system hosts both DCC and greylist servers, it is convenient for clients to use the same client-ID and password for both. It is also convenient for a greylist server and a DCC server on a system to share a common server-ID and password.

      The vast majority of installations do not have local DCC servers and can use the greylist server-ID generated automatically in the /var/dcc/ids file.

    2. Add the greylist server to /var/dcc/map

      If the cdcc "info" command does not show the correct greylist server, add it with something like

      	cdcc "add localhost greylist 32768 secret"
      
      The DCC makefile files add a greylist server at localhost or 127.0.0.1 to /var/dcc/map file created for a new DCC installation.

    3. Set /var/dcc/dcc_conf

      In most installations, enable a local greylist server by installing the script /var/dcc/libexec/rcDCC with a symbolic link, setting GREY_ENABLE=on in /var/dcc/dcc_conf and then running
      	/var/dcc/libexec/rcDCC start
      

      If absolutely necessary, override the greylist embargo, wait, and white values in GREY_DCCD_ARGS in /var/dcc/dcc_conf. Usually simply set GREY_CLIENT_ARGS=on

    4. Set /var/dcc/grey_flod

      Sites with more than one greylist server should arrange to flood data among them by adding lines to /var/dcc/grey_flod files in the same format as /var/dcc/flod files. Flooding among greylist servers uses port 6276 by default, and so that port may need to be opened in firewalls.

    5. Set cron job

      Install a daily cron job like misc/crontab and /var/dcc/libexec/cron-dccd to clean the database.

    6. Whitelist Mail Submission Clients

      Greylisting of local mail systems must be turned off because common mail user agents (MUAs) cannot handle temporary rejections. One way to turn off greylisting of local client is with submit lines in the main whiteclnt file as described above.

      An alternative to whitelisting mail submission clients is available with dccm and sendmail by using the misc/hackmc -T script to modify sendmail.cf to trust SMTP clients authenticated with SMTP-TLS or SMTP-AUTH.

  9. Start dccm

    If the DCC-sendmail interface, dccm, is not used, skip to the next step.

    The DCC sendmail milter interface dccm should be started before sendmail. That often requires changing an /etc/rc script or configuration file. The script /var/dcc/libexec/rcDCC should be installed, best with a symbolic link. The milter daemon can be started manually with

    	rcDCC start
    

  10. Start dccifd

    If the general MTA interface, dccifd, is not used, skip to the next step. If you are using SpamAssassin, then you almost certainly should be using dccifd.

    The general MTA interface dccifd should usually be started before the mail transfer agent or MTA. It should be enabled by setting DCCIFD_ENABLE=on in /var/dcc/dcc_conf. It is also usually necessary to change an /etc/rc script or configuration file to start and stop the daemon with the system. The script /var/dcc/libexec/rcDCC should be installed, best with a symbolic link. The daemon can be started manually with

    	rcDCC start
    

    Dccifd can be used as a Postfix Before-Queue Content filter as described the dccifd man page.

  11. Configure Uses of dccproc

    If dccproc is used with procmail, add rules to procmailrc files as described in the dccproc man page.

  12. Adjust Rejection Thresholds

    It is best to only mark mail with X-DCC SMTP headers before changing procmail or dccm to reject mail. Configure dccm with DCCM_LOG_AT in /var/dcc/dcc_conf to log bulk mail with somewhat lower counts.

  13. Additional Considerations

    Some additional mechanisms are available in the DCC client programs. They are often unnecessary when greylisting is used.

    When possible, it is almost always better to use dccifd than dccproc. This is certainly true with SpamAssassin. When using SpamAssassin, ensure that the SpamAssassin plugin DCC.pm is up to date. The DCC source includes a copy in the misc directory. Please consider setting dcc_learn_score to report spam to other SpamAssassin with DCC users.

  14. Update As Needed

    New versions released at the usual place can be installed by running the /var/dcc/libexec/updatedcc script. That script is (re)built by the ./configure script and runs ./configure with parameters and environment variables from the previous installation.

  15. Remove or Uninstall

    Most of the DCC can be removed by running /var/dcc/libexec/uninstalldcc script. Some logs and configuration files with locally chosen parameters in the home directory are not deleted. Manual changes such as links to /var/dcc/libexec/rcDCC or the installation of the cron job, /var/dcc/libexec/cron-dccd, are not reversed.

Installation Parameters

There are several installation configuration parameters that can set to suit individual preferences and systems.

Makefile and ./configure Script Controls
Do NOT set these parameters unless absolutely necessary.
./configure option env name or
make variable
used by default value use
--homedir=HOMEDIR   ./configure /var/dcc/ DCC home directory with most DCC files
--bindir=DIR   ./configure /usr/local/bin directory for DCC user commands including cdcc and dccproc3
--libexecdir=DIR   ./configure --homedir/libexec directory containing most DCC programs
--mandir=DIR   ./configure /usr/local/man directory for man pages3
  NOMAN1 make unset do not install man pages when set3
--with-installroot=DIR   ./configure unset prefix all directory paths to build a binary tarball
--with-configsuffix=str   ./configure unset append str to generated configuration file names
--with-uid=UID   ./configure root user name and set-UID for DCC programs and data
  DCC_OWN1 make bin, daemon on OS X, or current owner or UID of most installed files3
  DCC_GRP1 make bin, daemon on OS X, or current group of most installed files3
  DCC_MODE1 make 555 mode of most installed programs
  MANOWN1 make DCC_OWN or current owner or UID of installed man pages3
  MANGRP1 make DCC_GRP or current group of installed man pages3
--disable-sys-inst   ./configure enabled disable system installation or chmod, chgrp, and set-UID3
--disable-server   ./configure build but do not start do not build server including dbclean and dccd
--disable-dccifd   ./configure build but do not start do not build program interface
--disable-dccm   ./configure build but do not start do not build sendmail interface
--with-sendmail=DIR   ./configure ../sendmail or /usr/ports/mail/... directory containing sendmail milter header files
--with-cgi-bin=DIR   ./configure --homedir/cgi-bin directory for DCC whitelist CGI scripts
--with-rundir=DIR   ./configure /var/run/dcc "run" directory for PIDs and sockets
  CFLAGS1 make & ./configure   compiler options such as -g or -O2
  PTHREAD_CFLAGS2 ./configure depends on target compiler options for compiling dccm and dccifd with pthreads
  LDFLAGS1 make & ./configure   global linker options
  PTHREAD_LDFLAGS2 ./configure depends on target linker options for dccm and dccifd
  LIBS2 ./configure   additional libraries linked with all programs
  PTHREAD_LIBS2 ./configure depends on target libraries for dccm and dccifd
  CC make & ./configure cc C compiler such as "gcc" or "/opt/SUNWspro/SC6.1/bin/cc"
  INSTALL1 make ./autoconf/install-sh installation script
  DCCD_MAX_FLOODS1 make 32 maximum DCC server flooding peers
--with-db-memory=MB   ./configure 64 minimum server database buffer size between 32 MBytes and 49152 MBytes
--with-max-db-mem=MB   ./configure 1920 on 32-bit systems
49152 on 64-bit systems
maximum server database buffer size
--with-max-log-size=KB   ./configure 32 maximum dccproc, dccifd, and dccm log file size in KBytes; 0=no limit
--disable-IPv6   ./configure enabled; use IPV6 if available turn off IPv6 support even if available
--with-socks[=lib]   ./configure none location of SOCKS client library
--enable-64-bits   ./configure depends on operating system and hardware enable 64-bits on Solaris and Linux PowerPC
--with-make-cmd=pgm   ./configure make or gmake path to make command
--with-DCC-MD5   ./configure local library if available use MD5 code in DCC source instead of any local library
--with-kludge=FILE   ./configure none include header FILE, best with an absolute path
--with-fetch-cmd=pgm   ./configure wget, fetch, curl, or ftp program used by /var/dcc/libexec/updatedcc, and other utilities to fetch files
--with-fetch-cmd-addr=ip   ./configure none local IP address used by wget, fetch, or curl while fetching files
--enable-lang-Dutch   ./configure disabled enable Dutch dictionary in checksums
 
Note1
These values are not built into the Makefiles by the ./configure script but their current values in the environment are used by the script and the Makefiles.
Note2
These values are copied by the ./configure script from the environment into the generated Makefiles.
Note3
When --disable-sys-inst is specified, the current UID and GID become the defaults, and the man pages are not installed. If the ./configure script is not run as root, dccproc, cdcc, and dccsight are not installed set-UID. It is usually also necessary to set --bindir to a private directory such as $HOME/bin.

Compatibility

DCC is thought to work on several systems including:

BSDI BSD/OS
DCC works starting with version 3.0 of BSD/OS.
FreeBSD
The works starting with at least version 4.0 of FreeBSD.
NetBSD
DCC works without change starting with at least NetBSD 4.0.
OpenBSD
DCC works starting with at least 2.9 despite the lame mmap() implementation.
Linux
Avoid the several years old, modified versions shipped by some Linux packagers.

On 64-bit PowerPC systems with more than 4 GBytes, use ./configure --with-64-bits to build a DCC server that can benefit from a full sized database. A 64-bit sendmail milter library will be needed if dccm is used

AIX
DCC on 4.1.PPC has been tried but not well tested. Rumor has it that the 4.1.PPC pthreads code does not work with the sendmail milter library and dccm, but the rest of DCC does work.
Solaris
DCC compiles on several versions of Solaris with gcc or native C compiler by setting the environment variable CC appropriately. You must install gmake. Do not use "CFLAGS=-fast" with the native compiler.

While building the sendmail milter library, consider using _FFR_USE_POLL to avoid problems with large file descriptors and select().

On 64-bit systems with more than 4 GBytes, use ./configure --with-64-bits to build a DCC server that can benefit from a full sized database. A 64-bit sendmail milter library will be needed if dccm is used

HP-UX
DCC compiles on versions of HP-UX starting with 11.00. It requires gmake. Dccproc and dccm work. Dccifd does not work with UNIX domain sockets because select() and poll() do not notice the results of shutdown(). Dccifd does work with TCP/IP connections to MTAs or spam filters.
Dccproc should work on version 10.20, since it does not use pthreads.
IRIX
DCC compiles on IRIX 6.5. It requires gmake.
OSF1
DCC compiles on OSF1 V5.0 with gmake.
OpenUNIX
DCC compiles on OpenUNIX 8.0.1.
Mac OS/X
DCC compiles on at least some versions of Apple's OS/X.
Windows
The DCC client dccproc compiles and works on at least some versions of Microsoft Windows 98, XP, and Vista with Borland's free SDK and with Microsoft's SDK. See the main Makefile for Windows.

Those system names include trademarks. Please don't abuse them.

Troubleshooting

Much of the DCC list of frequently asked questions concerns troubleshooting DCC installations. Many of the messages in the archive of the DCC mailing list are also troubleshooting questions and answers.

Spam Traps

Dccm and sendmail can be configured to report the checksums of unsolicited bulk mail so that other DCC clients can reject later copies of the same unsolicited bulk mail sent from other sources. Such mechanisms are commonly called spam traps.

Entries in a sendmail access_db can also be rejected or discarded while they are reported to the DCC server by dccm. The script misc/hackmc modifies the output of sendmail .mc files to tell dccm about some undesirable mail. The script accepts one or more .mc files and generates the corresponding slightly modified .cf files. If the access_db entry starts with the string "DCC:", the message is reported by dccm to the DCC server as extremely bulky. Otherwise the message is rejected as usual. The remainder of the the access_db entry after "DCC:" consists of the optional string "DISCARD" followed by an optional SMTP status message. If the string "DISCARD" is present, the message is discarded instead of rejected. This is important to keep senders of unsolicited bulk mail from discovering and removing "spam trap" addresses from their target lists.

For example, a line like the following in an access_db can discard all mail from example.com while reporting it to the DCC server as extremely bulky. Note the quotes (").

    example.com     DCC: "DISCARD spam"

Spam traps can be configured with option spam-trap-discard and option spam-trap-accept lines in whiteclnt files.

Mail from a spam trap address can be sent to dccproc as described in the dccproc man page

SOCKS

The DCC client and server programs can be built to use the SOCKS protocol. The --with-socks ./configure parameter configures the DCC client library and the DCC server to use common SOCKS network library functions. If the SOCKS library is in a standard place, something like --with-socks=socks should be sufficient. Setting the environment variable DCC_LDFLAGS to something like -L/usr/local/lib is sometimes helpful. Otherwise, using --with-socks without specifying the library name and setting LIBS to the full pathname of the library might work.

DCC client programs including dccproc and dccm that use the DCC client library must be told to use the SOCKS5 protocol with the SOCKS on operation of cdcc. SOCKS5 is required instead of SOCKS4 because DCC clients communicate with DCC servers using UDP.

DCC servers can use SOCKS4 or SOCKS5 when exchanging floods of reports of checksums. Links between individual pairs of peers are configured with the passive and SOCKS flags in the flod file described in the dccd man page. In both cases, the SOCKS library code must be configured, often in the files /etc/socks.conf and /etc/socksd.conf.

When the DCC software is built with SOCKS, IPv6 name resolution is turned off.

The DCC server and client programs have been tested with the DANTE library and server. The DANTE SOCKS implementation is also one of the FreeBSD "ports" or packages.

Note that if a connection fails repeatedly, Dante will disable the rule that failed and will eventually try the underlying connect() call. This fails in almost every SOCKS environment because there is no available route for an ordinary connect(). Dante by default won't re-enable the failing rule. To fix this, change BADROUTE_EXPIRE from the default of 0*60 to 5 in include/config.h in the Dante source and recompile.

This document describes DCC version RHYOLITE_VERSION.