DCC version 1.3.54/2.3.54 released

Vernon Schryver vjs@calcite.rhyolite.com
Thu Mar 15 18:06:57 UTC 2007

> From: Sven Willenberger 

> >    Make `dccd -F` the default on Solaris to speed up the DCC server there.
> Just to verify, the desired behavior on solaris should now be read() and
> write() rather than mmap() ?

-F uses mmap(MAP_PRIVATE) to read the data from the disk into RAM,
but {memcpy(); write();} to send the data from RAM back to the disk.

The issue is that the default Solaris flushing timeout is far too short.
`dbclean -F` has long been the default because otherwise Solaris eats
up so much disk bandwidth flushing the mapped pages that dbclean is
working on that dbclean takes half of forever.   The original -F
code assumed all buffers were always dirty and always sent all of them
to the disk with write();  That's fine for dbclean, because dbclean
does touch everything.  Dccd does not change most of the database,
so the original `dccd -F` cost more than it saved.

Other UNIX-like systems such as FreeBSD also flush far too quickly, but
they also have mmap(MAP_NOSYNC) to turn off the kernels' flushing.
If you use mmap(MAP_NOSYNC), you generaly need to use the msync() system call
to flush manually.

However, other UNIX-like ysstem have their ownhologies that I spent a
lot of time during the last few montbs fighting.   Some of what I did
for them involved keeping more careful track of which parts of which
mmap() buffers are dirty and need to be flushed.  On the Solaris systems
where I've tested, `dccd -F` is now fastest.

If I'm wrong about this,
adding -f (small f) to DCCD_ARGS turns off -F (big F).

Vernon Schryver    vjs@rhyolite.com

More information about the DCC mailing list

Contact vjs@rhyolite.com by mail or use the form.