dccm running out of file descriptors

Gary Mills mills@cc.UManitoba.CA
Mon Feb 2 14:52:09 UTC 2004


On Sat, Jan 31, 2004 at 05:08:08PM -0700, Vernon Schryver wrote:
> > From: Gary Mills 
> 
> `netstat -a` while the system is sick would be interesting. 
> 3000 sockets in the same state should stand out.  My bet is that
> `netstat -a` will say they are in Close_Wait.

Well, we had another incident this morning, starting at 05:33:18.
It was still going on at 8 am when I noticed it.  Here's the TCP
connection states from `netstat' while it was going on:

  12 CLOSE_WAIT
   4 CLOSING
 770 ESTABLISHED
   4 FIN_WAIT_1
  23 FIN_WAIT_2
2832 IDLE
  22 LAST_ACK
  50 LISTEN
   5 SYN_RCVD
  35 SYN_SENT
2319 TIME_WAIT

The netstat output has 2832 connections in the IDLE state.  `lsof' of
`dccm' has 2822 in that state.  `netstat' shows them this way:

TCP: IPv4
   Local Address        Remote Address    Swind Send-Q Rwind Recv-Q  State
-------------------- -------------------- ----- ------ ----- ------ -------
      *.*                  *.*                0      0 24576      0 IDLE

`lsof' shows them this way:

COMMAND   PID   USER   FD   TYPE        DEVICE SIZE/OFF    NODE NAME
dccm    14294 daemon 3667u  IPv4 0x3002d0361f0      0t0     TCP electra.cc.umanitoba.ca:*->electra.cc.umanitoba.ca:* (IDLE)

> >                    Here's this morning's error messages.  First, eight
> > of these:
> >
> > Jan 31 00:58:46 electra dccm[20546]: [ID 109917 mail.error] DCC, mi_rd_cmd: read returned -1: Connection reset by peer
> 
> mi_rd_cmd() is in the sendmail libmilter.
> It would be nice to hear that there were 3000 of those messages, perhaps
> collapsed by syslog into a few "last message repeated 1000 times" entries.

Only 491, starting at 01:14:29.

Here are the TCP connection states after DCC was restarted:

# netstat -an -P tcp | awk '{print $7}' | sort | uniq -c | sed '1d;/[-,n,m]/d'
  39 CLOSE_WAIT
  10 CLOSING
 730 ESTABLISHED
   4 FIN_WAIT_1
  69 FIN_WAIT_2
  10 IDLE
  15 LAST_ACK
  50 LISTEN
   7 SYN_RCVD
  52 SYN_SENT
2285 TIME_WAIT

-- 
-Gary Mills-    -Unix Support-    -U of M Academic Computing and Networking-



More information about the DCC mailing list

Contact vjs@rhyolite.com by mail or use the form.