sysklogd has performance problem with heavey network logging

From: FACQ Laurent (facq@u-bordeaux.fr)
Date: Tue Apr 08 2003 - 15:59:38 CEST

  • Next message: bdcmoat2@email.com: "Plump Out Your Resume"

    (note: i'm not in this list - if needed, please, cc to facq@u-bordeaux.fr)

            I found a problem in syslogd.c (from sysklogd-1.4.1) running on :

    Linux apu 2.4.17 #2 SMP Mon Jan 21 15:45:12 CET 2002 i686 unknown
    Debian GNU 3.0
    sysklogd_1.4.1-10

            i have exactly the same problem described by Rechenberg, Andrew (ARechenberg@shermanfinancialgroup.com)
            (cf. http://www.ussg.iu.edu/hypermail/linux/kernel/0209.1/0218.html)

            in short : my syslogd is very busy, receiving a lot from network (-r enable)
            and i get a lot of "syslogd: recvfrom inet: resource temporarily unavailable" in my logs files
            and the recv-Q is filling up...

    apu:~# netstat -an | grep 514 | grep udp
    Proto Recv-Q Send-Q Local Address Foreign Address State
    udp 65444 0 0.0.0.0:514 0.0.0.0:*

            I first reduced the sleep(10) to a sleep(1) because i thought this could be too long to wait
            for my busy syslogd - it was a lot better, but i always had
            this anoying message "syslogd: recvfrom inet: resource temporarily
            unavailable" filling my disks.

            so i added an exception, like for EINTR, for EAGAIN.

            it works... but it may introduce performance/DOS problems i didn't saw yet.

            the remaining questions is WHY select return inet socket is ready, but a recfrom
            gives EAGAIN error... may be a kernel problem. - i didn't
            understood the comment about 'BSDCOMPAT'

            Laurent FACQ. (facq@u-bordeaux.fr)

    apu:~/sysklogd-1.4.1# diff -c syslogd.c-orig syslogd.c
    *** syslogd.c-orig Sun Mar 11 20:40:10 2001
    --- syslogd.c Tue Apr 8 15:28:07 2003
    ***************
    *** 1141,1153 ****
                                     */
                                    printchopped(from, line, \
                                                 i + 2, finet);
    ! } else if (i < 0 && errno != EINTR) {
                                    dprintf("INET socket error: %d = %s.\n", \
                                            errno, strerror(errno));
                                    logerror("recvfrom inet");
                                    /* should be harmless now that we set
                                     * BSDCOMPAT on the socket */
    ! sleep(10);
                            }
                    }
      #endif
    --- 1141,1153 ----
                                     */
                                    printchopped(from, line, \
                                                 i + 2, finet);
    ! } else if (i < 0 && errno != EINTR && errno != EAGAIN) { /* @LF@ added EAGAIN exception */
                                    dprintf("INET socket error: %d = %s.\n", \
                                            errno, strerror(errno));
                                    logerror("recvfrom inet");
                                    /* should be harmless now that we set
                                     * BSDCOMPAT on the socket */
    ! sleep(1); /* @LF@ was 10 */
                            }
                    }
      #endif

    This archive was generated by hypermail 2.1.4 : Tue Apr 08 2003 - 17:17:23 CEST