Re: questions/suggestion "RAM-only" log support in sysklogd?

Martin Schulze wrote:
> Hi!
>
> guy keren wrote:
>> i was looking for a way to have some log messages (mostly debug
>> messages, that i don't want to flood the disk normally) be stored in
>> a cyclic memory buffer, instead of logged into a file - and only
>> have this buffer dumped to disk upon demand (e.g. when there is a
>> problem that requires debugging).
>
> I don't know of such a feature.
>
> However, using a named pipe and attaching a postprocessing process
> (e.g. Perl or Python script) you should easily be able to achieve this
> feature.

i was considering this. my main concern with pipes (perhaps i'm simply
paranoid) is that they have a limited buffer space, and if the external
process doesn't work fast enough at some point - it could hurt the
working of syslogd. however, i will re-consider this.

do you know if it is possible to modify the amount of buffer space
allocated for a specific named-pipe in the system? i'll conduct a search
and see if this is configurable.

>> i was wondering if a patch that adds such support is available
>> anywhere, or if it will be considered for inclusion in sysklogd if
>> provided.
>
> Such a patch would at least be stored in the contrib area so that
> other users can make use of it as well. Your patch would be
> appreciated in any case.
>
>> the motivation is being able to debug various problems with various
>> software components the first time around - instead of having to
>> re-produce them with higher debug levels - and yet without filling
>> the disk with lots of junk and without causing too much disk
>> activity when there's no need to look at these logs.
>
> You could dump log messages to a solid state device or a file in tmpfs
> to avoid disk activity as well.
>
>> note: we did consider simply placing certain log files in a RAM disk
>> - but we then we'll need to handle the creation of this RAM disk,
>> very often logrotate (to avoid having it fill up), etc - and it
>> seems like a more awkward solution, and less robust in case of a
>> sudden log-flood.
>
> You could combine the ram disk with a named pipe and a process that
> flushes log messages to the ram disk after a certain amount of
> messages have accumulated.
>
> I believe you could solve your problem with a named pipe and a
> post-processing script. I would be glad to place such a script to the
> contrib area as well.

the idea is to have the data dumped to disk ONLY on demand - i.e. in
normal situations, the data will be in a cyclic buffer and old messages
will be over-written by new ones. ONLY if the user specifically asks for
this - the contents of the existing buffer (i plan to have several 10s
of MB of data there) will be written down to a normal file.

thanks for the suggestion - i'll check if this works good enough for us,
and will report back to the list.

--guy
Received on Wed Dec 23 2009 - 00:26:09 CET