linux - Capture logrotated files on change

05
2014-04

BeowulfOF

This is something rather special, but I bet you can help me out.

I'm working in an environment where many logfiles are generated and rotated, mainly from java-services runnning. As we are performing L&P tests on that environment, we often have the system generated more files, than are kept by logrotation. I now need a way to archive the rotated logs before they are deleted, to be able to have all logs generated during the test.

But I do not have the permissions to change the logrotation in any way, so I have to do it with bash and basic UNIX tools. Additionally, since it is Load-, and Performance tests the solution needs an absolutely small footprint regarding CPU and Memory usage.

First I considered using a hash like sha1sum to keep the hash and save all files that have altered, but generating and checking the hash seems somewhat time and CPU consuming.

This question is not necessarily about real code, but about a concept - although a code example would be very welcome.

Answers

MariusMatutiae

You should use inoticoming. It is in the repos for Debian-family distros. If you are on another distro, you can probably compile from source.

SYNOPSIS

inoticoming [ global-options ] directory actions*

DESCRIPTION

Inoticoming is a daemon to watch a directory with Linux's inotify framework and trigger actions once files with specific names are placed in there.

You can use it as follows:

   inotcoming --initialsearch /directory/with/log/files --regexp \
   last_log_rotate_file10.txt mv {} /directory/to/store/log_rotate_files/`date +"%T"` \;

The action must be terminated by \;;

the option --initialsearch performs the following action as soon as the daemon starts;

the --regexp precedes a regular expression to be satisfied by the name of the file you wish to store elsewhere; here I assumed you know that the 10th log_rotate_file is the last one that will be kept by your system;

then the command follows, with {} identifying the file name which was matched by the --regexp option, and I have assumed you will want to move the file to a new directory, with a name reflecting the time of the operation. You can adjust this at will.

Related Question

linux - logrotate configuration file syntax - multiple wildcard entries possible?

linux debian logging debian-squeeze logrotate

0xC0000022L

Since the man page doesn't answer my question and I don't want to force a rotation cycle, I decided to ask the question here.

The man page for logrotate gives the following example:

   "/var/log/httpd/access.log" /var/log/httpd/error.log {
       rotate 5
       mail [email protected]
       size 100k
       sharedscripts
       postrotate
           /usr/bin/killall -HUP httpd
       endscript
   }

All examples with wildcards contain only a single entry. Now, what I'm interested in is whether this one is also allowed:

   /var/log/httpd/*.log /var/log/httpd/*/*.log {
       # ... same as above
   }

Here's the reasoning: I have multiple vhosts and I split them up by the user that "owns" those vhosts. Since the log files are world-readable, I want to bind-mount a folder into the user home directory, but limit it to the log files that the user "owns", which is easiest achieved by separating the logs into folders (and bind-mounting requires that scheme anyway). So I'm looking for a solution to rotate both the log files under /var/log/httpd as well as all log files under subdirectories of that directory - without having to list each and every subdirectory by name.

In general the man page gives no clue whether multiple entries are at all possible for wildcard rules or only for full paths. I'm using logrotate version 3.7.8-6 which comes with Debian "Squeeze", but I reckon this is not necessarily specific to a distro or program version.

Home

linux - Capture logrotated files on change