Best way to follow a log and execute a command when some text appears in the log

I have a server log that outputs a specific line of text into its log file when the server is up. I want to execute a command once the server is up, and hence do something like the following:

tail -f /path/to/serverLog | grep "server is up" ...(now, e.g., wget on server)?

What is the best way to do this?

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

A simple way would be awk.

tail -f /path/to/serverLog | awk '
                    /Printer is on fire!/ { system("shutdown -h now") }
                    /new USB high speed/  { system("echo "New USB" | mail admin") }'

And yes, both of those are real messages from a kernel log. Perl might be a little more elegant to use for this and can also replace the need for tail. If using perl, it will look something like this:

open(my $fd, "<", "/path/to/serverLog") or die "Can't open log";
while(1) {
    if(eof $fd) {
        sleep 1;
        $fd->clearerr;
        next;
    }
    my $line = <$fd>;
    chomp($line);
    if($line =~ /Printer is on fire!/) {
        system("shutdown -h now");
    } elsif($line =~ /new USB high speed/) {
        system("echo "New USB" | mail admin");
    }
}

Method 2

If you’re only looking for one possibility and want to stay mostly in the shell rather than using awk or perl, you could do something like:

tail -F /path/to/serverLog | 
grep --line-buffered 'server is up' | 
while read ; do my_command ; done

…which will run my_command every time “server is up” appears in the log file. For multiple possibilities, you could maybe drop the grep and instead use a case within the while.

The capital -F tells tail to watch for the log file to be rotated; i.e. if the current file gets renamed and another file with the same name takes its place, tail will switch over to the new file.

The --line-buffered option tells grep to flush its buffer after every line; otherwise, my_command may not be reached in a timely fashion (assuming the logs have reasonably sized lines).

Method 3

This question appears to be answered already, but I think there’s a better solution.

Rather than tail | whatever, I think what you really want is swatch. Swatch is a program designed explicitly for doing what you’re asking, watching a log file and executing actions based on log lines. Using tail|foo will require that you’ve got a terminal actively running to do this. Swatch on the other hand runs as a daemon and will always be watching your logs. Swatch is available in all Linux distros,

I encourage you to try it out. While you can pound a nail in with the back side of a screwdriver does not mean you should.

The best 30-second tutorial on swatch I could find is here.

Method 4

It is strange that no one mentioned about multitail utility which has this functionality out-of-box. One of usage example:

Show the output of a ping-command and if it displays a timeout, send a message to all users currently logged in

multitail -ex timeout "echo timeout | wall" -l "ping 192.168.0.1"

See also another examples of multitail usage.

Method 5

bash could do the job by himself

Let see how simple and readable it could be:

mylog() {
    echo >>/path/to/myscriptLog "<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="3c187c">[email protected]</a>"
}

while read line;do
    case "$line" in
        *"Printer on fire"* )
            mylog Halting immediately
            shutdown -h now
            ;;
        *DHCPREQUEST* )
            [[ "$line" =~ DHCPREQUEST for ([^ ]*)  ]]
            mylog Incomming or refresh for ${BASH_REMATCH[1]}
            $HOME/SomethingWithNewClient ${BASH_REMATCH[1]}
            ;;
        * )
            mylog "untrapped entry: $line"
            ;;
    esac
  done < <(tail -f /path/to/logfile)

While you don’t use bash’s regex, this could stay very quick!

But bash + sed is a very efficient and interesting tandem

But for high load server, and as I like sed because it’s very quick and very scalable, I often use this:

while read event target lost ; do
    case $event in
        NEW )
            ip2int $target intTarget
            ((count[intTarget]++))
        ...

    esac
done < <(tail -f /path/logfile | sed -une '
  s/^.*New incom.*from ip ([0-9.]+) .*$/NEW 1/p;
  s/^.*Auth.*ip ([0-9.]+) failed./FAIL 1/p;
  ...
')

Method 6

That’s how i started doing this too but have become much more sophisticated with it. A couple things to be concerned with:

1. If the tail of the log already contains “server is up”.
2. Automatically ending the tail process once it’s found.

I use something along the lines of this:

RELEASE=/tmp/${RANDOM}$$
(
  trap 'false' 1
  trap "rm -f ${RELEASE}" 0
  while ! [ -s ${RELEASE} ]; do sleep 3; done
  # You can put code here if you want to do something
  # once the grep succeeds.
) & wait_pid=$!
tail --pid=${wait_pid} -F /path/to/serverLog 
| sed "1,10d" 
| grep "server is up" > ${RELEASE}

It works by holding tail open until the ${RELEASE} file contains data.

Once the grep succeeds it:

1. writes the output to ${RELEASE} which will
2. terminate the ${wait_pid} process to
3. exit the tail

Note: The sed can be more sophisticated to actually determine the number of lines tail will produce at startup and the remove that number. But generally, it’s 10.

Method 7

I use fail2ban to monitor and execute a command

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0