I’m trying to see how many times foo bar appears in /var/log/foo.log within an arbitrary amount of time on a remote server, but nothing that I’ve tried so far has worked.
I’ve already got a timer script that I use to keep track of how long it has been since I started tailing /var/log/foo.log, and now I’d just like a way to tell how many times foo bar has appeared in the tailed output.
I searched google, but I didn’t find anything pertinent within the first 10 pages of results.
Here’s what I’ve tried with frustrating results:
## works on local machine, but doesn't work as expected on remote
tail -f /var/log/foo.log | grep foo bar | sed '='
## works on local, but not remote
tail -f /var/log/foo.log | grep foo bar | cat -n -
## works on local, but not remote
tail -f /var/log/foo.log | grep foo bar | awk -F'n' '{printf "[%d]> ", NR; print $1}'
I even tried to write a sed script that’d act like tail -f, but I made limited-to-no headway with that.
NOTE
the remote server is running an older version of coreutils, and upgrading is an option, but is NOT in any way the desired solution.
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
tail -f | nl
works for me and is the first what I thought of – that is if you really want the lines numbered from 1 and not with the real line number from the file watched. Optionally add grep if needed to the appropriate place (either before or after nl). However, remember that buffering may occur. In my particular case, grep has the --line-buffered option, but nl buffers it’s output and doesn’t have an option to switch that off. Hence the tail | nl | grep combo doesn’t really flow nicely.
That said,
tail -f | grep -n pattern
works for me as well. Numbering starts again from the beginning of the “tailing” rather than beginning of the whole log file.
Method 2
I think this is better:
less -N +F <filepath>
Method 3
You can also pipe the output to less, it has a line number feature, -N which would allow you to scroll back and forth through the log.
$ tail -f /var/log/foo.log | less -N
Example
1 Jan 17 22:11:58 greeneggs fprintd[4323]: ** Message: entering main loop 2 Jan 17 22:12:01 greeneggs su: (to root) saml on pts/5 3 Jan 17 22:12:28 greeneggs fprintd[4323]: ** Message: No devices in use, exit 4 Jan 17 22:12:56 greeneggs gnome-session[1876]: 22:12:56 | Git | personal_repo | Checking for remote changes... 5 Jan 17 22:12:56 greeneggs gnome-session[1876]: 22:12:56 | Cmd | personal_repo | git rev-parse HEAD 6 Jan 17 22:12:56 greeneggs gnome-session[1876]: 22:12:56 | Cmd | personal_repo | git ls-remote --heads --exit-code "ssh://<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="95e6f4f8d5e6e5f4e7fef9f0e6fdf4e7f0bbfff4fef0">[email protected]</a> 6 8us.org/home/sam/SparkleShare/personal_repo.git" master 7 Jan 17 22:12:58 greeneggs gnome-session[1876]: X11 forwarding request failed on channel 1 8 Jan 17 22:12:58 greeneggs gnome-session[1876]: 22:12:58 | Git | personal_repo | No remote changes, local+remote: 532213be48cce3b93cb177d409faa 8 03b71d0cfa5 9 Jan 17 22:13:35 greeneggs gnome-session[1876]: 22:13:35 | ListenerTcp | Pinging tcp://notifications.sparkleshare.org:443/ 10 Jan 17 22:13:35 greeneggs gnome-session[1876]: 22:13:35 | ListenerTcp | Received pong from tcp://notifications.sparkleshare.org:443/
NOTE: Take notice of the output. You may or may not like this feature, but it will take long lines and chop them so that they continue on the next line, but still maintain the same corresponding line number. I find this feature invaluable when parsing log files that are wide! You can see the effect of this feature on lines 6 & 8.
Method 4
To grep new lines only in the log file as they come with their line number, you could do:
{
initial_lines=$(wc -l)
tail -n +1 -f | awk -v NR="$initial_lines" '/pattern/{print NR": "$0}'
} < file.log
(with mawk, you’ll want to add the -Winteractive option to prevent its input (!) buffering).
wc -l reads the lines that were already there and counts them (the newline characters which means it still works even if the last line is not full yet), and then we tail -f the rest (starting from where wc stopped reading) and tell awk what is the line number of the first one it sees.
Method 5
If you wanted to number from the beginning you’d need grep -n to apply to all lines.
tail -f -n100000000 filename.log | grep -n ''
If you then only wanted to show the last 10 I’d have thought you could re-tail the file:
tail -f -n100000000 filename.log | grep -n '' | tail -n10
The first is handy, but shows too much output. I don’t know why the second one doesn’t work.
Method 6
It’s the argument n or --lines (used slightly different ways, see below):
$ tail -f -n 25 /path/to/file.txt $ tail -f --lines=25 /path/to/file.txt
See also the help:
$ tail --help
Method 7
The command cat -n [filename] | tail will get a quick count and display of the most recent records if that’s what you’re looking for.
The -f switch makes it persistent until escape – which really doesn’t sound applicable in your scenario or is redundant.
wc -l [filename] will get a count of the lines in the target
wc -l [filenameprefix]* will count all lines in all files matching the pattern and even report a summary total at the end.
More complete detail can yield more complete responses.
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0