I want to get just e-mail addresses that end in “@xyz.nl” from my mail logfile. To achieve this I do:
# tail -f /var/log/mail.log | grep --i --line-buffered "@xyz.nl" | cut -d '@' -f 1 | cut -d '<' -f 2
The –line-buffered with grep is necessary because it will otherwise buffer its output because the pipe is not considered a terminal. Grep will output lines like these:
Aug 29 11:56:01 localhost postfix/smtp[4124]: 05491500123: to=<<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="5c2f3331393332391c242526723230">[email protected]</a>>, relay=123.456.123.456[123.456.123.456]:25, delay=2, delays=0.4/0/0.4/1.2, dsn=2.0.0, status=sent (250 2.0.0 u7T9twxN074009 Message accepted for delivery)
The first cut then makes:
Aug 29 11:56:01 localhost postfix/smtp[4124]: 05491500123: to=<someone
The second cut should generate:
someone
However it seems that cut is also buffering. If I start the command with cat instead of tail -f I get all relevant results (in the prefered format) from the log file. But I need the results from the log file in real time.
I tried using unbuffer for this:
# tail -f /var/log/mail.log | grep --i --line-buffered "@xyz.nl" | unbuffer cut -d '@' -f 1 | cut -d '<' -f 2
Also tried:
# unbuffer tail -f /var/log/mail.log | grep --i --line-buffered "@xyz.nl" | unbuffer cut -d '@' -f 1 | cut -d '<' -f 2
…which should remove the 4K buffering from the first cut. However, this doesn’t work. I know it is buffering because if I grep for our local domain it gets a lot more hits, the buffer is filled sooner and output is generated earlier (in 4K batches).
So my question is: how do I unbuffer cut?
Related: I know sed and (g)awk can deliver the e-mail addresses to me. I have been trying but as yet without any result. Answers using sed or (g)awk are welcome and may solve my direct issue but I remain interested in the nominal answer of the question how to unbuffer the cut command. The cut command doesn’t speak of (un)buffering.
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
If you’re on a system using GNU Coreutils (almost any Linux), you can try stdbuf:
… | stdbuf -oL cut -d '@' -f 1 | …
-oL makes it line buffered, which seems like what you want.
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0