If I want to tcpdump DNS requests by clients (on an OpenWrt 10.04 router), then I
<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="30425f5f4470627f65647562">[email protected]</a>:/etc# tcpdump -n -i br-lan dst port 53 2>&1 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on br-lan, link-type EN10MB (Ethernet), capture size 96 bytes 22:29:38.989412 IP 192.168.1.200.55919 > 192.168.1.1.53: 5697+ A? foo.org. (25) 22:29:39.538981 IP 192.168.1.200.60071 > 192.168.1.1.53: 17481+ PTR? 150.33.87.208.in-addr.arpa. (44) ^C 2 packets captured 3 packets received by filter 0 packets dropped by kernel
That’s fully ok. But. Why can’t I pipe the tcpdumps output in realtime?
<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="a2d0cdcdd6e2f0edf7f6e7f0">[email protected]</a>:/etc# tcpdump -n -i br-lan dst port 53 2>&1 | awk '/?/ {print $3}'
^C
<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="d4a6bbbba094869b81809186">[email protected]</a>:/etc#
If I awk, etc. anything after tcpdump, I don’t get ANY output. Why is that? Why can’t I process the output of tcpdump with pipelining in realtime? (so that e.g.: in the example in only outputs the 3rd column)
Are there any solutions for this?
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
Straight out of man tcpdump
-l Make stdout line buffered. Useful if you want to see the data while
capturing it. E.g.,
tcpdump -l | tee dat
or
tcpdump -l > dat & tail -f dat
Note that on Windows,``line buffered'' means ``unbuffered'', so that
WinDump will write each character individually if -l is specified.
-U is similar to -l in its behavior, but it will cause output to be
``packet-buffered'', so that the output is written to stdout at the
end of each packet rather than at the end of each line; this is
buffered on all platforms, including Windows.
Method 2
Use the option -U in combination with -w so that tcpdump
writes packets immediately.
Method 3
The tcpdump is apparently buffering output when it writes to a pipe. It’s not flushing output for each write, so the system will write the output in about 4k byte chunks. Your filter is limiting out put so you won’t see anything until that filter has written enough output. Once it collects enough it will be written out in a chunk and you should see several lines emitted then.
Try to trigger DNS lookups many times and see what happens then.
Method 4
I’m building a real-time monitoring wrapper around tcpdump that needs to see packets as soon as they are available. Even with -l there is some delay.
tcpdump now has --immediate-mode, which solved this problem for me. In order to get it to work I used it in conjunction with -l.
See this answer.
Method 5
expect has an unbuffer command to fool commands into assuming they are writing to a tty, so they don’t buffer.
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0