I am using daemontools to monitor a process and its output log. I am using multilog to write the logs to disk.
The run script for the log is:
#!/bin/bash PATH=/usr/local/bin:/usr/bin:/bin cd /usr/local/script_dir exec multilog t s16777215 n50 '!tai64nlocal' '!/bin/gzip' /var/log/script_log
The process being monitored, also writes output to stderr. So in the run script for the process there is the following lines to redirect stderr to stdout:
exec 2>&1 exec ./my_process
However, while tailing the log file, I see hundreds of lines of output coming in bursts (the monitored process writes output every few seconds), and the timestamp on the log lines differs in sub-microsecond levels. I know from the nature of the process that time difference between the log lines is not so small. Clearly multilog is buffering output and then adding the timestamp when it is ready to write to file. I would like the timestamps to more closely reflect the time at which the line was output. How can this be fixed?
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
The script being monitor was a Python script. To make all standard streams unbuffered, I found that one can just pass the -u option to the interpreter. This solved the problem in my case.
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0