Is there a command like “tee” that limits the file size and treats the output file like a fixed-size queue?

Does there exist a command where one can do something like:

someprogram | tee --limit=1MB afile

Whereby “afile” would contain the most recent 1 MB of output of “someprogram”?

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

Exactly that, probably not. Once it had warmed up and filled the file, it would be rewriting 1MiB-N bytes of old data, to shuffle them along the file, for every N bytes of new data that it added to the end of the buffer. This is not exactly an ideal mechanism, and moreover reading the file as it is being written would be prone to “tearing” problems.

Close to that, many commands exist. Raise the number of files to 2, a current file and its immediate predecessor, so that at any given time one has between 1MiB and 2MiB of the most recent output, with output not being copied around once it has been written; and you have something that a whole bunch of daemontools-family logging programs do by design. They are, in essence, exactly this sort of “tee plus”.

• With Dan Bernstein’s multilog from daemontools and Bruce Guenter’s multilog from daemontools-encore:

  someprogram | multilog n2 s1048576 ./logdir/

• With Laurent Bercot’s s6-log from s6:

  someprogram | s6-log n2 s1048576 ./logdir/

• Gerrit Pape’s svlogd from runit with a configuration file that says n2 s1048576:

  someprogram | svlogd ./logdir/

• Wayne Marshall’s tinylog from perp:

  someprogram | tinylog -k 1 -s 1048576 ./logdir/

• With my cyclog from nosh:

  someprogram | cyclog --max-file-size 1048576 --max-total-size 1048576 ./logdir/

Further reading

• Jonathan de Boyne Pollard (2015). “Logging“. The daemontools family. Frequently Given Answers.
• Jonathan de Boyne Pollard (2017). Don’t use logrotate or newsyslog in this century.. Frequently Given Answers.
• https://unix.stackexchange.com/a/326166/5132

Method 2

In package apache2-utils is present utility called rotatelogs, it fully meet to your requirements.

Synopsis:

rotatelogs [ -l ] [ -L linkname ] [ -p program ] [ -f ] [ -t ] [ -v ] [ -e ] [ -c ] [ -n number-of-files ] logfile rotationtime|filesize(B|K|M|G) [ offset ]

Full manual you may read on this link.

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0