In the past, on linux systems, I’ve been able to truncate large, open log files (that is, a file that is being actively written to by a process) using cat /dev/null > file.log.
However, on 10.9 (Mavericks), that doesn’t seem to be the case. I’ve got an 11GB file that is being logged to by an application, but when I perform the same command with said file, nothing seems to happen.
When I try this on a file of trivial size, it does work.
Here is ls -l /dev/null:
crw-rw-rw- 1 root wheel 3, 2 Dec 16 12:49 /dev/null
I’ve also tried cp /dev/null file.log to no avail.
Thinking that I might take advantage of the truncate function (man 2 truncate in Darwin) I compiled this and ran it against two files, one of trivial size and the other the actual log file. Again, it worked against the trivial file and did not work on the much larger log.
/*
* Copyright (c) 2013 Thomas de Grivel <<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="07736f686a6674476b68706f29696273">[email protected]</a>>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
...
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <unistd.h>
int main (int argc, const char **argv)
{
int e = 0;
while (--argc) {
argv++;
if (truncate(*argv, 0)) {
e = 4;
warn("%s", *argv);
}
}
return e;
}
The process returns 0 regardless of which file I use.
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
cat /dev/null is a bit convoluted a way to write a command that produces no output. : or true are more obvious ones.
In all of cat /dev/null > file, : > file, and even > file in most shells, the shell opens file with O_TRUNC on stdout, then runs the application which doesn’t output anything, then the file is closed and left truncated.
However, in that case or when using the truncate system call, if the process that is filling up that file didn’t open it with the O_APPEND flag, the next time it writes to the file descriptor it has open on the file, it will write the data at the offset it was within the file.
Because HFS+ doesn’t support sparse files, that means that the space before that offset will have to be reallocated and filled with zeros by the system.
So, you need to kill the application that is writing to that file before truncating it. Or you need to make sure the application opens the file with O_APPEND (like with >> if using shell redirection).
If you want to experiment with it:
$ exec 3> x $ yes | head -n 50000 >&3 $ ls -ls x 200 -rw-r--r-- 1 me me 100000 Dec 16 21:32 x
Now the fd 3 of my shell is 100000 bytes within the file
$ : > x $ ls -ls x 0 -rw-r--r-- 1 me me 0 Dec 16 21:34 x
Now the file is truncated (size 0, no space used on disk).
$ echo >&3 $ ls -ls x 200 -rw-r--r-- 1 me me 100001 Dec 16 21:34 x
Writing 1 byte to the file at offset 100000, the file is now
100001 bytes large, the first ones all zeros, would use over 100k
on HFS+, but about just one disk block in most other Unix file systems
On the other hand, with:
$ exec 3>> x $ yes | head -n 50000 >&3 $ ls -ls x 200 -rw-r--r-- 1 me me 100000 Dec 16 21:35 x $ : > x $ echo >&3 $ ls -ls x 8 -rw-r--r-- 1 me me 1 Dec 16 21:36 x
Writing 1 byte to the file not at offset 100000, but at the end of the file
because of O_APPEND. The file is 1 byte large, and takes the space needed
to hold that one byte.
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0