When writing content to removable devices on Linux (USB sticks/HDDs, SD cards, etc), I often see incredible write speeds in the first few seconds, sometimes in the order of GB/s (filling of the write buffer) followed by several minutes of quiet (buffer actually being written to the device).
It’s misleading. It makes it hard to judge the actual write speed. It’s annoying (can’t interrupt a program while it’s flushing buffers). It can cause FS damage, and at the very least unwritten content when the device is unplugged while it’s still being written.
The write buffer appears to be over a GB in my case. Is there a way to control it? I’d like to reduce it to a more sensible value.
Note: This is not about filesystem cache, this effect also appears on raw devices
Note: I do not want to disable write buffers entirely
Thanks!
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
I found the answer. 64-bit Linux maintains a large write buffer (20% of available memory!) by default. (Interestingly, 32-bit Linux limits itself to at most 180MB) To change the dirty buffer size to e.g. 200MB, one can use
echo 200000000 > /proc/sys/vm/dirty_bytes
OR to use a percentage of RAM, e.g. 1%:
echo 1 > /proc/sys/vm/dirty_ratio
More information: https://lwn.net/Articles/572911/
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0