How can I limit the output speed of stdout?

I’m running CentOS 5.7 and I have a backup utility that has the option of dumping its backup file to stdout. The backup file is rather large (multiple gigabytes). The target is an SSHFS filesystem. To ensure that I don’t hog the bandwidth and degrade the performance of the network, I would like to limit the speed with which data is written to the “disk”.

How can I limit the ability of stdout based on a byte number? For example, limiting a process’s ability to write to about 768Bps.

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

You can add a rate limiting tool to your pipeline.

For example there is pv which has a rate-limiting option:

-L RATE, --rate-limit RATE

Limit the transfer to a maximum of RATE bytes per second. A suffix of “k”, “m”, “g”, or “t” can be added to denote kilobytes (*1024), megabytes, and so on

An alternative is the tool buffer which has:

   -u microseconds

After every write pause for this many microseconds. Defaults to zero. (Surprisingly a small sleep, 100 usecs, after each write can greatly
enhance throughput on some drives.)

Method 2

I would be tempted to dump this on a local disk instead of writing it directly to the remote volume. This way you can use rsync with --bwlimit to easily limit the transfer.

One of the benefits to this approach is that if there is a connectivity hiccup to the remote machine, the transfer is interrupted, but the backup itself doesn’t explode in a blaze of glory.

Method 3

pv (pipe view)

echo asdlfkjasdf | pv -q -L 12

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0