I operate a Linux system which has a lot of users but sometimes an abuse occurs; where a user might run a single process that uses up more than 80% of the CPU/Memory.
So is there a way to prevent this from happening by limiting the amount of CPU usage a process can use (to 10% for example)? I’m aware of cpulimit, but it unfortunately applies the limit to the processes I instruct it to limit (e.g single processes). So my question is, how can I apply the limit to all of the running processes and processes that will be run in the future without the need of providing their id/path for example?
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
nice / renice
nice is a great tool for ‘one off’ tweaks to a system.
nice COMMAND
cpulimit
cpulimit if you need to run a CPU intensive job and having free CPU time is essential for the responsiveness of a system.
cpulimit -l 50 -- COMMAND
cgroups
cgroups apply limits to a set of processes, rather than to just one
cgcreate -g cpu:/cpulimited cgset -r cpu.shares=512 cpulimited cgexec -g cpu:cpulimited COMMAND_1 cgexec -g cpu:cpulimited COMMAND_2 cgexec -g cpu:cpulimited COMMAND_3
Resources
http://blog.scoutapp.com/articles/2014/11/04/restricting-process-cpu-usage-using-nice-cpulimit-and-cgroups
http://manpages.ubuntu.com/manpages/xenial/man1/cpulimit.1.html
Method 2
While it can be an abuse for memory, it isn’t for CPU: when a CPU is idle, a running process (by “running”, I mean that the process isn’t waiting for I/O or something else) will take 100% CPU time by default. And there’s no reason to enforce a limit.
Now, you can set up priorities thanks to nice. If you want them to apply to all processes for a given user, you just need to make sure that his login shell is run with nice: the child processes will inherit the nice value. This depends on how the users log in. See Prioritise ssh logins (nice) for instance.
Alternatively, you can set up virtual machines. Indeed setting a per-process limit doesn’t make much sense since the user can start many processes, abusing the system. With a virtual machine, all the limits will be global to the virtual machine.
Another solution is to set /etc/security/limits.conf limits; see the limits.conf(5) man page. For instance, you can set the maximum CPU time per login and/or the maximum number of processes per login. You can also set maxlogins to 1 for each user.
Method 3
Did you look at cgroups? There is some information on the Arch Wiki about them. Read the section about cpu.shares, it looks like it’s doing what you need, and they can operate on a user-level, so you can limit all user processes at once.
Method 4
For memory, what you are looking for is ulimit -v. Note that ulimit is inherited by child processes, so if you apply it to the login shell of the user at the time of login, it applies to all his processes.
If your users all use bash as login shell, putting the following line in /etc/profile should cause all user processes to have a hard limit of 1 gigabyte (more exactly, one million kilobytes):
ulimit -vH 1000000
The option H makes sure it’s a hard limit, that is, the user cannot set it back up afterwards. Of course the user can still fill memory by starting sufficiently many processes at once.
For other shells, you’ll have to find out what initialization files they read instead (and what other command instead of ulimit they use).
For CPU, what you wish for doesn’t seem to make sense for me. What would be the use of letting 90% of the CPU unused when only one process is running? I think what you really want is nice (and possibly ionice). Note that, like ulimit, nice values are inherited by child processes, so applying it to the login shell at login time suffices. I guess that also applies to ionice but I’m not sure.
Method 5
Since your tags have centos, you can use systemd.
For example if you want to limit user with ID of 1234:
sudo systemctl edit --force user-1234.slice
Then type and save this:
[Slice]
CPUQuota=10%
Next time that user logs in, it will affect.
Man pages: systemctl, systemd.slice, systemd.resource-control…
Method 6
Since you are stating that cpulimit would not be practical in your case, then I suggest you look at nice, renice, and taskset, which may come close to what you want to achieve, although taskset allows to set a processes’s CPU affinity, so it might be not immediately helpful in your case.
Method 7
If you want to limit the processes that are already started, you will have to do it one by one by PID, but you can have a batch script to do that like the one below:
#!/bin/bash
LIMIT_PIDS=$(pgrep tesseract) # PIDs in queue replace tesseract with your name
echo $LIMIT_PIDS
for i in $LIMIT_PIDS
do
cpulimit -p $i -l 10 -z & # to 10 percent processes
done
In my case pypdfocr launches the greedy tesseract.
Also in some cases were your CPU is pretty good you can just use a renice like this:
watch -n5 'pidof tesseract | xargs -L1 sudo renice +19'
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0