- Pip install Boto3 - Thu, Mar 24 2022
- Install Boto3 (AWS SDK for Python) in Visual Studio Code (VS Code) on Windows - Wed, Feb 23 2022
- Automatically mount an NVMe EBS volume in an EC2 Linux instance using fstab - Mon, Feb 21 2022
Perhaps the most common usage scenario for cpulimit is the throttling of a backup program. You probably don't mind if your backup takes a couple of minutes longer. Thus, it doesn't make sense to slow down a busy server and let end users wait just to make a backup program finish as fast as possible.
Let's say you create your backups with tar and compress them with gzip. You will notice that gzip is a real resource hog and will quickly seize all available CPU resources. This is where cpulimit comes in.
The installation of cpulimit is pretty simple. On Ubuntu, you just run the following commands:
apt update apt install cpulimit
Most examples on the web will look more or less like this:
tar -c myData | gzip -c > myData.gz & cpulimit -l 10 -e gzip
Because tar does not use much CPU power, we only need to throttle gzip. The -e parameter tells cpulimit to search for a process named gzip, and -l specifies the percentage of CPU allowed. In our case, cpulimit can only use 10% of the available CPU resources.
The example above has two problems. First, you can't be entirely sure that no other gzip process is running. cpulimit might then catch the wrong process. Second, on a busy server, cpulimit might start before gzip appears on the process list. In that case, cpulimit will exit with the message "No process found."
One way to solve the second problem is to add the sleep command after the first line. If you wait for three seconds or so, gzip most likely will be running. However, this solution doesn't solve the first problem. A solution to both issues is this:
tar -c myData | gzip -c > myData.gz & cpulimit -l 10 -p $!
Instead of the -e parameter, I've been using -p, which allows you to pass the process ID of the process that you want to throttle. The $! variable refers to the process ID of the most recently executed background command.
Two gzip processes are running where one has been throttled using cpulimit
Perhaps you only want to limit the CPU usage of gzip if there is too much load on your server:
#!/bin/bash
LIMIT="0.5"
L1="$(uptime | awk -F "load average:" '{ print $2 }' | cut -d, -f1)"
RESULT=$(echo "$L1 > $LIMIT" | bc)
tar -c myData | gzip -c > myData.gz &
if [ "$RESULT" == "1" ]; then
cpulimit -b -l 10 -p $!
fi
I am using the uptime tool here, which gives us information about the average load over a period of one, five, and 15 minutes. The output is essentially the same as the first line of the top command which you can see in the screenshot above.
With awk, we can extract information from a string. I am searching first for "load average:" and with print $2 , I am getting the second field after the search string, which corresponds to the three average load times.
Those three load times are then passed to the cut command. I am using the field delimiter "," for cut, and then I get the first field of the three average load times, which corresponds to the average load in the last minute.
For a system with one core, the value 1 would mean that the CPU is busy 100% of the time. A value greater than 1 indicates that processes had to wait for CPU time. If you have eight cores, a value of 8 means on average the system was running at full capacity for one minute. You can use the LIMIT variable to configure the average load above which you want cpulimit to kick in. I added the -b parameter to make cpulimit run in the background.
The bc command I am using in the $RESULT variable allows me to compare the "size" of the two strings. You might first have to install bc on your Linux machine.
Subscribe to 4sysops newsletter!
Notice that I didn't cover all the parameters that cpulimit supports.
