Understanding vSphere shares, reservations, and limits

The primary goal of the ESXi hypervisor is to distribute physical resources to virtual machines (VMs) while ensuring fairness of consumption. Shares, reservations, and limits settings allow you to prioritize VMs that are more important when resources are under contention.
Contents of this article

Sharing physical server resources such as the CPU, memory, and storage is what makes virtualization so useful. You can run more virtual workloads on a single box while saving money and management on the hardware layer.

In reality, each VM has a different priority and relative importance (e.g., a VM that runs a production application is more important than your colleague's test VM). However, the default ESXi hypervisor behavior is to treat all like-configured machines the same way (a "fair share").

The CPU scheduler uses a complex accounting algorithm mainly determined by configured resources: shares, reservations, and limits. In this post, I will explain how each of these settings works.

Shares ^

Shares can be a relative priority when there is a resource contention. The hypervisor automatically sets them to each VM. Shares maintain a default ratio of 4:2:1 for high, normal, and low values as shown in the table below. All VMs have normal shares by default to ensure equality.

Default share values

Let's create two VMs as an example:

  • VM1 with 4 vCPUs and 8 GB of RAM
  • VM2 with 2 vCPUs and 8 GB of RAM

As all shares are set to normal by default, the VMs will have the following shares configured:

  • VM1 has 4,000 CPU shares and 81,920 memory shares
  • VM2 has 2,000 CPU shares and 81,920 memory shares

As you can see, VM1 has 2:1 CPU shares compared to VM2 just because you have added more resources to it. Now if there is a CPU competition, VM1 will get two times more CPU time than VM2. This is a fair share from the CPU scheduler's point of view.

However, what if VM2 is more important to you than VM1 even it has less vCPU configured, and you want to make sure it will have priority in the case of competition? Well, let's change it then!

To modify the share value of your VM, go to Edit Settings… and expand the CPU configuration. You can select Low, Normal, High, or define a Custom value.

Modify CPU shares

Modify CPU shares

I have set high shares for VM2. Now both VMs have 4,000 CPU shares. As I want VM2 to have more CPU time than VM1, I need to set the VM1 shares to low. As a result, I will have:

  • VM1 with 2,000 CPU shares
  • VM2 with 4,000 CPU shares

By changing the share settings, I have instructed the CPU scheduler to change its behavior and prioritize VM2 over VM1, which would have priority by default.

Memory shares work the same way. Since both of my VMs have the same memory share values, they will get equal resources.

Please remember: shares only apply when resources are under contention.

Reservations ^

By configuring reservations, you guarantee a specific amount of resources to the VM. If you need to make sure the specific VM will always have given amount of physical resources available, this is the right setting to configure.

The resources must be available at the time of VM power on; otherwise, it will fail to start! As reservations are applied on the fly, the same requirement is valid if the VM is already running.

Insufficient resources error message

Insufficient resources error message

There are some differences between CPU and memory reservations. If I set a full memory reservation for my VM1 and VM2, no other VM can use the physical memory, even if my machines do not use it. You can easily set a full memory reservation by checking the Reserve all guest memory (All locked) option.

Reserve all guest memory

Reserve all guest memory

You should also know that setting a memory reservation reduces the size of the VM swap file. The swap file size is equal to the configured RAM deducted by the reservation size. If you set Reserve all guest memory, there will be no swap file at all. This may be useful to save some storage space. Be aware that this only applies during the power cycle. If you configure the reservation on a running VM, you need to reboot to reduce the swap file.

On the other hand, if you set a CPU reservation for a VM and the VM is not using the CPU cycles, the CPU scheduler distributes CPU cycles to other active VMs. To be able to do this, the CPU scheduler calculates a MHzPerShare metric. However, it would be too complex to cover this in detail here.

Be careful; if you configure too many or too high reservations, you may negatively affect performance since you are limiting the ESXi resource scheduler to do its job.

Limits ^

You can configure a maximum of what the VM can consume. I may limit my VM1 to use only a maximum of 4 GB of physical memory and 1,000 MHz of CPU cycles. Now if the VM tries to access memory above the 4 GB limit, it will always swap to disk. For the CPU cycles, it will never get more than 1,000 MHz even if there are free resources on my host.

Limits may be useful in a test environment, and you should not configure them without reason. Personally, I have never used them.

Wrap-up ^

The ESXi hypervisor does a great job of distributing physical resources by default. In case you need to modify it for your specific needs, make sure you monitor the VM performance afterward, so your changes do not have a negative performance impact.

Want to write for 4sysops? We are looking for new authors.

Read 4sysops without ads and for free by becoming a member!

8+
avataravataravatar
Share
0 Comments

Leave a reply

Your email address will not be published. Required fields are marked *

*

© 4sysops 2006 - 2020

CONTACT US

Please ask IT administration questions in the forums. Any other messages are welcome.

Sending

Log in with your credentials

or    

Forgot your details?

Create Account