- New free VMware tool: Virtual Machine Desired State Configuration (VMDSC) - Fri, Oct 22 2021
- VMware vSAN Automatic Rebalance vs. Proactive Rebalance - Fri, Oct 15 2021
- VMware Tanzu Services overview - Fri, Oct 8 2021
My first optimization tip is that the CPU settings on your freshly unpacked hardware might not be optimized for your workloads. One of the best settings you can activate is when the host cooperates with the BIOS to determine the processor clock rate. HP systems that support this technology have a BIOS setting called Cooperative Power Management (enabled by default). I'll discuss a scenario later in the article when even this setting might not be optimal.
If your systems are already installed and in production, you'll need to check that the CPU settings are configured the way you want. There are two ways to check this, depending on whether you're using vCenter Server.
Where to check host CPU settings on ESXi 7.x and vSphere ^
You can use the ESXi host client by connecting via your web browser directly to your host. Then go to Manage > Hardware > Power Management > Change Policy.
If you're using vCenter Server to manage your vSphere infrastructure, you can connect via the vSphere Web Client. Select your ESXi host. Then select Configure / Hardware > Overview > Power Management.
If the host hardware does not allow ESXi software to manage power, then only the Not Supported policy is available, and everything else is grayed out. However, on some systems, you might only have the High-Performance policy as an option. If that's the case, it should be the option to use to get the best performance for your VMs.
Performance tip settings in BIOS
Update BIOS to the latest release—Update the BIOS to the latest release before you start installing ESXi. Your hardware manufacturer may have released new updates, which usually fix some bugs or allow the use of newer processors.
Enable Turbo Boost—If the processor supports it, you should enable Turbo Boost in the BIOS. Intel Turbo Boost Technology is a built-in software accelerator that can speed up the CPU depending on the workload. Whenever the workload requires more speed, Turbo boost accelerates the CPU up to the limit allowed in the tech specifications. If the CPU is working below its limits and the user's application needs more performance, the CPU is able to dynamically increase until the upper limit of frequency. To maximize performance for short periods of time, Turbo Boost Technology can make the processor operate at a power level that is higher than its TDP configuration and the power specified on the data sheet.
Enable All Sockets and Cores—Verify that the BIOS is configured to enable all populated processor sockets to be active and to enable all cores in each socket.
Enable Hyperthreading—Verify that hyperthreading is enabled in the BIOS for processors that support this technology. Hyperthreading is usually enabled, especially on server boards. Hyper-Threading Technology is an Intel hardware innovation that allows more than one thread to run on each core. And as you know, more threads mean more work can be done in parallel. When activated, one core can execute two threads, so if your CPU has 12 cores, then it is able to process 24 threads in parallel. One core works as two logical cores, allowing each to process one thread.
Enable VT-x, AMD-V, EPT, RVI—Verify that any hardware-assisted virtualization features such as VT-x, AMD-V, EPT, and RVI are enabled.
Disable unnecessary devices—Yes, this is also a BIOS optimization task. You should disable any device you will not be using, such as serial, USB, or network ports.
Memory scrubbing options—If the BIOS allows for the memory-scrubbing rate to be configured, VMware recommends leaving it at the manufacturer’s default setting. The technology reads from each computer memory location, correcting bit errors (if there are any) with an error-correcting code (ECC), and writing the corrected data back to the same location. It uses more power, so it is not a continuous operation; instead, it is a scheduled one. Memory scrubbing increases reliability.
You should check the VMware documentation, which might be appropriate for your workloads. Read this document, called Best Practices for Performance Tuning of Latency-Sensitive Workloads in vSphere Virtual Machines, and Performance Best Practices for VMware vSphere 7.0.
What's the difference between C states and P states? ^
C states allow the CPU to go into sleep mode whenever they are idle. This means that at some point, when it's not necessary to use full power, the system will lower CPU performance. If you do want to get the best performance per watt, you should enable all C states in BIOS.
P states make turbo mode work when something requires extra performance but doesn't need all the cores.
The way to go is to disable C states and enable P states. It is the setting called OS Control Mode. If you set High performance, the ESXi software won't use any optimization techniques, and the CPU will run full speed all the time.
When you enable the custom power policy, unlock all the advanced options. You can change the individual settings, such as C states or P states, one at a time.
Do you run latency-sensitive applications? ^
VMware introduced support for latency-sensitive applications back in vSphere 5.5 and keeps improving it with new features. Applications that do not support latency are used in banking, stock exchanges, and Java applications.
It is a per-VM feature called Latency Sensitivity. This feature allows the VM to exclusively own physical cores, which avoids overhead related to CPU scheduling and contention.
If you use VMs that are activated for low-latency workloads (it's a per-VM setting), your hosts should have the power management policy set to Maximum Performance. These settings will bypass the ESXi layer, and ESXi then won't manage the power of the CPUs at all.
Free tools and scripts ^
There are also free PowerShell scripts or free software that can be used to retrieve CPU settings and BIOS settings across your cluster.
One of them is RVTools, which is well-known freeware. Below you can see a screenshot showing the BIOS versions on several ESXi hosts within our cluster. The software can also show the configuration power management policy or CPU model.
Subscribe to 4sysops newsletter!
Final words ^
Monitoring BIOS and CPU settings is an important task for VMware administrators to ensure that applications run on performance-optimized hosts. I hope this short guide helps you get started with optimizing performance in your VMware environment.