We have a vast topic today. We'll discuss some VMware VM performance tips and tweaks. Application users often complain about performance. Before telling them that everything is OK, let's have a look what you should check at the VM level before searching for performance improvements elsewhere (storage, network).

One important factor in performance tuning is the number of applications and services that run on a particular VM. Think about a desktop PC where you see a user running 100 tabs on a Chrome browser, complaining that their machine is slow. We all know that Chrome is memory hungry, so most likely the machine lacks crucial physical memory in this example.

The rule of thumb for achieving the best performance in VM is to strip the number of applications and services running to the bare minimum because that's where the consumption of vCPU and memory are most often the first two limiting factors. Do this before adding vCPU to the VM to see if application performance improves. I can recommend a free tool from VMware called the VMware OS Optimization Tool (OSOT), which is designed to help with guest OS optimization.

You can deactivate services, registry keys, and many other options while still being able to revert to default. OSOT uses templates, VMware recommendations, and best practices. In addition to desktop operating systems, such as Windows 7x–10x, the tool also supports server operating systems such WS2008x–WS2019.

You can copy existing templates, save them as your own, and personalize their content so you only keep the optimizations that you really need.

VMware OSOT allows creation of your own templates

VMware OSOT allows creation of your own templates

Other VM performance improvements

Other than the guest OS improvements, we can also improve the way each VM consumes the overall host resources, such as CPU and memory.

In fact, we can reduce to the maximum virtual hardware consumption configured (or attached) to each VM. Those steps might reduce the physical CPU consumption only a little, but when you apply those tips at scale, it saves a lot of CPU cycles.

VMs often waste their time with CD-ROMs being inserted. You want to prevent this, so you should disable the CD-ROM for each VM. To configure this, select VM > Actions > Edit Settings. Then expand the CD-ROM drive and clear the Connected and Connect At Power On checkboxes.

Clear the Connect At Power On Checkbox

Clear the Connect At Power On Checkbox

Check CPU overload and CPU limit

If your ESXi has a CPU overload, you can connect via an SSH client such as Putty and initiate an esxtop command to verify the %READY field. This gives you an idea of how long (in percentage) the VM was ready but could not be scheduled to run on a physical CPU. This value should stay under 5% under normal workloads.

Make sure the VM is not configured with a CPU limit that is a limiting factor. The CPU limit can be set on the per-VM level or at the resource pool level (if used).

If there is no limit configuration on the VM, then the load is too high and the only thing you can do is either increase the physical CPU number on the host (if possible) or reduce the number of virtual CPUs within each VM. Finally, you can move some VMs to other hosts to free resources.

Memory overcommit

We can also check whether ESXi is struggling from a memory standpoint and reclaiming some memory from VMs. This is often the case when the physical memory of the host is fully utilized, and you have actually configured more memory for your VMs (overcommit).

Again, we can use the esxtop command here and check the value. On the main screen, type m for memory, then type f for fields and select the letter J (memory ballooning statistics). Then look at the MCTLSZ value, which shows the amount of guest physical memory that was reclaimed (if any) by the balloon driver. If you see only values of zero, as in our case, then your ESXi has still enough physical memory, and this memory reclaiming technique has not been used (yet).

Check the MCTLSZ value for memory ballooning statistics

Check the MCTLSZ value for memory ballooning statistics

Slow storage

While the newest virtual infrastructure deployments get the best of the best, all-flash storage, an old infrastructure might still be using spinning storage. "Spinning rusts," as we say. Spinning media often caused poor performance in virtualization infrastructures. Volumes created on top of spinning medias were slow, unreliable, and take a long time to rebuild if any of the underlying hard drives failed. You can easily test it if you migrate your VMs to different, more performant storage, attached to the same host.

You can also reduce the number of VMs per LUN if you're not using VMware vSAN, which is not relevant for this situation.

If you identify a storming-related problem, you can also verify that:

  • Your host bus adapters (HBAs) are on VMware HCL and are certified for ESXi.
  • Make sure that you have the latest BIOS on your physical hosts.
  • Check to make sure the firmware of your HBAs is up to date.

Final Words

Troubleshooting a performance problem is a job in and of itself. In this post, we gave you a few paths to explore, but we have just scratched the surface.

One recommendation here is to isolate the problems and see whether it is the VM itself or the physical ESXi first. Then you can investigate further to see whether it is the shared storage, network, physical CPU, or a physical RAM problem.

Subscribe to 4sysops newsletter!

Often there is only one limiting factor that causes problems, but there are cases where two or more limiting factors might be involved. Good luck with your troubleshooting.

  1. Igricheff 3 years ago

    No recommandation for network settings?

  2. Andy A 1 year ago

    Sometimes unexpected changes can have beneficial effects.

    I run a Server 2019 VM on my daily driver Win10 laptop, which allows me to use WSUS for my home domain by collecting updates where data is cheaper.

    I have been progressively increasing the VM’s RAM allocation to get better performance, but adding a third virtual processor improved things enormously, to the point where I could reduce its RAM by half a gigabyte.

    That might not seem a lot, but don’t forget that the VM is competing with the “normal” tasks on that laptop.

    I have to conclude that the extra “processor” allows the VM to shift its workload more easily, avoiding having to queue things up in memory awaiting the resource.

    The laptop does not seem to suffer much from allocating more CPU time to the VM.

Leave a reply

Your email address will not be published.


© 4sysops 2006 - 2023


Please ask IT administration questions in the forums. Any other messages are welcome.


Log in with your credentials


Forgot your details?

Create Account