Latest posts by Paul Schnackenburg (see all)
- Project Honolulu - A new way to manage Windows Server - Wed, Nov 22 2017
- Use Azure Managed Service Identity (MSI) to store passwords in your code securely - Thu, Nov 9 2017
- Azure Data Lake overview - Fri, Sep 22 2017
Seasoned sys admins have a good idea of how to use Task Manager and Performance Monitor to understand what’s going on when there’s a server performance issue. Those skills transfer well to the virtual world but there are some gotchas that we’ll cover in this article.
Task Manager in VMs lies ^
The first rule is don’t ever measure performance of a VM from within a VM. Most sys admins first reaction to performance complaints will be to have a look in Task Manager. Unfortunately that doesn’t work in a VM because it can only see its little keyhole view of the world. Here’s a simple test to understand this; run one VM on a host, run an application (such as Prime95) that maxes out the virtual CPU(s). Task Manager will report 100% CPU but if you now start another VM and run the same application in that VM, Task Manager will still report 100% but it’s actually running at half the speed compared to earlier. This example disregards additional processors in the host but clearly demonstrates: Task Manager in VMs lies.
Create baselines ^
The second rule for performance monitoring, whether physical or virtual is to create baselines when things are humming along. Unless you’ve got a baseline when performance was good and users happy you have nothing to compare with when issues arise. Learn how to create Data Collector Sets; log for a few days and archive the results.
Use Hyper-V specific counters ^
When using Performance Monitor in the host, be aware that it might lie as well, don’t use normal counters as they often don’t understand VMs, use Hyper-V specific counters. For processor performance use the Hyper-V Hypervisor Logical Processor\% Total Run Time counter; this will let you monitor physical processors in the host. For VMs use Hyper-V Hypervisor Virtual Processor\% Guest Run Time which lets you monitor virtual CPUs for each running VM (or as a total). As a rule of thumb, less than 75% for this counter (total for all VMs) is healthy, over 75% is a warning sign and more than 85% should definitely be investigated.
Committed Memory ^
In today’s Hyper-V world, B DM (Before Dynamic Memory), you should monitor \Memory\Available Mbytes on the host, healthy is more than 10% of RAM free, at less than 10% it’s a warning than something is going on and if it’s less than 100 MB it needs to be looked at. Old hands will know that “High pages per second” means a low memory condition but there are cases where this isn’t true (due to memory mapping of files), for more information see here. Use the \Memory\Committed Bytes counter to see how much memory each VM is using when planning your VM memory allocation.
Average Pressure ^
In the After Dynamic Memory (with SP1) world watch the \Hyper-V Dynamic Memory Balancer\Average Pressure counter, healthy is less than 80; a value between 80 and 100 deserves attention whilst over 100 indicates a critical condition.
Memory buffer ^
In the physical world as sys admins we always like to have a bit extra up our sleeve. If you were designing a server that you knew would use about 1.5 GB of memory you’d install 2 GB just to be on the safe side. In the VM world this is adjusted with the buffer, the default is 20%. For workloads that need a lot of file cache increase this value. This screen is also where you can assign a higher or lower weight to each VM which will allow a VM to have a higher (or lower) priority for memory as the host starts running low on available memory.
Disk latency ^
Monitoring disk is done via the \LogicalDisk(*)\Average Disk Sec\Read or Write which indicates your disk latency. Healthy is less than 10ms (0.010), if it goes up to 15ms or above (0.015) it’s time to check it out, at 25ms or above (0.025) the situation is critical.
Network monitoring ^
For network monitoring use the counter \Network Interface (*)\OutputQueue Length, less than 1 on average is healthy, warning is when it’s above 1 on average, critical is when it’s 2 or more on average.
In this article series we’ve looked at tips for how to design a Hyper-V environment for best performance, some tricks and gotchas to look out for and finally we covered how to monitor performance in a virtualized world.
Here are a list of websites and whitepapers that you can use to delve deeper into this fascinating topic.
Virtual Reality Check compares performance of hypervisors.
Download a performance tuning whitepaper for both the physical and virtual world from Microsoft for Windows Server 2008 R2 here.
The Performance Analysis of Logs is a PowerShell script that can help you out in deciding which performance counters to collect and how to analyse them.
For disk performance tasks have a look at Iometer
Detailed technical information on how to measure performance in Hyper-V