Networkworld recently published an interesting article entitled “7 side effects of sloppy virtualization”. This title seems to indicate that the problems server virtualization might cause are solvable by being well prepared. Nevertheless, all seven arguments discussed can be considered disadvantages of server virtualization. Because I am seriously thinking of virtualizing all of our servers, I read the article with interest. So far, we have only four virtual servers with about fifteen virtual machines, but we have already encountered some of the problems mentioned in the article.
- Pip install Boto3 - Thu, Mar 24 2022
- Install Boto3 (AWS SDK for Python) in Visual Studio Code (VS Code) on Windows - Wed, Feb 23 2022
- Automatically mount an NVMe EBS volume in an EC2 Linux instance using fstab - Mon, Feb 21 2022
I will discuss all seven downsides from my own perspective and share some of the experiences we have had with server virtualization.
1. Magnified physical failures
Imagine you have ten important servers running on one physical host and its RAID controller runs amok, wiping out all of your hard disks. Don’t say that this is not very likely, as we have already had two or three incidents from malfunctioning RAID controllers from well-known brands.
There are several ways to compensate for this downside. One is clustering, which certainly entails extra efforts. Another answer is to backup the virtual machines with a CDP (Continuous Data Protection) solution. If your physical server goes down, it is possible to restore all VMs quickly to another host. This solution implies that you have enough capacity left on another host. Thus, if your virtual infrastructure is well planned, physical failures may be less problematic. However, this means that you have to invest in redundant hardware, which more or less eliminates one of the alleged advantages of server virtualization.
2. Degraded performance
There is no doubt that virtualization requires extra hardware resources. The problem is that it is almost impossible to estimate in advance how many extra resources will be needed. I know that there are capacity planning guides and tools but from my experience every piece of software behaves differently in a virtualized environment. We have applications that are quite modest as long as they run on a physical server, but when they were virtualized their resource requirement multiplied.
You can’t do much if you have such applications. In our case, we had no choice but leave them on physical servers. Hence, the only solution to this problem is to thoroughly test each application with the virtualization solution of your choice.
3. New skills
It sounds so easy - install a virtualization solution and then just deploy your servers as you are used to. Not really! Many things are different in a virtual environment. I will give you just one example. When we installed our first server virtualization solution, I instructed our administrators to test some of their servers in the virtual environment. After a week or so, an administrator told me that he could not test his server because there was no more RAM available on the host. I was quite surprised, as this server has enough capacity for 10 VMs.
When I logged on, only 3 VMs were actually running. What happened? Some of his fellow administrators had assigned the same amount of RAM to the virtual servers as their physical servers had required. It took me quite some time to convince them to change their working habits. When you buy a new physical server, it is common practice to equip this server with as much memory as your budget allows. This makes sense, as it takes time to order new memory modules and add them to the server. Even if you do not require it now, you will most likely require more RAM very soon.
Of course, this situation is different in a virtual environment. I assign blame to myself, as we should have discussed things in advance. I should have told the administrators that they first need to figure out how much RAM their servers really need using a performance monitoring tool. If their server requires more RAM later, it is not a big deal to assign more. I chose this simple example because it demonstrates that you have to do some rethinking when you work in a virtual environment. The fact that several administrators share one physical server causes problems that didn’t previously exist. Of course, it is also necessary to acquire many new technical skills.
4. Complex root cause analysis
Virtualizing a server certainly implies big changes to the whole system. A new layer of complexity is added and can cause new problems. However, the main difficulty is that if something doesn’t work as it is supposed to, it can require considerable extra efforts to find the cause of the problem. I have another example for this downside of server virtualization.
We installed a SUSE Linux server under Virtual Server 2005. Everything worked fine at first. But, then the admin reported that his SSH sessions often got disconnected. We had another Linux machine running on the same host which didn’t show this behavior, so we thought it must be a configuration problem on the Linux system. However, this skilled Linux administrator wasn’t able to find the problem’s cause. I then had the idea to move this virtual machine to another Virtual Server host - and the problem was gone. So did Virtual Server or Linux cause the problem? Well, I can’t tell you. We never figured it out.
5. New management tools
Virtualization also has advantages, such as easier migration, cloning or snapshots. However, you can only take advantage of these new capabilities if you have the proper tools. Often, the tools that come with a virtualization solution are not enough, only supporting basic management tasks. This means that you need additional utilities, which cost both money and time. I am not only talking about such tools as VMware Virtual Center or Microsoft Virtual Machine Manager (VMM).
Another important field is backup, or more precisely, disaster recovery. Of course, you can use your current backup software to secure your virtualized servers. However, one of the advantages of server virtualization is that disaster recovery becomes much easier and faster provided you have a backup solution that is able to perform live backups of the virtual machines and not just of the virtualized servers running in these VMs.
The problem lies in that there are no real standards when it comes to virtual server management. But, there are standards for server management in general. For example, there are many backup tools that allow you to secure your Windows, UNIX and Mac machines, but it is difficult to find a disaster recovery solution that supports all the various virtualization solutions out there. All in all, this means that your zoo of management tools will grow, meaning more work for you.
6. Virtual machine sprawl
Even though virtual server management can get quite complex, installing a new virtual machine is a piece of cake. You need a new server? Just clone your master image to a new VM and you are done within a few seconds. The problem is that the number of servers might grow faster than the number of admins who are supposed to manage them. It is good that even virtual servers have physical limits. As soon as you reach the limit of your virtual capacity, the virtual machine sprawl will naturally stop.
The number of servers in my department has grown significantly since we started working with server virtualization. As a matter of fact, quite a few of them exist only because it is so easy to create them in a virtual environment. Thus, you have to be very careful that you don’t waste the resources of virtual server hosts with unneeded virtual machines.
7. Virtual habits
I am not sure if I understand this argument in the Networkworld article:
Once IT organizations start to use virtualization, they can't stop themselves, Coyle said.
Coyle is the Gartner analyst who is the source of the insights in the Networkworld article. He makes it sound like virtualization is a kind of addictive drug. Perhaps he meant that organizations that discovered the values of virtualization are prone to introducing virtualization technology in areas where it is better they stay physical. I also see this danger, and it is difficult to give advice on this subject. Sure, there are some general rules, such as don’t virtualize servers that have unpredictable resource requirements. However, I think that virtualization often causes problems in unforeseeable situations. Coyle gives the only possible advice here: “test, test, test.”
What are your experiences with server virtualization? Do you think that the downsides outweigh the benefits?
Articles in this series:
Subscribe to 4sysops newsletter!
- Seven disadvantages of server virtualization
- Does server virtualization reduce costs? Part II - Power savings
- Does server virtualization reduce costs? Part I - Hardware expenses
- Does server virtualization reduce costs? Part III - Software and payroll costs
I think virtualization is a great way to deal with applications that require their own machine, and does not run well with others on the same OS.
Yet, everything that i consider critical (Domain Controllers, Exchange, File & Print Servers, DMZ Hosts, etc.) is located on dedicated hardware.
Why? Virtualizating Exchange and Fileservers does not have any advantages (and in the case of Ex07 is even unsupported). Domain Controllers are far to critical to be used with another layer of potential goofyness.
Lukas, I had the same opinion a while back. Meanwhile I think that it makes sense to virtualize especially critical machines. The point is that you can reduce the failure risk because of the better disaster recovery options and because you can move critical server to another hardware easily if problems occur. And I think Exchange is supported on Hyper-V.
In the company that I work, we are just beginning to use VM-Ware, and I am just now, setting up a new serverpark for a customer of ours. So far VM-Ware impresses me. It seems solid, and quite intuitive. I concur in our statement about the complexity. Everything gets an extra layer of complexity. I needed to isolate the networks, and has done this with VLAN’s, which was a steep latter to crawl.
We have made one of our DC’s virtual, and keeps on physical. I find this configuration good, since VM-Ware gives High Availability, if one of the ESX servers goes down. And the physical DC is located out of VM-Ware to ensure that if VM-Ware should fail, there is a fail over.
We choose VM-Ware because I think that Microsoft’s solution needs to mature, as always 🙂
To expand on Michael’s comments to Lukas, there is some benifits to a “virtalize everything” approach. For instance ESX + VMotion gives you the capability of high availability without the tedious and combersome clustering approach. A standard backup approach is another.
Lou, ESX + VMotion certainly greatly improve availability. In my view, it is just too expensive. I think Hyper-V plus VMM 2008 is a better choice. You might have to wait a few more seconds until a VM has been transferred to another host, but this is not really a problem.
Jacob, I wished I could choose VMware, too. It is just too expensive. It is true that usually only the second version of Microsoft is mature enough for prime time. But maybe one can perceive Hyper-V as the successor of Virtual Server. In our tests, Hyper-V made a good impression. Also, I don’t want to start now with ESX and then move to Hyper-V in one or two years. Changing a virtualization solution can be quite time consuming and expensive.
Michael,
First let me respond to your response to Jacob. You said changing virtualization is time consuming and expensive, but this is not really true. You can do a V2V conversion rather quickly. So unless you are talking about the money you are tossing away on the VMWare license, then you really don’t loose much time or money.
But getting back to your article, let me respond point to point with your responses to the Networkworld article.
On point 1, you said ‘you have to invest in redundant hardware, which more or less eliminates one of the alleged advantages of server virtualization’
I disagree. If you consolidate 25 servers to one server, with a second one for clustering/high availability, you have still gone from 25 servers to 2 servers. You’ve just saved a gazillion jigawatts in power and reduced lifecycle management costs and spare part costs.
On point 2, you said ‘The problem is that it is almost impossible to estimate in advance how many extra resources will be needed. I know that there are capacity planning guides and tools but from my experience every piece of software behaves differently in a virtualized environment. We have applications that are quite modest as long as they run on a physical server, but when they were virtualzed their resource requirement multiplied.’
In my experience, virtualization IMPROVES performance. Why? You go from a 800 mhz dog to running on a 3ghz racehorse, of course it’s faster in a virtual environment! It reboots in 10 seconds instead of 10 minutes. And as far as estimating tools go, I have found that MAP 3.1 is extremely accurate because it literally captures performance metrics and estimates (accurately) what a consolidated server’s workload would look like.
On point 3, this is not a disadvantage of server virtualization, it’s a management issue for not training people before throwing technology at them.
On point 4 this one example is not enough for me. Especially because you never found the root cause, you can’t really say for certain that virtualization even contributed to the problem. I’ll grant you that it is perhaps an additional layer to the troubleshooting matrix, but the SSH example is a poor one because most likely this was a linux issue, not a virtualization issue.
On point 5, you seem to cite more advantages of virtualization than the one disadvantage of learning knew tools. I think everyone in IT has accepted the fact that with new technology comes new tools, but I see this as a more pessimism than a negative about virtualization. I happen to LIKE learning the new tools and find they make my job more efficient.
Point 6 – virtual server sprawl is an aspect of poor execution by IT administrators, not a disadvantage of virtualization. It’s like saying it is the car’s fault for getting the speeding ticket. Just about anything can be poorly implemented, but that isn’t the technology’s fault. Operator error =)
Point 7 – again, it is the fault of IT administrators not virtualization technology itself for pushing virtualizaztion. If you’ve hired an administrator who puts Exchange in a virtual server, then its his fault, not VMWare of Hyper-V!
I find myself defending Virtualization a lot these days from folks who in general resist change. The new reality is that most things will be virtualized because of reduced hardware and energy costs. Let’s get educated about how to implement it properly, and not be so quick to blame virtualization. Michael, all of your points would be valid if the title was perhaps ‘disadvantages of poorly trained IT staff who attempt virtualization for the first time’ or ‘mistakes to avoid with virtualization.’
Joe,
“If you consolidate 25 servers to one server, with a second one for clustering/high availability, you have still gone from 25 servers to 2 servers.”
The point here is that your two servers have to be much more powerful than each of the 25 servers. High end servers tend to be disproportionately more expensive than smaller ones. One reason are the memory modules. A 4GB module is more expensive than 2 2GB modules. I do think that you can save resources with virtualization. I just wanted to say that this is often exaggerated. Besides, 25 servers still have more redundancy than 2.
“In my experience, virtualization IMPROVES performance.”
It is possible that in some cases you experience better performance. However, in general this is not true. The virtualization software simply needs computation power.
“On point 3, this is not a disadvantage of server virtualization, it’s a management issue for not training people before throwing technology at them.”
This is not a management issue, this is a cost issue. Training people costs money. Thus it is a disadvantage of server virtualization which is often neglected.
“I think everyone in IT has accepted the fact that with new technology comes new tools, but I see this as a more pessimism than a negative about virtualization. I happen to LIKE learning the new tools and find they make my job more efficient.”
Joe, I like to learn new things, too. I think your perspective is typical for an admin. Try to imagine being a controller for some minutes, and you will see everything with different eyes.
“Point 6 – virtual server sprawl is an aspect of poor execution by IT administrators, not a disadvantage of virtualization.”
Agreed!
“Point 7 – again, it is the fault of IT administrators not virtualization technology itself for pushing virtualizaztion. If you’ve hired an administrator who puts Exchange in a virtual server, then its his fault, not VMWare of Hyper-V!”
You know what? I plan to run Exchange on a Hyper-V machine ;-). I think the point here is just that a new complexity layer makes mistakes more likely and that means it would cost extra money.
“I find myself defending Virtualization a lot these days from folks who in general resist change.”
You won’t believe it, but I often find myself in the same role. I am a big fan of server virtualization. The advantages outweigh the disadvantages by far. However, that doesn’t mean that there are no disadvantages of server virtualization.
Great responses. I think we are very much on the same page. Lets be clear on the savings of managing 2 servers over managing 25 servers. I went to the VMWare site filled out their TCO calculator, and it said for 25 servers with 2CPU each, I could expect a savings of $450,000 dollars over three years. So I think the point is, that by consolidating down to 2 servers, it is well worth the expense of those high-end servers because over the long run (in this case 3 years) you will save much more than your investment in those two costly servers. Right?
http://www.vmware.com/products/vi/calculator.html?
Even for a total Virtualization newbie and IT-admin-hobbyist, this article/discussion is interesting. Virtualization seems to be a really good idea but when you only read about it from the producer of the software it almost sounds too good to be true.
It’s nice to have a more nuanced view.
It is critical to know the disadvantages of virtualization before you bet your life on it – i.e. before you decide that since everybody does it, you too must do it. Otherwise many unpleasant surprises happen and usually in the most critical moment.
Everything has its advantages and disadvantages. We always wanted the advantage part, but we must remember even though how dim disadvantages would ever occur, we always have a fall-back plan. Like for instance your 25 server rolled into 2, dont just throw away your 25 servers, keep them in a safe place, youll never know when you would need them. 🙂 And who would throw away 25 servers anyway? 🙂
Good points. In my opinion virtualization is great for development and QA where you need to test in many environments, rather than a whole DEV/QA lab you’ve got just your laptop. Its also extremely useful for legacy systems. I use it almost daily for all of the above.
However I suspect that in many cases people will be quite disappointed when pushing it into up to date production environments. If your hardware is well utilitized, virtualization for the most part can only make things worse. I would never put a heavily loaded server in a VM.
i thank u a lot for publishing this..
this really helped to know the actual cons of virt
Virtualization Deployment requires careful capacity planning.It is a fairly mature technolgy.
I do not agree in jamming 25 servers into 2 Physical Servers.
Convey the following to the business.
Review upfront savings and operational aspects, recognizing that goals vary among implementations and technology domains. For example, there are no consolidation savings for desktop virtualization, but there may be for storage and servers.
Identify what not to virtualize. Not every desktop, server or storage array should be virtualized. Understand the goals of the implementation. Some workloads are less virtualization-ready than others.
Do not focus only on virtualized workload management. Identify those workloads that should remain physical, and manage them accordingly
Hi Michael, Could you give me some reasons not to virtualize SQL 2010 Windows 2008 server? Our management team wants to virtualize our SQL cluster enviroment and I do not think it is a good ideal but need a reason why this is not good. Currently we have two Dell PowerEdge R900 64bit Windows 2003 R2 servers clustered with 32 GB of RAM and 4 quad core procs and I was looking to add two more clustered servers with Windows 2008 SQL 2010.
James,
VMware maxes out at 8 vCPUs per VM, Hyper-V at 4.
James, why do you think it is not a good idea? Why don’t use this as a reason? 😉 I think it depends very much on the application you run on SQL server. Applications that need maximum performance better run on physical servers.
Hai,
In our company we have more than 30 servers, we are planing to virtualisation.I would like to known about disadvatages in virtualisation and what are the main things we have to take care. Could you please help me.
My thoughts for using VM on Production Machine:
1.. There will be processor overhead using VM
2. More memory utilization
3. All environments of Dev/Test/Production on production server has below demerits as well:
a. If the server is gone down, none of the ambience will be available.
b. For any patch upgrade at OS, all environments need to go through maintenance same time.
c. Multiple VM will need to share the disk drive and failure of main OS will affect all the VM environment.
4. The Admin Server for the Administration of your virtual machine needs to installed on a separate machine. You can’t install it as a VM on the production VM server. It requires external server .
5. Degraded Performance: Virtual machine is not that efficient as a real one when accessing the hardware.
6. When multiple virtual machines are simultaneously running on a host computer, each virtual machine may introduce an unstable performance, which depends on the workload on the system by other running virtual machines.
7. Potential Downtime: A VM coming online from a host crash will need to wait for their physical infrastructure to boot, plus the time it takes for your virtual machines to boot. The additional downtime may not be outrageous, but it can add up.