Tue 19 Feb 2008
Adrian Kingsley-Hughes from ZDnet ran a comprehensive performance test comparing Windows Vista SP1 with Windows XP SP2. He only tested file copy related tasks, though. Now, guess which OS is faster? Good guess! Anyway, it is interesting to see the exact difference because it allows you to estimate in what way a move to Vista might affect performance. However, in one sense all these performance tests are pointless.
What I like about this test is that the he used a relatively new PC for his benchmark tests: 3.4Ghz, 4GB, 2×250GB. I think it doesn’t make much sense to measure Vista’s performance on an outdated machine. In my opinion, a PC that is more than three years old is simply not “Vista compatible” even if you manage to get all device drivers.
These are the results of the test. “-“ signifies that Vista SP1 was slower than XP SP2, “+” that it was faster.
- Boot-up time: -13.3%
- Drive to drive copy - single file/ multiple files: +216.7%/-23.4%
- Add to compressed folder - single file/ multiple files: -9.9%/-19.7%
- Extract files from compressed folder - single file/ multiple files: -64.8%/75.4%
- Push files to a network share over a 1Gb Ethernet connection - single file/ multiple files: +7.1%/6.3%
- Pull files over a network share over a 1Gb Ethernet connection - single file/ multiple files: 0%/0%
- Push files to an external USB hard drive - single file/ multiple files: -6.3%/-29.5%
- Pull files from an external USB hard drive - single file/ multiple files: -36.4%/-36.4%
Quite impressive numbers! Don’t you think so? Drive-to-drive-copy of a single file is much faster with Vista SP1, single file copy to a network share is also a little faster. Although in most categories XP proved to be the better performing file shuffler.
You might wonder how this is possible. Copying a file from one place to another can’t be too difficult. After all, you just grab the file and shove it to its new destination. Why is Vista so slow with such an easy task?
It is interesting to note that Adrian Kingsley-Hughes doesn’t even try to explain his test outcome. He delivered the results the crowd wanted to hear:
Looking at the data there’s only one conclusion that can be drawn - Windows XP SP2 is faster than Windows Vista SP1. End of story.
End of story, mission accomplished?
First of all, he didn’t really run a complete performance test as he says. He just examined file copy processes. Of course there are myriads of other tasks an operating system has to accomplish. I don’t want to claim that Vista would do a better job in other performance disciplines, though. Most likely it would not. However, from this performance test alone you can’t conclude that Vista is slower than XP.
Second, his results are not really surprising. This is exactly what you would expect from a new operating system. The more complex an OS gets, the more resources it needs for every task. Some days ago, I linked to an article of Mark Russinovich where he outlined some of the considerations one has to take into account when it comes to file copy. In the section “Vista improvements to File Copy”, he gave some examples of problems Microsoft’s developers had to face. For instance, he touches some issues of the Windows Cache-Manager. Copy processes can waste a lot of memory and CPU resources. Since Vista generally has a higher demand of these resources, improvements were necessary here.
In the mean time, Adrian reacted to critics. It seems as if some also referred to Mark’s article. So Adrian ran another test to prove his point that Vista SP1 is always slower no matter from what angle you look at it. However, my objection to his test comes from a different dimension. In my view, it is rather pointless to compare the performance of a new operating system with one that was developed five years ago.
With every new line of code you add to an OS, you make it slower, too. Of course! So if Vista would really be faster than XP, I would suggest asking your money back. The point is that not only software is improving over time, but hardware also. Advances in operating systems are only possible because hardware is getting faster, too. So if you really want to know if XP or Vista is faster, you should compare your 6 year old PC running XP with Vista on a computer you bought a year ago. Which one do you think is faster? Good guess!
Update: Ed Bott gets completely different results in his performance test: “Vista SP1 was consistently as fast as or faster than XP SP2″. I am now thinking of getting my money back. 
Newsletter: 
Loading ...
[...] Vista SP1 vs. Windows XP SP2 file copy performance Michael talks about the results of a Vista vs XP test, and comments on it’s validity. (tags: XP Performance ) [...]
Excuse me Michael, but I think your trying too hard.
It’s ok to defend Vista, but you look like a Microsoft comercial.
And now, face it, Vista is slower, and will ever be. Two years from now, it will be bearable, but now? No thanks.
Indeed, while technically you may be right to say that it is not fair to compare a one year old OS with a more than 5 years old one, what people are interested in is sheer performance.
You write this :
“relatively new PC for his benchmark tests: 3.4Ghz, 4GB, 2×250GB.”
4GB of ram is by no mean “relatively new”, it is a high-end PC.
I’m consider myself happy when I go on people’s PCs and see that they have 1GB of ram and the point is that they simply won’t upgrade their PCs just to run Vista…
Also, the problem is that the added features simply do not explain the added overhead in performance which is sometimes massive…
The number and purpose of the new features simply do not justify the added overhead…
While I can be OK with boot time, why do we see so much difference in files copy performance ?
The problem is that customers are not going to buy the “It is OK for it to be slower, because it is brand new and have added features” excuse.
Granted, the same was true with 98 and XP (many users, especially home users, didn’t get through the Win2000 phase), but it was a whole different kernel and the gap in requirements wasn’t as massive as from XP to Vista.
While you are right that better hardware is required for better (I would prefer to say “newer”) software to run, it is something that software use and abuse with the hope of this to cover their occassional sloppy programming.
I think it is clear that today, the focus is not on performance but on features and looks, and who cares how bloated it is becoming : it is why people have the feeling they need to renew their PC to do barely the same thing as before.
If programmers had performance more in mind when writing programs, our PCs would run much faster today than they did in the last years.
Unfortunately, I am not feeling that my PC that I renew almost every year has been much faster than a few years ago in day-to-day tasks and I can’t help but be sad when I realize that the added performance from the new hardware i have been buying has been sucked away from me to make up for software bloat and occasionnal sloppy programming (I’m not only speaking of Vista here but almost any software vendor today).
My view of Vista is that it has wanted to do too much at once : new user-interface (very confusing and breaking years of user-experience and habits, ending up in huge training costs and productivity loss), new security features (while they are certainly good, some seem overkill over the loss of compatibility and it is why most admins think that XP is “secure enough”) and insane hardware requirements compared to what most people or companies are equipped with.
Not saying that things shouldn’t move, but Microsoft tried to do too much at once, resulting in a lot of rejection from its user (worst point for me over all I said is user-interface : it is why I am not planning to deploy it yet in my organization).
I hope Microsoft can get back to its sense with Windows 7… as this service pack doesn’t seem to solving much issues, people may lose any hope for it and simply skip it…
I am not as tech savvy as i would like to think but i will give it a shot. Windows XP requirements calls for 512mbRAM and Windows Vista calls for 1gbRAM. Running XP on a machine with <7x the memory needed and Vista on a machine with <3x the memory required, to me, seems to be a flawed test. Especially when the numbers show Vista faster at single, less memory draining procedures. Mike -edit if not :)- is right on the money when he says the test should be conducted on pc’s with the hardware available at the time of Introduction. The test is obviously skewed in favor of XP. I would drool all over XP as well, if i had almost 4gb of RAM. I will shortly be drooling all over Vista when i go x64 with 8gb or maybe try for 16gb.
When XP cames out 7 years ago, all the same BS was told (”it´s a resource hungry”, “the luna interface makes it slower”, “keep with Win 98 because it´s faster”, “blah blah”)… Remember, XP needed a 512mb machine to run relatively good, in a time when 128mb was the default… Come on ppl, move on. Vista is a great OS, much better than XP ever was, and it´s shares the same code Server 2008. It´s funny how Server 2008 never gets all this “critics”…
The problem is that once you set the hardware requirements too high, the companies won’t be upgrading because it would mean to renew 90% of their workstations, which they may or may not have credits to do so.
In organizations having little to do with IT as their business (let’s say a clothes factory), executives sees IT costs as a something they have to pay for employees to work, but they may not want to invest more for little benefits.
IT Managers have to have a great deal of good reasons to tell the executives they want to order new PCs in the organization (in most of the case, the rest of the PC like the CPU or Hard drive is too old as well to run Vista decently, so a simple memory upgrade won’t do).
Now, it is quite unfair to compare 98 with XP since they were different kernels types, but as I said, it is true that reactions were more or less same.
However, with XP, you got huge improvements over 9x : stability is several orders of magnitudes better, you could join it to domains, etc…
Vista has some extra-features (most of these can be obtained through third party program running on XP anyway), but not enough to justify the upgrades for most people.
Now, regarding Windows Server 2008, things are pretty different for a server OS than a client OS.
You rarely work on the server directly, unless it is a terminal server, so the user interface doesn’t bothers as much as on a workstation OS.
It is also easier to justify hardware upgrades on some servers than for lots of clients.
Same with Security : you obviously need even more security on the server than on the clients, so having to solve some either software or hardware incompatibilites is worth the security gain here.
As soon as companies will renew their computer fleets, they will either keep the preinstalled Vista or deploy their own once most of the fleet will meet hardware requirements with dignity, but before, it seems hard to justify.
From what I’ve read disabling the background search process will improve performance quite a bit.
[...] Willen jullie nog on-topic? Want dan heb ik een interessant (in het Engels) artikel eventueel. 4sysops - Vista SP1 vs. Windows XP SP2 file copy performance Lees verder voor de uitslagen. Dat komt door een bug(je) en is niet "standaard" in [...]
Victor, thanks. I was thinking that I was bashing MS too hard recently. It seems that is not the case then. And you’re certainly right, Vista is slower than XP. You know what? Windows 7 will be even slower. I absolutely count on that.
JC, I think 4GB is not really high-end anymore. Memory has become quite cheap lately. I don’t know what apps you are running, but I have absolutely no performance problems with Vista. I still have my old XP machine at home. When I bought it, it was absolutely high-end and I think it was only 2 years old when XP came out. This machine is definitely slower than my Vista laptop now which I bought short before Vista came out. All you need is enough RAM and, as Aaron rightly mentioned, to disable Windows Search.
Ronald, nothing to edit
Andy, I think we still have to wait if Server 2008 will get the same critics as Vista. But bashing a desktop OS brings more page views, so I think it won’t be so bad for Server 2008.
C’mon Micheal. Just re-visit what you have said here. Microsoft are asking (soon it will be forcing when the drop XP supprt) me to upgrade my organisations OS to Vista. In the process I will need to spends thousands buying new kit, re-training staff, and updating applications. For my efforts I will get slower performance, less stability, less compatibility, and a brand new interface that to 90% of users will seem unintuitive. And you are defending them?!
Think if they were selling a product in any other market - cars, TVs, holidays…. IT needs to join the rest of the sane business world and drop the notion that money doesn’t matter. Your comments re 4GB of RAM clearly illustrate that you speak for yourself only and not for any business or organisation of more than 1. How can you possibly expect any company to regard putting 4GB of RAM in every machine as normal?!
For my company, XP does everything I need it to - and seems to do it relatively well. Please give me a real business reason why I should upgrade. What will I get for my money? As we say - What is the ROI?
Ian, Vista runs fine on a 2GB machine. It is just that when we buy new PCs they now always have 4GB. Memory is quite cheap now. If you are sure that XP does everything you need, you certainly should not move to Vista. But did you really check out the new Vista features? Isn’t the extra security important to you? And what about those new deployment features?
I’d love someone to explain to me why my aunt’s Dell Dimension 520 has the BIOS modified specially by Dell to dis-allow the USB keyboard to work properly (a planned incompatibility if you like). This prevents Windows XP setup (and my Linux boot disc) from working and forces the user to stick with Vista… Which crashed explorer.exe about 20 times in the first day, fresh from the box without modification of the software. Hmmm. Apparently Dell did this under pressure from Microsoft. If the product is good enough, this kind of B.S. isn’t necessary. Go Figure.
The comparison is not fair. Vista runs Windows Defender by default but XP does not. Everyone knows that spyware and virus scanners slow down file access and copy operations somewhat. Also it is not clear if the test system used a multi-core CPU, it sounds like it was single core. The comparison would be better using a dual or quad core system, I would expect Vista to be better optimised for this.
Steve, performance comparisons of an old with a new operating system are never fair. Usually, there are plenty of new functions which slow down the new OS. Vista’s new security features are just one reason why it’s slower than XP.
Ben, what makes you believe that this was planned and what has Microsoft to do with it?
I read somewhere that someone had been forced to regress their BIOS version which solved the precise problem with the USB keyboards. Dell support couldn’t tell me why the keyboard wouldn’t work, even after a LONG time on the phone and email contact gave them a good chance to. Perhaps they couldn’t admit something. I’m not a paranoid conspiracy theorist, but this is the way Microsoft licensing works, it’s not exactly good capitalism with an open market, is it?