Comments on: Beta of MCSA/MCSE 2008 upgrade exams

By: Mohamed samir

Mohamed samir — Fri, 02 Nov 2007 19:39:46 +0000

Dear all,
can any one help me getting code to test these beta exams

By: Seth Janowiak

Seth Janowiak — Thu, 26 Jul 2007 16:06:32 +0000

Actually, I’ve filed very few bug reports. The most I ever submitted was four reports for IE4. It’s a relatively trivial process of filling in an on-line form or bug report interface in the application like in Mozilla. Obviously I’m testing the application. But my methodology is simple: “See if it works.”

If it’s a dramatic change, like Exchange 2003 from 5.5, it’s important to test the upgrade paths. Especially when products like Win2K and Exchange 5.5 are EOL.

I don’t intend to QA the products, and I don’t think that’s the intention of MS in the first place. Sure I’ll report a bug if it’s not doing what I want it to do, but I’m not going to apply a testing methodology, I’m simply evaluating.

Regarding your question about why beta test without writing bug reports, I think there is a difference of opinions. Microsoft offers up Beta software for evaluation purposes quite often. In fact looking at the Longhorn Beta 3 site, there is no mention of reporting bugs as a prerequisite. Of course you can argue that by not submitting enough bug reports as a beta tester, you are not pulling your weight, but I bet the vast majority of beta testers are “evaluating”.

By: Lukas Beeler

Lukas Beeler — Thu, 26 Jul 2007 11:27:07 +0000

Seth, you seem to forget that submitting bug reports is a lot of hard work (if done right), and not related to “staying ahead”.

Of course you can use beta software without writing any bug reports, but that’s not really the point or is it?

By: Seth Janowiak

Seth Janowiak — Wed, 25 Jul 2007 22:50:00 +0000

While I agree that beta testing is very helpful to the software company, the concept of “paying” the Beta testers has always been a thorny issue. I beta test software for two reasons: to stay ahead of the curve, and because I enjoy finding out how new software works. I’m not expecting to be compensated for that at all. In fact many organizations pay for the privilege of sampling software products before they are released (MS Select).

By: Michael

Michael — Sat, 21 Jul 2007 09:37:29 +0000

Yeah it would cover my costs too. But only if I get cash. This product is hopelessly overpriced.

By: Lukas Beeler

Lukas Beeler — Sat, 21 Jul 2007 05:26:08 +0000

A copy of WS2003 Enterprise retails at 5000 CHF. I have no idea how much you earn, but for me this would cover quite some time

By: Michael

Michael — Sat, 21 Jul 2007 04:44:03 +0000

Well, a free copy wouldn’t be enough pay for me to ruin my nerves with a Microsoft OS that is in its early beta phase. Don’t get me wrong. I am not a moralist. This strategy works fine for MS and other popular companies. It is not against the law, so why should MS change it? Many seem to enjoy testing software and reporting bugs. I only test beta software because I need it for my job. Microsoft is doing all this for money, and so do I. 215 CHF sounds like a fair price for taking part in such an exam, though. I just hope you passed.

By: Lukas Beeler

Lukas Beeler — Fri, 20 Jul 2007 20:14:17 +0000

Beta exams aren’t a bad deal:

* You can do the exam for free (Instead of paying 215 CHF/140 EUR)
* You’ll have to wait several months till you get your result
* The beta result counts the same as the final result

You can’t really eliminate the end users from the QA process – beta tests are a good way to bring some real world load on the software you’re trying to sell. I agree that some companies do this step prematurely to save on extended testing cost.

However there are problems that can’t be found by a normal testing process – it would be impossible to simulate all the usage scenarios, hardware combinations, etc. And i think that Microsoft should honor that a bit more.

In the Vista beta, they handed out free copies to everyone who was a registered beta testers (they early betas that were available on connnect) and submitted at least one bug. This was a step into the right direction, but didn’t go far enough.

Office Beta testers didn’t get anything. And licenses are free for Microsoft, however i see one problem that might arise when you’re offering free licenses for everyone that beta tested your product und submitted bugs: people that just want free stuff. So you’d have to weed these out, which probably isn’t easy and just adds another layer of work. Some thinking would be required here.

I have no idea how licensing is handled for Microsoft s TAP partners, who put beta products into supported production use.

By: Michael

Michael — Fri, 20 Jul 2007 19:54:21 +0000

I agree with you that beta testing can improve the quality of a product. This also applies to exams. However, I also think it is a method to let beta testers do part of the work that should be done by the company in the first place. That’s why I usually report bugs only when I beta test freeware. Some companies tend to abuse the curiosity of their customers. I wonder how many Vista beta testers helped to improve this product and got nothing for it in return.

By: Lukas Beeler

Lukas Beeler — Fri, 20 Jul 2007 18:29:10 +0000

Beta exams make a lot of sense, at least in my opinion.

The problem that Microsoft has is that their exams are somewhat disconnected from reality, because they only ask theoretical questions – they don’t directly test your job skill, just your skills to answer theoretical questions.

(Of course, the appropriate job skills make it somewhat easy to answer those questions)

Now a beta exam is the final QA step. Let’s assume a simple questions with four answers, all worded quite ambigously. The test makers decided that answer b is right, but 75% of the candidates answered c. Something is obviously not right with that question.

In this case, all these questions were mostly answered by people who hold an MCSE on Windows Server 2003, AND are scouring the internet on information about beta exams. That means the amount of “paper MCSEs” would be rather low.