I also whole-heartedly agree with his overall point that skill-based testing is much more valuable than the ability to mindlessly create and run hundreds of scripted tests. The only issue here is that, in order to make Holland’s point a bit more realistic, you really need to add a few layers to this hunt:Also each bug may only appear when certain other conditions are in specific states and when they are not in those states everything works fine (pairwise and combinatorial bugs). Like I can't use your comment system with Chrome ("Disqus seems to be taking longer than usual. Reload?" - just forever, reloading...) but I can with Firefox.When you include these factors in the equation, scripted tests can become an extremely valuable asset.
- The eggs aren’t always visible at first glance.
- Each hunt lasts a finite amount of time.
- After each hunt, someone else renovates the building and hides new Easter eggs.
The most apparent/predictable pairwise and combinatorial bugs can be caught with exloratory testing. Many may not be though.
Scripted testing is good to check specific settings for specific results. Doing a bunch of this programmatically is very useful (especially to catch unexpected bugs from minimal code updates - doing full exploratory testing of an entire application every time any code is updated would take a great deal of resources - and likely slow things down too). Doing some of it with a person looking for issues in specific test cases is wise.
Thinking this is all you have to do is very unwise. You need exploratory testing by a knowledgeable software tester (or if this isn't possible then exploratory testing by a user proxy - this is not perfect but is much better than nothing) if you care about the quality of your software.
Related: Which is Better, Orthogonal Array or Pairwise Software Testing? - Maximizing Software Tester Value by Letting Them Spend More Time Thinking - Cem Kaner: Testing Checklists = Good / Testing Scripts = Bad?
No comments:
Post a Comment