When a Test case passes, is it "successful" ?
By John Morrison on Aug 28, 2007
One thing thats interesting to observe when interacting with folks on the subject of testing is the use of the words - "successful" and "unsuccessful" when talking about test cases that have been executed on a particular product / feature.
Generally, people tend to associate the term "successful" with a test case that has "passed' without encountering any bug / error during execution and "unsuccessful" with a test case that "fails" due to a bug / error during execution.
From a Quality / Testing perspective, the above reasoning sounds counterintuitive and contrary to what we should really be saying which is ... a test case that fails is in reality "successful" and a test case that passes is actually "unsuccessful".
Lets imagine this scenario - my car leaks oil, belches dark fumes, rattles and makes enough noise to wake up the dead. Sensing that something could be amiss, I decide to take the vehicle to a nearby garage to check for problems and fix them. The friendly mechanic runs a set of "tests" on the automobile. After a while (and after reducing my net worth by a small fortune), the skilled tester, oops mechanic declares that all his "tests" passed and did not identify any problems with my car. Can he now claim that the "tests" were "successful" because they all "passed" ... talk of a 100% pass rate ?
Interesting point though .. it boils down to what you think is the purpose of testing ? Is it to prove that a program is "error-free" ? Me thinks .... ok, will save that as fodder for the upcoming posts !
| Slashdot It! | Submit to del.icio.us