Friday, September 9, 2011

Questioning Simple Test

This is the 2nd post about the testing challenge that was given to my testing team during one of the internal testing workshop. The presenter was trying show us how often requirements are misunderstood even by technical people which often leads to making false assumptions.

Major of attendees suggested to turn 1) card #1 (card with 'A' vowel letter on one side) expecting even number on the other side and 2) card #3 (card with '4' even number on one side) expecting vowel on the other side. That was a trap. We can't say based on given requirements that if there is even number on one side there must vowel on the other side. It works only other way around: if there is vowel then there must be even number. It was pure false assumption.
According to presenter we should turn 1) card #1 (with 'A' vowel letter on one side) to verify if there is an even number on the other side as a positive test and 2) card#4 (with '7' odd number on one side) expecting consonant letter only on other side as negative test. This challenge however was not as easy at seemed because you can spot more false assumption in it.

First of all I think there is problem with statement: "Can you optimize testing and have only two cards that you might turn to test and verify that your system works correctly."
We can never ever be 100% sure that something works correctly when it comes to testing software application. And in this exercise according to presenter 2 simple tests were enough to prove that this system works! It's like with testing calculator summing function, when you put '2+2' and you get '4' which is correct but you can never be sure if this '4' was the result of summing function or maybe hardcoded result value for each operation.
Secondly let's focus on this requirement : "system returns playing cards with numbers or characters on both sides". English is not my mother language and maybe that is the problem but based on above statement I can't assume that for each card I will have letter on side and number on other side. I think that "letter or numbers on both sites" suggest that we can also have cards with only letters and only numbers on both sides.



  1. Are you saying that the requirements are ambiguous and it cannot be tested by just flipping two cards?

    Maybe the whole point was to make a tester realise the importance of negative test cases.

  2. Hi Anuj,

    The whole point of this exercise (according to presenter) was to show us how often (even technical) people are making false assumptions. The thing is the result suggested by presenter also had false assumption - number on one side and character on the other side (I can't assume this based on given requirement: "system returns playing cards with numbers or characters on both sides").

    Secondly I don't agree with the exercise suggesting that we will know if the system works correctly or not by simply checking 2 cards only.

    thanks for your comment