WTA07 - when do we really start testing ?

Last Saturday I was participating in Weekend Testing Americas #7. This session realized me once again how little I know :).

The Challenge: Cross the River
Press the round blue button to begin. The goal is to get all eight people across the river. You must respect the following rules:

Only two people can be on the raft at one time.
Only the mother, father & officer can operate the raft.
The mother cannot be left with the sons without the father (or she’ll beat them).
The father cannot be left with the daughters without the mother (or he’ll beat them).
The thief cannot be left with anyone without the officer (or there will be even more beat-downs).

The Mission:
"I’m hiring staff for my IT department. I was told that this simple program will help me in finding the smartest candidates. Your mission: test the program and report how it suits my needs!"

Session from my perspective
I read the mission and downloaded the application. Then I started testing - I was trying to find any problems with this application as fast as I could. Meanwhile I was reading main chat. It was strange that no one was writing about the "findings" or "issues" related to the program. The discussion was mainly focused on the mission. Instead of pure clicking on the program, people were trying to:
- get as much details from the customer as possible
- challenge the mission
- understand what "smartness" we are looking for
- to understand how a piece of software can evaluate the smartness
- to verify if we are smart enough to evaluate others smartness
- and so on

Above questions reminded me, how many times I start testing something without asking for more details and without raising questions. I realized that testing hardly ever start form executing program.

The debate was very inspiring. One of the brillinat thought was from Adam Yuret:

Bugs are meaningless in this context. Like finding that the logo on one side of the sinking titanic is the wrong color.

If I could remember only one thing from the Weekend Testing session it would be from Justin Byers:

Avoid jumping into the software looking for bugs until you know what you’re really testing.

For whole chat session see chat transcript.

Participants: Michael Larsen, Justin Byers, Albert Gareev, Shmuel Gershon, Markus Deibel, Mohinder Khosla, Adam Yuret, Perze Ababa, Phil Kirkham, Aleksander Lipski

Thanks guys for another testing lesson.
Alek

Weekend Testing for the first time

I heard about Weekend Testing some time ago but I haven't had chance to participate in it until last weekend.

The testing challenge for WTA06 was prepared by Eusebiu Blindu. It was a game that was using a hand of cards, and based on the hand selected, an image was generated when the player clicked on the "Generate Image" button. The session was divided into 2 parts - testing and discussion.

There were more then 20 participants. Even though it was american chapter, we had people from England, Czech Republic and Poland too :). In the first part we could test separately or work in pairs. It was up to us how we would manage this challenge. I didn't have chance to work in pair so I started working alone. The beginning was hard because all the messages which came every second, were very distractive. But after a while I got used to it. I was picking a test idea and following it. In the case of finding anything interesting I shared it on main chat. In the case of lack of ideas I picked someones else's and started following it. It was amazing how quickly, that one hour flew by. I had never had chance to participate in such insightful testing session.

The second part was mainly a discussion. We were discussing what skill were essential in this task. We had chance to see someone else's point of view, to challenge our own conceptions and verify our own thoughts. I have learnt a lot by watching other people's different approach, as how they use different tools and skills for the same task. I could again realize how litte I know

One more time thanks to Eusebiu Blindu for the challange and Micheal Larsen for leading the session.

If you ever have 2 hours free on Saturday or Sunday try to "taste" Weekend Testing if you want to stay in shape as a tester.

Alek