Pro-testing Against Bad Software Quality

ParkCalc done differently

The third testing dojo was organized on Tuesday, May 31st in Helsinki. It was also the last testing dojo before summer break. We will be back in August!

In the earlier testing dojos the format was really simple: Pairwise exploratory testing of selected application with short debriefings after each 5-minute session, a break in the middle and then more testing before wrap-up and free conversation. Based on the feedback received, some people thought this boring and I can easily see why since people only got to participate actively for a total of 10 minutes (5 minutes recording and 5 minutes testing) during the entire 3-hour dojo and the rest of the time they were only allowed to observe.

While I do like this format as well, I felt like it needs something more to keep the event interesting and engaging. So, this time we did things in a completely different manner.

Changes in the programme

I had a couple of ideas before the dojo and only decided on the way we were going to go this time mere 5 minutes before session start. Here's what we did in the third testing dojo:

Participants were divided into 3 groups of 3-4 people each. Then the application to be tested was introduced (the infamous Gerald R. Ford International Airport parking calculator aka ParkCalc). The mission was: "You have been told to test this application. Each group will have about 15 minutes for the testing so you know you will not have enough time to cover everything. Come up with ideas on how you feel you could best test the application within the given time".

After 15 minutes of discussion (and after emphasizing that people are free to collaborate between groups as well) the groups were told to select the 3 ideas they saw as the most important ones and write them on a piece of paper. Since I am a mean, mean person, there was a twist at this point: I collected the papers, shuffled them and then gave them back so that no group would be testing their own top-3 ideas.

At this point we had a break because I wanted to kind of silently encourage people to use all the time available for collaboration. Plus, I had told the participants the same thing Michael Bolton told us during Rapid Software Testing course I had attended a week before: it's okay to cheat. In this context: if you have limited time for preparations, it's okay to take advantage of any time that you have together - and I'm happy to say people did that, too!

After the break the groups had 5 minutes each for asking clarifying questions from the group that had prepared the ideas, in order to gain a common understanding of what was wanted and to be able to test those ideas effectively. Then we started the actual testing. We had a brief discussion on how the groups would like to perform the test execution and it was decided that one person from each group sits at the computer and the other members feed him/her ideas and suggestions on what to test next. At this point we were a little over halfway through the time available for the dojo and I was very happy that the time had been spent on continuous collaboration within and between the groups.

Each group had about 15 minutes for test execution, constantly telling other participants what it was they were doing and why. After each session the group that had prepared the ideas got to assess the work done and decide if their requirements had been met or not.

After the test execution rounds and debriefings it was time for sauna and free conversation.

Feedback

At the end of the dojo, I asked people to give feedback about the new format and the dojo in general and, as promised, here are the comments received (exactly as written):

- "A (really) short introduction on the purpose of exploratory testing in the beginning would be nice" (We had 4 new people participating in the dojo and I was so concentrated on refining the last details of the new format in my mind that I completely forgot this. Sorry, my bad!)
- "The group was diverse and brought / came with intresting ideas"
- "I don't know if there should have been more moderation for some of the discussions... I'm knew to this format"
- "3 topics in 15 mins means when you find something - need to move on"
- "Planning was good, many ideas"
- "+ Group planning"
- "+ Testing other group's plan"
- "+ Open discussion"
- "Idea connected to lightning talks: have someone give a presentation on some techniques and then try those"
- "Planning in the groups was nice"
- "Give little different focus for each group so that they can have more "fresh" look at the program"
- "Nice session, format worked quite well. Liked the idea of "management team""
- "A less obvious target would've been more interesting"
- "Sandwiches, even for vegetarians"
- "The welcome was a bit "cold", the person opening the door didn't even know about the dojo" (Again, my bad, I should have been at the door earlier myself. Sorry!)
- "Maybe a little more "intimate" time with the software would be good"
- "Bad design leads to people designing, not testing"
- "More focus on methods, easy to start chasing results"
- "Simple example helps keep focus"
- "Hands need to get more dirty"
- "Planning too long w/o access to spec or the software. Less time on planning & two rounds of testing (fb taken into account) would be nice"
- "Prioritizing bugs for their relevance would have been a fun exercise" (We did this in a previous dojo - need to incorporate it to this new format too!)
- "No focus on debriefing / reporting - should practice that"
- "Fixed ideas for third group gets a bit boring..."
- "More active session than previous times"
- "Deviding people in groups makes sense"
- "More communication & collaboration"
- "Better ideas generation etc..."
- "While a group is "performing" other groups may be bored / not follow (maybe it would make sense to keep them "actively" busy)"
- "Continue being proactive and organize those! Thats awesome!"
- "Discussion format(?) Result of the exercise? What was the conclusion? What did overlap / wasn't done? How do we are more efficient next time?"
- "More examples. Examples sell better than reason and to sell is what exp. testing needs"
- "Introduction?"
- "Less arguing, less battle of egos, more possibilities for creative thinking (is there really right or wrong here?)"
- "More "important" sw to test. Now people were thinking less of their testing, not yielding everything"
- "+ Groups nice change; less chaotic."
- "Too simple app?"
- ""Management" feedback idea needs improvement."
- "Good: Short missions = structure of dojo (ideas, prioritizing, questions etc.)"
- "Good: Discussion (with whole group & small team)"
- "Missing: Not everybody could do testing"
- "Perhaps reserve time for "best practices discussion""

It was great getting this much feedback. They will be taken into account when planning for the next testing dojo!