PEST 4.5: Workshop on Visualization

Apparently, both Kristjan Uba and me were toying with the idea of visualization and wondering how to use it communicate different aspects of testing and its results. Therefore, we quickly came to the agreement that the topic of the next peer conference should be visualization. We also agreed that this should be a practical workshop where something could be tried out instead of coming with a prepared presentation. The reason behind it was that well… I have done little of it and do it selectively; pretty much the same applies to Kristjan.

I hosted the event at my office where we gathered on the morning of April 26. Participants at PEST 4.5:

  • Kristjan Uba
  • Oliver Vilson
  • Rudolf Elbrecht
  • Kadri-Annagret Petersen (newcomer!)
  • me

We had some last minute cancellations as well.

Kristjan did the necessary introductions and explained the four tasks he wanted solutions for in visual form:

  1. Feature/area vs bugs/severity (how to show you have major and minor bugs in different areas of the product/system under test)
  2. Test coverage in different areas throughout iterations
  3. New features vs risks/bugs
  4. Status of the project/product overall

For bugs, he specified the following kinds:

  • “obviously major” bugs
  • important but can wait until next release to be fixed
  • “stupid bugs” that happen once every full moon
  • a lot of small things

We also had an imaginary project: electric bicycle for Dakar rally with all sorts of awesome features (just to prime our brains and have something tangible to start with).

Everybody picked a topic, grabbed the coloring pens and papers and got to work. Afterwards everyone explained what they did, then questions were asked, comments made. You know, the regular peer conference stuff. Since the group was so small, we didn’t use the cards to moderate discussions.

I picked the first task. Here’s how I went about it.

Attempt 1

I imagined boxes. Structure. Here’s the result:

Helena Jeret-Mäe_1


There were four areas of importance to my electric bicycle, so I each of the large quadrants is an area. Each of them is divided in 4 by severity of the bugs. There’s color coding: each severity has a corresponding color and my idea was that visualizing the clusters of bugs would be helped by colors. The letters are just placeholders: in real life there could by bug titles as hyperlinks or some kind of tags or other useful names.

For whatever reason, it seemed like it makes sense to have the critical bugs in the centre to make sure their clustering would catch everyone’s eye. But then again… what about the clustering of minor issues? This wouldn’t be as clear… which is why I changed tactics in the second attempt.

Attempt 2

I still went for a central cluster but thought of it more in terms of a bull’s eye.

Helena Jeret-Mäe_2

You may throw arrows at it and fix the bugs you pierce through 😛

Anyway, now I got some concentric rings. The good thing is that it’s easier to compare the bugs and their amount between areas. But then again you’d need to zoom in to the critical area because there may be too many and it won’t show up well. Also, the minor bugs at the edges may get less attention which is why Oliver suggested that in this model you could reverse the order and cluster the small ones in the centre to get attention.

You may notice some bugs have special marking: a circle around them. I used the severities I use daily: critical, major, medium, minor. But then I remembered Kristjan’s “stupid bugs” (see above) and decided that it may be worth marking those special cases while keeping their severity as is.

Attempt 3

Then I felt there was too little color… What about making it stronger?

Helena Jeret-Mäe_3

I used the same model but changed the means. Now color was in the background but bugs were spots. When I was working on it, new ideas spawned (or dawned on me):

  • and area could be broken down into sub-areas if there are some that are worth paying attention to. So the dotted lines represents breakdowns of an area.
  • I accidentally drew some bugs where I drew the dotted line but then it dawned on me that oh, I may have to show bugs that are between areas. Of course, there is a difficult limitation: you can only show “integration” bugs between areas that are side by side in your model already. But as we found out during the workshop, such relationships are hard to show.
  • You notice some spots are bigger than others… which is where I started playing with a new idea which I realized in attempt 4.

During the discussion, Kristjan pointed out that one good thing in this model is that we can place the release criteria into it. That’s pretty much what he meant:

Helena Jeret-Mäe_3 - Copy

The release criterion is simple: if there is anything inside the orange circle, we won’t release. Stuff outside is something we can live with. A clear goal emerges: let’s clean up the bull’s eye and its neighbouring areas.

Attempt 4

Helena Jeret-Mäe_4

I guess I was getting more abstract now… There were jokes about who can spot a constellation 🙂

I did return to my original structure but modified it. This visualization of bugs in 4 areas is meant to experiment with the following ideas:

  • size of the dot as the indication of severity
  • different sizes color coded vs without color coding
  • 4 areas divided into sub-areas
  • each sub-area gets its bug count but the whole status can be deduced by looking at this area (and comparing it with others if needed).

Somehow I avoided numbers and counts of bugs in my models. Others did use numbers and I think it is alright. But my brain was wired to create pictures of bugs, their whereabouts and importance for quick assessment. The details would have to be discussed if there is a need. But if you just want to know what’s going on… you don’t details at first.

I had the annoying thought of coverage that become harder and harder to suppress. Because hey… it is important if I found critical showstoppers in the first few tests which means the rest of the area is not covered or if I found them after having covered some ground already. So I just drew two slices of cake (I may have been getting hungry).

Helena Jeret-Mäe_5

The slice on the left is a sketch of showing what kind of test techniques were used. Maybe I used a lot of scenarios but didn’t focus on the data as much.

The slice on the right does a bad job but it was meant to communicate a hazy idea: the thickness of the slice would indicate how much of something was covered. The problem is obvious: the size of the slice itself is fixed and I don’t know how big exactly the “whole slice” is anyway.

Afterwards I realized that my visualizations were quite abstract, more like proofs of concept. But that’s okay: I have been thinking that I should try something for a loooong time but still haven’t. So this workshop was great for me to actually get visualizing and thinking about the problems. I got some good ideas from other participants but I’ll comment on those once they’ve summarized their ideas (hopefully, it will be soon-ish).

Before we wrapped up, Kristjan showed how he has tried to do some visualization and add colors to help with understanding. I definitely want to steal some ideas from how he has worked with test results to report them. He said that well, it’s not much visualization but some of the things he had I hadn’t really thought about even. Awesome!

All in all, PEST 4.5 was short and sweet and very useful. Looking forward to the next one!


Positive Peer Pressure at PEST

A little alliteration doesn’t hurt, right?

The three day weekend which consisted of Rapid Software Testing Management course and the first peer conference of context-driven testers  in Estonia (in other words: Peers of Estonian Software Testing) was awesome. Literally. I went to work today and I was happy and energetic. Despite the fact that the days were intensive, I felt as if I had just come back from vacation. That’s what being around great people can do to you!

Also, everything that I experienced gave me the relevant kick-start to actually post to the blog I had created a while back. So I get to “blame” Oliver Vilson for inviting me to that event and convincing me that I have something to talk about, and James Bach for telling me he wants to hear more from me. The sarcastic part of me asks if this blog post is all about vanity then but it is not so. I’m sincere and I mean every word here.

What does it take to create the magic? Get great and thoughtful people together and have them share their experience with others about something that really matters to everyone involved. I truly hope the magic will last and I hope I can contribute to it as well.

The most important gains for me were

  • the people – I got to know a number of human beings I didn’t know anything about. And I feel lucky now that I do know them.
  • the revitalized self-belief – I had been bogged down by the routine of everyday work; I’d got tired and didn’t have the strength to rear my head and be hopeful. Don’t mock me if I say I feel like I’m breathing again
  • the stories and the ideas – I can’t wait to do great things with my team!
  • the community – if I get tired again or I feel overwhelmed, I know where to go. I am thankful for this.

Special thanks goes out to Rasmus Koorits and Indrek Kõnnussaar for making me rack my brain. It felt great and I realized I had also been losing some of my edge (similar to Rasmus’s experience story except it happened to me at work – so beware!). I also learned something about how the wheels in my head turn.

Eventually, one of the most important thoughts I took with me was about my background and its value. But that’ll be another blog post.