A Workshop on Testing for Programmers

An idea had been twirling around my brain for a while. Then it became more and more insistent on breaking into the realm of reality. In the context of continuously learning about testing, and in the context of recognizing the ongoing challenges of programmers and testers working together, I realized that I need to do more about it. Quick conversations over lunch or in the hallway about one or other aspect about testing just didn’t seem to be enough. There needed to be something that’s a full package, coherent,  concentrated and experiential. There needed to be something that would renew the foundation for future conversations between testers and programmers.

Mission: create a full-day workshop on testing that would be tailored and targeted at the programmers I work with.

I consider this workshop as one of the great outcomes of Transpection Tuesdays. Without the work I’ve done with Erik Brickarp and without his support and encouragement, it would’ve taken me much longer to muster up the confidence to create and carry out the workshop.

The keyword here is “tailored”. Fun and exercises aside, I focused on tying together the general introduction into testing, the way testing had been done by my team, and whatever else was relevant in the company’s context in relation to testing. I wanted to frame some of the specific problems in our company as testing problems and have a discussion about them.

Goals:

  • give a concentrated introduction to testing so that fragments of what programmers already knew were tied into a whole
  • introduce and explain testing problems to  programmers so that they could be understood and discussed better
  • immerse programmers in some testing problems so that they would see and experience them up close (followed by a discussion and explanations)
  • explain what testers do and tie it into their experiences of working with testers so that programmers can make sense of their experience and forge even better relationships with testers
  • explain the tester’s mindset and vocabulary so that further conversations between testers and programmers could be more productive

Why I’m Sharing This

I’m sharing this just because I believe from my experience that creating such a workshop can be useful and educational in many ways, and there are surely people out there who need just a bit of nudging to do something similar where they work.

Slides and some accompanying materials can be found here.

It doesn’t contain all handouts because well… what you think you need to provide to people you work with will really depend on the kind of workshop you create. I also didn’t include my notes for slides because they have quite a lot of context-specific stuff in them and it didn’t make sense to try to clean it up.

You can grab them and run but it would be so much more useful for you if you read the following as well 🙂 Then you will know WHY it is the way it is.

Planning: content and structure

Preliminary notes

One of my favourite ways of getting some mental work done is to go to a long lunch alone, take my notebook with me and just write my ideas down. I did quite a bit of work on the workshop in that manner, clarifying my initial ideas about the structure of the workshop, taking notes about what kind of exercises to use etc.

Mindmapping

  • I created a mindmap using Xmind to sketch the initial structure of the workshop. The mindmap I’m sharing with you isn’t perfect and it looks sketchy. I just used it as a starting point.
  • I mindmapped the big sections and also drafted ideas for subtopics to be covered in sections
  • I added labels to each chunk estimating the time I’d be spending on it. Then I summed them up and the result was the time I was going to spend on each section. And from there I could calculate the time I’d spend on the whole workshop.
  • Estimating the time spent helped me balance the sections and decide whether to include or exclude content. It also kept me in check and guarded against adding too much.

Slides and notes

  • Based on the mindmap structure, I started drafting my PowerPoint slides, taking notes about the content and gathering relevant references.
  • I kept track of references in a separate document to make sure I can hand that list to the attendees later.
  • Working with the mindmap and slides I did a lot of gauging of the flow and used internal monologue or tried explaining some concepts to myself. The purpose was to make sure the flow of ideas made sense and that each section built upon the previous one. In that process I made structural changes to the mindmap which provided me with a good backbone and overview of the workshop.
  • For each section, I added a separate title slide and a recap slide. The idea was that the day will be so packed with information that it’s better to add some points where I can concisely summarize what we just talked about. All those recaps would then go on one sheet of paper. If that was all they ever kept from the workshop… they would have the essential ideas captured.

Exercises

General idea behind beer

  • I wanted to have a “thin red line” that would run through the workshop. There would be so much ground covered in terms of concepts that I felt like falling back on a single constant thread would help pull us all back in.
  • I found that thin red line in the workshop by Louise Perold on test planning that I attended at Copenhagen Context this January. She used bottles of beer as the product in the workshop and it worked really well for a test planning exercise. Sure, it’s not software but… beer is fun, fairly well understood and doesn’t need much explanation. So I discussed this with Louise and she was fine with me picking up the idea and using it myself. Many thanks for that! 🙂

Kick-off exercise

  • Testing the beer was to be the thin red line and the kick-off exercise.  Each team of 3-4 people got two different bottles of beer. The only brief I would give to testing beer at 10am in the morning was to “test the beer and explore it” in 20 minutes. I didn’t give them the bottle opener as I wanted to see if any would ask for it. I gave them a sheet of paper and colorful markers to write down whatever they wanted about the beer they tested. I made observations and took notes about what they said in their interactions while they were testing the beer – I’d use these notes for debriefing and for tying the discussed topics with the rest of the workshop. For instance, if I’d ask how they knew the beer was OK, they could give me answers like “from my personal experience” and later when getting to oracles, we could discuss their initial test run in the light of heuristics and oracles (or anything else we covered during the workshop).

Follow-up exercises with beer

  • Testing mission – since very few asked why they were testing the beer and what was supposed to be accomplished, I used this as an example of how not to approach testing… and then asked them to figure out a few testing missions for beer that would frame the testing activities. I also brought in the project lifecycle to show how different things can be tested for over time.
  • Oracles – since it wasn’t that easy to find actual issues with the beer, it gave me a good segue into asking why couldn’t they spot some problems. And from there we could get into oracles and how they’re useful for testing. Later I used as another recap of the kick-off exercise when trying to come with some oracles for testing the beer and pointing out the ones they had actually used but didn’t realize it.
  • Test strategy – I gave them a brief, then asked to come up with a simple description of a strategy. I mostly meant this one as the “dip into this but don’t get too deep” kind of an exercise. The point was to teach about test strategy per se but to make them feel how difficult it may be to word it. Also, I gave them the whole HTSM on paper but limited what they had to use. It was probably a bit too much…. in terms of balancing time and materials. But I feel like they got the sense of complexity out of it at least…
  • Creating a script for opening the beer. This was a pretty fun one… The idea was to create an experience where you’re supposed to do something simple – just follow the instructions. But then describing how to open a beer in a specific manner isn’t THAT easy. So on the one hand, people could get creative (some drew up pictures to illustrate the steps) but then when executing the steps, I limited them in their interpretation. One of the coolest instances was when the script was followed precisely but the beer bottle ended up with its cap on not off  🙂 There was a very interesting contextual cue that a programmer decided to “misinterpret”…
  • Testability – when discussing testability, we returned to the beer trying figure out what would make it easier to test beer.

Other logistics

  • I divided programmers into 3 groups of 9 which meant I ran this workshop 3 times. This seemed like a manageable amount of people for me.
  • Since I knew them all well (I’ve worked with them for years), I tried to balance out the groups so that programmers from different teams could interact as well.
  • I tried to make the first group the most difficult for me in order to make sure I will find any issues in how I’ve set up the workshop and fix them before the next one. That one I got right – they were active, wanted to argue (not just with me… even more amongst themselves), wanted to discuss problems and potential solutions to them. I was able to figure out some issues in the order of sections which was a good thing to fix.
  • I had snacks ordered for the breaks and pizza for lunch – just to make it easier for everyone.

Lessons learned

  •  Oh my god… was I tired after the first workshop! I felt completely exhausted. It’s the type of feeling when everything is in slow motion… like reaaaaaallllyy slllooowwww…. But I still felt great about how it all turned out. Luckily, the 2nd and 3rd runs weren’t as exhausting.
  • I learned how much time actually goes into putting a workshop together. I think I may have spent at least 100 hours when preparing for it.
  • I learned that I enjoy working on a workshop like this. It helped me to freshen up what I know (well, it’s fairly well known that teaching someone teaches yourself a lot, too). I also enjoyed delivering workshop even though I got tired and some people started arguing a lot 🙂 It was fun to observe how programmers from the same team started arguing how to test better and what they could or couldn’t do on their end. I felt like getting such a conversation going was an accomplishment on its own.
  • I wasn’t very good at using all of the information I picked up from eavesdropping on the conversations taking place during the exercises. I guess this is a matter of practice and a few other things… like having a much better command of the material. I feel like I could’ve done this part a bit better.
  • The second half of the workshop could’ve used more actively engaging exercises. For some reason I figured people could be too tired for them… but I guess there should’ve been something that would shake people up 🙂
  • After the first run, I did change the order of topics in part two: I figured I should introduce the terminology of oracles and heuristics before starting to talk about heuristic test strategy model. Else I would introduce words I would only explain later.
  • Good feedback from participants was along the lines of “I had no idea testing is so complex” and “I didn’t know how much more there was to it”. Someone even said that “I figured testers had to be smarter than programmers” 🙂 Since I left the company about a month after wrapping up the workshop series, I can’t tell what kind of overall influence the workshop had. But a few programmers I’ve talked to afterwards have told me they’ve thought about the workshop and what they took from it, so… I guess that’s a good thing. Even though I didn’t mean it this way, the workshops turned out to be my gift to my colleagues before I left.
  • It was also pointed out the using the image of a house helped to pull together some loose ends. Talking about the point of a house (it’s where people live in) nicely summed up the reason we ever develop something – there is a purpose for some human beings.
  • Would I change anything next time? Yes. I would probably pick a piece of software or prepare it myself (you know, learning challenges for me too…). I’m not sure I’d pick a different one for each exercise or not but I’d consider it.

 

Am I happy with my first attempt at putting together a workshop? Yes, quite 🙂

 

Cheers!

Advertisements

Planning and managing my learning… at least trying to

The truth is that when I think of all the things I want to learn (or think I HAVE to learn) about testing… then I want to go HOLY CRAP!!!! THERE IS SO MUCH TO LEARN! I FEEL LIKE I HAVE TO RUN IN EVERY DIRECTION!!!!!!!1!!!!

Yes, really.

And of course I’ve realized multiple times that I need to plan my learning and manage it. Somehow.

Tobbe Ryber’s keynote at Nordic Testing Days in June this year about what he did to become a better tester was inspiring. But I also felt ashamed that I actually HAVE NOT transferred my plan from the back of my head into reality. So I vowed myself that after I have been able to catch my breath (I had just defended my thesis in May), I’ll get to work.

It’s freaking October now, and I can’t say I have made much progress.

And then I thought it’s so true that nothing changes if you’re comfortable. I think I should go DIY and make posters with this message, and plaster my home AND my desk at work.

So I can freely admit that I had been comfortable for a while. And tired, too. It’s only recently that I feel like I can probably switch to top gear and do a lot of stuff like I used to. It’s not like I’ve done nothing but if I compare myself to what I was like earlier, I see the difference.

So after Nordic Testing Days I was brooding over where to start. I got involved in some programming (just a bit) and it was very interesting. Even from this bit I learned a lot about my thinking.

I do a lot of thinking at the back of my mind when doing other things… so my brain hasn’t been completely idle.

Then in September I read the post by Rosie at Software Testing Club (http://www.softwaretestingclub.com/forum/topics/self-managed-learning).

The question is simple: what do you do and how you do it?

Now I have my mind set to “do something about it”. But then I got stuck with the the “how”. I know that I have the tendency to overthink things (sometimes :))… but it seems like the execution part IS very important to me to get right. The truth is that I have tried it before. So what I’ve ended up with are:

  • notes about articles in my notebook
  • notes about these articles on the printouts of those articles
  • bookmarks to these articles in my browser
  • some notes on ideas in some file on the hard drive
  • some notes on a random piece of paper
  • some ideas lost…
  • some organized and some disorganized mindmaps
  • files scattered between my work laptop and personal one
  • bookmarks in another browser
  • stuff in Google reader
  • some random Google docs

You get the picture.

I can be systematic and organized with other things…. or maybe this is what I like to think of myself… But here the problem seems to be that there are many different sources of information that it probably takes a lot more discipline to get it organized than I have exercised so far.

What to do about it?

File distribution between computers is not really a problem. I’ve been using dropbox for years, and now there are other solutions available that are even more convenient (such as SugarSync). So I just have to start using it for storing and organizing my files, articles, mindmaps, etc. PERIOD.

Notes are a different matter. I like to take notes while reading but I read online, I read books, I sometimes print stuff out so I can curl up on the couch with it. Soon I’ll have a Kindle, so that will complicate it even further. I guess I should choose one method, and then if I use another one, be disciplined about transferring information.

I have used Notepad++ but then saving the files was uncomfortable and I had to remember where my stuff was. I have used mindmaps but they don’t always work for note taking. I like to scribble things using my pen as I go.

Recently, I have started using Evernote. I like the way I can easily create new notes, I see what I have created and I can switch between them without opening a file from hard drive. I love that I don’t have to worry about saving stuff anywhere. So I guess I will stick with this one for a while unless somebody can recommend me something better.

I’ve also seen people use Sublime Text for note taking. Not a bad idea, actually, considering this example: https://tutsplus.com/lesson/pretty-task-management/ I haven’t used it for notes but I like it in general. I think it has great features (and I like their color schemes for code…).

Bookmarks seem to be a trivial topic and I think I just haven’t researched enough to find a good solution. For some reason, I haven’t used Google Bookmarks so far. I’ve tried Delicious but this wore me out. So now I have looked into some options, and I have started organizing the links in Google Bookmarks by tagging them. Other suggestions are also welcome.

Mindmaps are a tool I like to use for different things… Brainstorming, planning, note taking on some occasions… It’s their vice and virtue that they can be used for so many different things. So I’m thinking it may make sense to use them for specific things such as building a model of something I have thought about but not for note taking so that I could keep my READING notes in one place only.

Blogs are a massive source of information, inspiration, and ideas and I usually keep them in my Google Reader. I suspect I have quite a few links among my bookmarks as well. Sometimes I don’t want to subscribe but just store this one entry, hence the bookmarking. I should also make it a task for myself to comment more on the blogs. Sometimes I read a blog and think “hmmm…. this got me thinking”. But then I’d have to sit on this for a while to be able to think it through… and by that time… I’ve forgotten or I don’t go back. Or I’ve lost the link.

Good old notebook is what I like to use from time to time. Yes, it’s a Moleskine. It’s black. I can take it with me wherever I go. Sure, I have the Evernote app and MS Office on my Windows Phone but you know… sometimes, just sometimes it’s good to be offline and not at your computer. I feel that I should cut the time I spend at laptop. I work hard, and then I need to read about stuff… so it’s easily at least 12 hours a day that I spend staring at the screen.

Also, I like to go to a cafe and sit down with an article or a book on testing and read it, and take notes, sip my coffee and let my mind wander. So my notebook is not going anywhere.

Planning my learning is something for which I haven’t used a specific tool yet. Pekka cut to the chase and created a mindmap: http://how-do-i-test.blogspot.com/2012/06/be-best-known-if-not-best.html

Another pretty cool tool I learned about from a colleague (she uses it for managing the writing of her MA thesis) is Trello: https://trello.com/

So I haven’t decided on this one yet but it would be nice to use something to track the progress (which would allow me to feel good about myself, of course).

Reflection. Thinking about my thinking and discussing it with someone is by far the most important learning activity I can think of. This is what I need to take time for more often. I need to reflect on my experiences, what I do, how I solve problems at my job. My mentor says that I have quite a bit to share. However, I don’t tend to see it that way most of the time as I perceive the things I do as… usual, regular, not newsworthy in general. One of the reasons, though, can be that I don’t distance myself enough from the daily toil. So to reflect, I need to take the time, talk to people, take notes on my thoughts, and I need to blog more often!!!

Anyway, if you got to the end of this rant (thanks for bearing with me), I’d like to hear your thoughts about how you organize your learning.

I think the best thing about writing this blog post is that it enabled me to organize my thoughts and realize that I shouldn’t be sitting on my hands. Even if I feel fed up and tired I should remember that nothing changes if you’re comfortable.