There were 4 more or less recent events that pushed me out of my comfort zone and that made me see how stupid and futile were my attempts to hide behind “I don’t know how” and “they don’t care” in the context of reporting potential and actual risks in the product by talking business to decision makers.
Scott Barber’s keynote (slides) at Let’s Test blew me away because of how he managed to connect the value of testing to speaking the business language for me. It’s not like I *didn’t* know that information from testing should be used for informing business decisions. But I’d say my recognition of it was shallow and I didn’t fully realize it or act upon this knowledge. It isn’t that me and my team *don’t* give out any information but I realized that we could do it so much better than before by focusing less on giving information to programmers.
Scott’s keynote really pushed me to think seriously about what I’ve done so far. Somehow his presentation of it ended up being so liberating for me that I could take a very long and hard look in the mirror without trying to rationalize the mirror away by saying that I’m somewhat of a victim to my context.
Kristjan Uba’s workshop at Nordic Testing Days (brief description and slides) dealt with test reporting and also tied it to mission of testing which, in turn, was tied to the business needs. Briefly: teams got a brief from the customer and then had an hours to test the product, then they had to report their results in 4 minutes. I saw different examples in reporting and recognized the ones I’d like to sample. It became even clearer to me that I need to try and do things differently.
Over the past weekend I read “Perfect Software” and I felt as if I had been tweaked by my ear. Sure, a lot of what is in this book is what I know from other sources or that relates to something I’ve experienced. It wasn’t really new information for me. However, reading it made me more decisive about having to try to do better because I felt like Gerald was hammering me for my mistakes… I haven’t met him in person but well, if he’s so effective via a book, then… 😀
This is probably the most important one but it is also the most painful one. It is the original source to my discontent with how I’ve been giving information obtained from testing. And also a great lesson. This happened a couple of months ago when I ended up between a rock and a hard place, and boy… did I hate this feeling!
The details won’t matter (and I can’t really share them) but here’s what happened: I resisted agreeing to giving beta status to certain important parts of the product by claiming that we haven’t been able to test those parts. Then I was asked “but do you know of any important problems?”
I only had my gut feeling (which turned out to be right…) that certain things in the product are going to cause problems. But what does the business have to do with my gut feeling if they really-really want to release AND I don’t have any better information to give them? It also doesn’t really matter why I didn’t have that information at the time (objectively, the plan I knew about was different and I was supposed to have time to test to find that information, etc). The fact is that I didn’t have it when it was needed.
So for the next release I wanted to do things differently. And I did it today.
Full of determination but without an exact plan I sat down yesterday and started putting together a risk report based on information my team had gathered while testing. I wanted it to be very simple to read (ideally at a glance), fairly short, containing teasers not full details. I thought of my audience (CEO, a few managers directly below CEO, COO…) and remembered how they have got entangled in details that I have brought to their attention previously. I started putting it together so that it would be fairly high level and easy to read for myself, too.
Here’s the sample: Risk report from testing
The total length was 2.5 pages and I almost didn’t want this to run longer than 2 pages. I should note that this report didn’t aspire to be a full report on even every risk I am vaguely aware of. I just focused on the most important stuff right in front of me. I can start experimenting later which is going to be the most interesting part…
You probably noticed I don’t have the “low risk” category at all. I excluded it to avoid diluting the report or losing sharpness because I feel like this is what is needed right now.
I also had a another document about areas that I think are high and medium risk but that we haven’t started testing yet. I wanted to highlight those areas in case anybody wanted to change the risk assessment (nobody did). This document should potentially include more information about what are the important things we may not be able to test on time (shuffling priorities, unexpected hiccups…). I also asked if there is anything they know of that I should add here or if there is anything in the product they want more information about. Lo and behold, I got 3 more items on my list (plus a few hints).
Maybe if I look at it tomorrow, I will think it looks ugly or maybe I want to cave out my eyes because of how I have used formatting.
But today it looks great enough.
Don’t get me wrong: I have tried to bring different risks to decision makers’ attention but I’ve never presented it in such a way. Also, it’s not like the business side never asks for information from us. But I think they should be better at it and realize that they could ask for and get more and better information. I can help them discover this to some extent.
So I sent it out and I saw a fairly quick effect on some showstoppers (tasks were reprioritized or reassigned…). Today I went through this report at a meeting and well, all those items got sorted out (fixes got higher priority, one feature got pulled out of the release, etc) without any hassle.
My boss said, “Very impressive!”
As for me, I am pleasantly surprised 🙂 And happy because it went better than expected. And even more happy because I learned that I can report to business in a way that is heard, that I broke the vicious circle in my head. I hope I can use this experience to tackle those other circles…
What’s next? I want to keep experimenting with the content, structure and formatting to see if I can find something better (testing reactions…). I may end up changing it around completely because I don’t intend this to by my Template of Final Destination in Reporting. Maybe it’s a good idea to keep people on their toes about how the risk report looks like. 🙂
I expect to run into issues and challenges as I keep going, too but I know I can and will tackle them.
Feedback and criticism is welcome as always.