Reporting Risks to Business: Truly Trying

There were 4 more or less recent events that pushed me out of my comfort zone and that made me see how stupid and futile were my attempts to hide behind “I don’t know how” and “they don’t care” in the context of reporting potential and actual risks in the product by talking business to decision makers.

I

Scott Barber’s keynote  (slides) at Let’s Test blew me away because of how he managed to connect the value of testing to speaking the business language for me. It’s not like I *didn’t* know that information from testing should be used for informing business decisions. But I’d say my recognition of it was shallow and I didn’t fully realize it or act upon this knowledge. It isn’t that me and my team *don’t* give out any information but I realized that we could do it so much better than before by focusing less on giving information to programmers.

Scott’s keynote really pushed me to think seriously about what I’ve done so far. Somehow his presentation of it ended up being so liberating for me that I could take a very long and hard look in the mirror without trying to rationalize the mirror away by saying that I’m somewhat of a victim to my context.

II

Kristjan Uba’s workshop at Nordic Testing Days (brief description and slides) dealt with test reporting and also tied it to mission of testing which, in turn, was tied to the business needs. Briefly: teams got a brief from the customer and then had an hours to test the product, then they had to report their results in 4 minutes. I saw different examples in reporting and recognized the ones I’d like to sample. It became even clearer to me that I need to try and do things differently.

III

Over the past weekend I read “Perfect Software” and I felt as if I had been tweaked by my ear. Sure, a lot of what is in this book  is what I know from other sources or that relates to something I’ve experienced. It wasn’t really new information for me. However, reading it made me more decisive about having to try to do better because I felt like Gerald was hammering me for my mistakes… I haven’t met him in person but well, if he’s so effective via a book, then… 😀

IV

This is probably the most important one but it is also the most painful one. It is the original source to my discontent with how I’ve been giving information obtained from testing. And also a great lesson. This happened a couple of months ago when I ended up between a rock and a hard place, and boy… did I hate this feeling!

The details won’t matter (and I can’t really share them) but here’s what happened: I resisted agreeing to giving beta status to certain important parts of the product by claiming that we haven’t been able to test those parts. Then I was asked “but do you know of any important problems?”

SLAP!

I only had my gut feeling (which turned out to be right…) that certain things in the product are going to cause problems. But what does the business have to do with my gut feeling if they really-really want to release AND I don’t have any better information to give them? It also doesn’t really matter why I didn’t have that information at the time (objectively, the plan I knew about was different and I was supposed to have time to test to find that information, etc). The fact is that I didn’t have it  when it was needed.

So for the next release I wanted to do things differently. And I did it today.

Full of determination but without an exact plan I sat down yesterday and started putting together a risk report based on information my team had gathered while testing. I wanted it to be very simple to read (ideally at a glance), fairly short, containing teasers not full details. I thought of my audience (CEO, a few managers directly below CEO, COO…) and remembered how they have got entangled in details that I have brought to their attention previously. I started putting it together so that it would be fairly high level and easy to read for myself, too.

Here’s the sample: Risk report from testing

The total length was 2.5 pages and I almost didn’t want this to run longer than 2 pages. I should note that this report didn’t aspire to be a full report on even every risk I am vaguely aware of. I just focused on the most important stuff right in front of me. I can start experimenting later which is going to be the most interesting part…

You probably noticed I don’t have the “low risk” category at all. I excluded it to avoid diluting the report or losing sharpness because I feel like this is what is needed right now.

I also had a another document about areas that I think are high and medium risk but that we haven’t started testing yet. I wanted to highlight those areas in case anybody wanted to change the risk assessment (nobody did). This document should potentially include more information about what are the important things we may not be able to test on time (shuffling priorities, unexpected hiccups…). I also asked if there is anything they know of that I should add here or if there is anything in the product they want more information about. Lo and behold, I got 3 more items on my list (plus a few hints).

Maybe if I look at it tomorrow, I will think it looks ugly or maybe I want to cave out my eyes because of how I have used formatting.

But today it looks great enough.

Don’t get me wrong: I have tried to bring different risks to decision makers’ attention but I’ve never presented it in such a way. Also, it’s not like the business side never asks for information from us. But I think they should be better at it and realize that they could ask for and get more and better information. I can help them discover this to some extent.

So I sent it out and I saw a fairly quick effect on some showstoppers (tasks were reprioritized or reassigned…). Today I went through this report at a meeting and well, all those items got sorted out (fixes got higher priority, one feature got pulled out of the release, etc) without any hassle.

My boss said, “Very impressive!”

As for me, I am pleasantly surprised 🙂 And happy because it went better than expected. And even more happy because I learned that I can report to business in a way that is heard, that I broke the vicious circle in my head. I hope I can use this experience to tackle those other circles…

What’s next? I want to keep experimenting with the content, structure and formatting to see if I can find something better (testing reactions…). I may end up changing it around completely because I don’t intend this to by my Template of Final Destination in Reporting. Maybe it’s a good idea to keep people on their toes about how the risk report looks like. 🙂

I expect to run into issues and challenges as I keep going, too but I know I can and will tackle them.

Feedback and criticism is welcome as always.

Advertisements

Experience Report on Presenting an Experience Report

Sunny morning. A cup of coffee. Kurt is singing. What else to do than reflect on my first conference talk!

So here goes… Hot off the press, uncut version of my experience.

How I Ended Up at Nordic Testing Days

At the last year’s Nordic Testing Days, I was nicknamed QA – Question Assurance. When I attended a track, everyone was assured that questions would be asked. Back then Raimond said to me that maybe I should present next year. I waived this aside as a joke… Me? Presenting at an international conference? Dude, stop it…

This really looked like a huge mountain to climb. The idea kind of stuck with me, though, and got the cogwheels turning (very slightly…). When it was time to submit a CFP, I had a bunch of loosely related ideas but a skype call with Raimond helped me to find better focus, so I was finally able to put it together. He’s the main reason I was able to present at Nordic Testing Days. so a big “thank you” goes out to Raimond Sinivee!

Getting the CFP together was a slow process because I was undermining every sentence. As always.

But then I got accepted and then… shit just got real!

The Topic

My extra special and awfully long title of the talk:

“Knocking on the Door with Kinder Surprise in Hand: Experience Report on Building and Maintaining Relationships between Testers and Programmers”

Why so long? Well, if you’re an English major, you have a knack for long and fancy titles in two parts.

I wanted to frame the talk as an experience report because I don’t feel like I am in a position to present the little I know as a bunch of universal truths. I also framed it like that for myself so that I wouldn’t go into the lecturing mode. I feel very strongly about bad leadership and lack of integrity (leadership and integrity are essential to building great teams, too) so I can get very agitated and go into lecturing mode. So reminding myself that it is my context and my context only helped me keep the focus where it should be.

I thought that probably I have ideas that I can talk about and explain how and what we have done with my team, and then people can figure themselves if they trust my presentation of those ideas to try some of them out. If there was at least one person who will try something I suggested, I think I have done well.

And what about the Kinder Surprises? This ended up being the twist I added to my talk. But it’s a true story: I have given Kinder Surprises to a couple of programmers who have helped with something. Essentially, in the context of the talk a Kinder Surprise is a symbol for the building blocks (attitude, leadership, humanity) that make up the bridge between testers and programmers.

Knocking on the door? This is a reference to trying to open the door between my team and the programmers to get the collaboration going.

Journey

After my CFP was accepted, I didn’t start working on the talk heavily right away. But what I did was that during my 20-minute walks to work and back home, I thought about the different components of my talk, how to tie them together, how to flesh out the key points in sufficient detail, how to support my points and stories with examples from my experience, and which examples to use. Looking back, I guess I did the majority of the work during those brief walks.

When I was little, I used to enjoy switching on the autopilot on my way from home to music school or volleyball practice. My feet knew where I was going but in my head I was narrating all kinds of stories. And then I was suddenly in front of the music school and had to wake up. So I did the same thing now: walking at a leisurely pace while my mind being focused on the talk. The 20 minutes was enough for getting something done in my head without being exhausted.

My main worry was that it’s clear in my head but not expressed clearly. So I tried to retell a piece of my presentation to myself and then see if the words and phrases I used made sense or if the example I used is properly linked to the point. On the one hand, I think it helped. On the other hand, it kind of hurt me too because I was very critical of most of my content… So it was somewhat painful at times…

The other big problem for me was the lack of belief that the talk would be successful. But let’s not revisit those dark depths of my mind… The short explanation is that getting crap about my background in the humanities has played a role.

Also, throughout April and May I had quite a bit of family drama going on. A couple of my closest family members were hospitalized one after another and I didn’t know how well things would turn out… The future looked very gloomy at times. So the time I had planned to spend on preparing had to spent on other things. In the end things got better but I admit I was drained… depleted.

That’s how I arrived at Let’s Test: in desperate need of something to kickstart me and kick me out of the gloom and doom. I still can’t believe my luck that it happened. The energy I sucked in at Let’s Test helped me over the finish line. Not to mention the people whom I have thanked profusely but whom I need to thank again.

I asked Jari Laakso for help and he engaged in a skype discussion with me. He asked a lot of insightful questions and took me on a rollercoaster ride: a tough question or challenging my points followed by cheering me on. This was a good experience for shaking me up.

My talk needed some polishing so my new friends Erik Brickarp and Huib Schoots  from Let’s Test delicately gave me constructive feedback about my talk. I think the most important thing they helped me with was that I saw my ideas meant something for them, so I truly started to believe that these ideas matter to other people as well. That gave me the confidence and I quit putting out the fire in my heart.

Last but not least – my wonderful team! I’m nothing without them and I’m thankful for their support!

The Conference

I arrived in Tallinn the day before and decided to relax and just hang out. I felt the nervousness build up but when I felt that, I just retold the beginning of my talk to myself. This is a great tip I got from Tobbe Ryber. I did that for a few weeks before the conference already and I found that it helped me deal with the adrenaline rush. I usually get this rush just before I have to speak in public and this is normal. But if it’s too much, the heart starts racing too fast, and then it’s difficult to breath normally, and then it’s difficult to think clearly.  And then I may fumble. And stumble. Forget an important thing to say. Et cetera.

But rehearsing the first minutes of my talk helped me to kind of “relive” the talk beforehand and I maybe signalled my body that there’s nothing to worry about.

Despite that I still jolted awake a few times the night before thinking “OMG! I HAVE TO GIVE THE TALK!” and then dropped back to sleep. So in the morning I slept in on purpose. I just wanted to take it slowly and not put any unnecessary pressure on myself or give rise to anxiety.

At breakfast I managed just a small bowl of cereal but I got to hang out and chat with Tobbe Ryber and Sami Söderblom. They also fulfilled the roles of “familiar faces to rely on in the front row” 🙂

So I got my props, put the mic on, and got on stage after Lloyd Roden’s talk on building great teams. What a coincidence… 😀

I had a bit of trouble with the remote/clicker for switching the slides. I don’t know if the transmitter wasn’t working very well or I didn’t press the button as the designer of the clicker had expected. So sometimes I had to press it several times. However, I didn’t let that disturb me.

The next day I saw how the pros do it: Tobbe had brought his own hardware for the presentation (a clicker with a timer…).

The nervousness had turned into some sort of excited, sparkly, and confident calmness. If this makes sense…

I felt good and remembered to enjoy myself.

I felt good on stage even though the room was fairly large and also full of people. As Sami and Tobbe later commented, this track was more like a keynote 😀

But I kind of felt how the people gave me the energy as I wanted to embrace the entire room.

I know I stumbled with my words sometimes. I don’t like to learn talks by heart, I want to be able to improvise. But this also means that a problem of mine becomes apparent: I start a sentence using one sentence construction but then somewhere in the middle I change my mind (because there are so many wonderful sentence constructions out there!) and I have to stop, and say it differently.

I know I spoke fairly fast (but later Tobbe said Julian Harty spoke even faster :D) but that was because I was worried about the time limit. I forgot to ask someone to let me know the time in 10 minute segments (there wasn’t a clock in sight anywhere…). That would’ve helped me to time the talk better. Or then the clicker with the timer…

I did look at my slides during my speech but hopefully not too much. I tried to face the people as much as I could (except when the clicker didn’t want to cooperate) and also move around the small stage. The stage was placed diagonally on one side of the room and slides were on the other side.

Here’s a very rough sketch of the setup:

confroom

I would have liked to use the full length of the room to walk back and forth but then I would’ve had to get off the stage and people at the back wouldn’t have been able to see me very well. And then I would’ve had to get back on the stage (anybody up for stumbling and falling over in the middle of their talk?).

I looked at some people specifically every now and then to see if their faces go totally “WTF IS that?!” or if they’re engaged. I could improve the eye contact I think.

But in general I remember feeling good about “transmitting” my message. When I used an example from my team then thinking about them made me feel good, too. They ARE my source of inspiration!

And before I knew it, it was over… Question time! I had extra incentives for people (tied to the theme of my talk) and this worked surprisingly well 😀 Namely, I had some Kinder Surprises with me and offered them for each question. All 6 of them were gone!

Sami also asked a question (that I couldn’t fully answer because he asked about what is going on behind the Closed Door) and he got a Star Wars character Count Dooku as the surprise. Well, good karma was instant: his talk was up next but in a different room and basically, he had to get by without slides because he could show us 2 of them before the loudspeakers started banging some techno and the system had to be turned off. So he had attached Count Dooku to his neckstrap and Sami looked for comfort by fiddling with it while giving his talk 😀

I think Sami did an absolutely awesome job! When the projector went crazy, he just picked up the marker and drew models on the flip-over chart. He totally kept his calm. All I could do was sit in the front row and cheer him on.

Of course it’s sad we couldn’t see his wonderful slides. But like Sami later said himself: his topic was exploring the unfamiliar and that he got to do himself during the presentation.

When I thought about this afterwards, I think Sami’s experience is something I will use the next time when preparing for a talk. I will try to do it without the slides entirely and have “back-up visuals” in my mind for when things should go badly.

But I am extremely grateful my slides worked for me 😀

And then I was really tired. So I went to my room for a while because I felt I could fall asleep on spot. But when I laid down, I couldn’t take a nap. So yeah, then it was time for some beer .

We had a great time with some Dutch folks (hello, Iris, Armando, and Bram!) discussing some “interesting Estonian words”. Sami turned out to be very funny as I was in stitches about some of his facial expressions. Too bad he doesn’t remember all those “context-specific” jokes… 🙂 But there’s the next time, and if anyone wants to know the dirty side of Estonian language, have a beer (or two) with me.

***

There you go…

I feel content as I got good feedback also from people I didn’t know and I guess I surprised others and also myself when I considered that this was my first talk given at such an event to such audience. Armando said something about me being famous now but we’ll have to see about that 😀

I also have ideas about how to evolve this talk because there were a bunch of ideas I had to drop (or else people would have to spend two hours with me :D).

I have some public speaking experience but nothing on this scale. Now that I’ve got this talk under my belt I feel like I’m in a different place now in terms of confidence and outlook on my work.

Needless to say, I’m hungry for more now…