Meta: The Next Level in Your Written Test Reports

Once upon a time I wrote an article for the Testing Planet. I never got to publishing it on my blog. It doesn’t seem to be featured on the Ministry of Testing website any longer, however, I got a question via LinkedIn about the “lost” article. Apparently, a teacher of English used it in her class. So I’m happy to republish what I think is the actual version that got published in the Testing Planet magazine. 

In this article I’d like to explore the importance of recognizing cultural thought patterns and how they have an impact on expressing ourselves in writing. I will discuss how I use Robert B. Kaplan’s schema of thought patterns as a heuristic when reading and writing texts (be them bug reports, status reports, etc) to recognize problems in them and to remind myself to consider the audience’s needs.

As a tester I’m looking for and handling information about the product, about the overall status, risks, and perceived quality of the product. I deal with ideas, perceptions and assumptions of my own and others about the product. That’s a lot of information of different kinds and on different levels of abstraction. Therefore, a big part of my job is to convey information by using language in a way that makes the information tangible and easy to understand for my stakeholders.

Different languages enable us to express ideas and convey information differently because a language is usually tied to a distinct culture or cultures. For example, translators bump into this daily: what is available as a concept (expressed in a single word or a phrase) in one language may not be available in another, or it may have a different meaning when translated directly or word for word. Therefore, what makes the translators’ job difficult is not the lack of words or phrases but a lack of matching ideas available in the languages at hand.

The world looks different looking through the lenses of different languages that originate from the context of a culture. For instance, when an American novel was translated from English into Estonian in the 1960s, a problem occurred: how to translate the word “hamburger”? Sure, you can split the compound noun into “ham” and “burger”, translate these into Estonian and put them back together to get a raw and direct translation. You could just keep the original word “hamburger” and introduce it as a new term. However, the problem remains: Estonian culture lacks the concept of what a hamburger is like as it wasn’t available as a regular food item in the Soviet Union. At the time, translators came up with a new term which could be translated back into English as “cutlet bread”. This term is actually still in use from time to time but nowadays “hamburger” as the original term is more common: the concept and the corresponding word from one culture have traveled across time and space, and made home in another culture as well.

Alright, enough of hamburgers, let’s get to the real meat.

Testers working in cross-cultural distributed teams may end up explaining the impact of a technical issue to business people, and explaining an issue with solving a business problem to programmers. That’s where the jargon of different specialties also adds to the complexity of language use. If we end up using our second (or even third) language to communicate with different parties about all kinds of different information as mentioned above, we’re set up for a fair amount of misunderstandings and miscommunication. Communicating ideas efficiently between languages isn’t just about using correct terminology and grammar. It’s about the overall logic, structure and how information is ordered in your written reports or other communication such as e-mails. This is why we need to go on the meta-level and understand how the language we’re speaking influences our thinking.

I shall digress here. When I entered the university to study English language and literature, I quickly learned that I didn’t know how to write essays anymore. I used to be good at it in high school but suddenly, I wasn’t anymore. When I saw the schema about thought patterns by Robert B. Kaplan, I finally understood why. Here’s the schema:


The schema, or rather a series of sketches, illustrates cultural thought patterns in some languages. It is a very high-level and rough model (remember, all models are wrong, some are somewhat useful) and it doesn’t pretend to be the ultimate truth. Kaplan used this to explain why teaching the correct grammar and syntax isn’t enough to help produce understandable and acceptable texts by people who speak English as a second language.

That being said, the importance of the schema lies in how it highlights the differences in thought patterns using what Kaplan calls “the contrastive analysis of rhetoric”. Importantly, Kaplan points out that since sentences are usually used in a context and not by themselves, it makes sense that the writer of a text also “recognizes the logic on which the context is based”. In other words, we can’t make ourselves clear in English for native speakers of English if we’re following the thought patterns of our own native language. 

Going back to my experience as a freshman at the university, it wasn’t that my writing skills had disappeared, it was that my thought patterns, how I used to express my thoughts in my native language was different from English and the Anglo-American writing conventions. The pattern of Estonian is close to that of Russian or Romance writing. Looking at the schema you can see that parallel thoughts, making different moves, inserting digressions or even approaching the subject in a circular fashion are acceptable in different languages but not so in English which tends to be very straightforward.

When writing in English, you have to go about writing a text differently or else your writing is likely to be considered unclear and incoherent by native speakers. Following these three simple rules when writing in English may help:

  1. tell them what you’re going to tell them (make it explicit from the start what this piece of writing is about and what the reader will learn);
  2. then tell them it (explicitly present information, preferably using linking words like “firstly”, “therefore” etc); 
  3. tell them what you’ve just told them (summarize the content). 

In addition, it should be noted that digressions are not tolerated because they don’t add anything substantial to the central idea in the paragraph or paper. Nevertheless, looking at the schema you’ll notice that other languages are accepting of wanderings and digressions which means that when writing the “meta-level” of thought patterns of your native language will “shine through” your writing.

Ignoring the thought patterns of English when writing in English will come at a cost: you’re likely to “lose” your reader in your wanderings which means you’re running the risk of remaining obscure. Also, you’re forcing the reader to make a bigger effort to understand what you’re saying by shifting some of the responsibility of understanding to the reader. In my experience, the underlying logic of English makes the writer almost fully responsible for expressing ideas clearly and helping the reader understand what is being said. In other languages, it’s acceptable to be somewhat mysterious and have the reader do the work.

Now you may say that it’s all fine for writing a paper but what does this have to do with testing at all?

Firstly, I believe it’s useful to be aware of the difference in thought patterns when writing about testing (or any other topic for that matter). There are plenty of bloggers and authors who write about testing in English as a second language. To spread the word about your awesome ideas, it makes sense that you consider your audience and the distribution of responsibility of understanding the ideas expressed.

Secondly, since the thought patterns are ingrained in you and you’re unlikely to be able to switch your native pattern off and another one on, awareness of the pattern will help you evaluate your bug reports, for instance. Are you giving the most important information up front? Is your summary of the problem concise? Is the information in the bug report strictly relevant or do you sometimes digress when describing the steps for reproducing the bug? If you add extra information, is that relevant? If it is, do you signal the reader somehow that this is, in fact, extra information and not part of the central problem? 

Thirdly, if you’re someone who has to write lengthier test reports, you can benefit from observing what the order of information is and how descriptions of problems, risks, and observations about the product are linked to each other using linking words.

To summarize, being conscious of the thought patterns may help you realize why your writing may not be well-understood, and why you may be failing to deliver your point. It’s also a valuable reminder of the complex tool that we use to communicate our ideas, namely, the human language placed in the context of a culture. Keeping the meta-level in mind can help you take more responsibility for communicating clearly with your stakeholders when the working language is a second language to you because it will help you recognize the influence of your native patterns. It can also help remind you to zoom out from the level of terminology or syntax to remember to look at your own patterns from afar and be willing to look inside the patterns of your audience.




Heuristics for Mushroom Picking (and testing)

Mushroom picking has been part of my autumns since I can remember. At least here in Estonia our hunter-gatherer roots still run strong. Wandering around the forest with a bucket and knife in hand while enjoying the tranquility and calm of a pine forest is one of the most enjoyable moments before the doom and gloom of winter.

I find mushroom picking to be a meditative activity where one part of my mind focuses on mushrooms while the other one meditates on whatever comes to mind. So this time I found myself pondering the similarities between mushroom picking and testing.

And yes, I found them. Why I wanted to write about it is because heuristics and oracles aren’t that easy to grasp for beginners or people interested in testing. I’ve taught several workshops and have coached people about these concepts. So at least for people who are familiar with mushroom picking or other foraging activities, comparing mushroom picking to bug hunting and testing could be very relatable.

So here goes…

Choice of location vs product coverage

Typically, I go to a handful of forests that I know like the back of my hand and I know what type of mushrooms I can find there. Since my mission is to effectively collect a sufficient amount of mushrooms, going to the same locations again and again satisfies my needs quite well. However, if I wanted to collect new types of mushrooms, I’d have to take more risks: scout for a different forest, ask people around (though the secret locations aren’t divulged that easily if at all), do more research but I may not end up with any mushrooms after all.

In testing, visiting the same parts of the product over and over again would mean that your test coverage of the product is limited in terms of scope. Unless the product is very unstable, you’re also not likely to find anything very new and exciting there. But if you spend a lot of time in a limited area, you also get to know it really well, so your testing could go in more depth.

It’s always a trade-off when choosing how to spend your time, but we know what to focus on based on our mission: do we need to discover new information (mushrooms) and test previously uncovered parts of the product or not?

Making preparations

When planning to go mushroom picking, I typically keep an eye on rainfall and temperatures, and also the media since news about “good” or “not so good” mushroom season get published in major online news sites. Going mushroom picking later in the autumn when more and more leaves turn golden yellow may also cause a lot of “false positive” alarms (you think you saw a mushroom but… it was a leaf). In testing, I also try to be in the know about recent changes in the software and any rumors and opinions floating around. It’s data that I can use to decide if and what needs testing before I set out.

I make sure I have my rubber boots, proper knife and a bucket or a basket. The basket shouldn’t be too small (too many interruptions when going back to the car to get another one which wastes time) or too large (too heavy and difficult to carry it around as it fills up). Some people also carry a little brush to clean the mushrooms on the spot (I can never spare that time while in the forest, so I rather do it afterwards). Similarly, I will think of tools I may need when approaching a testing problem. Do I need any or do I need help with some of those tools? Do I know the best options available or should I do some research?

Know your oracles

One of the first things I can remember that I was ever taught about mushrooms was how to recognize the poisonous ones. Picking mushrooms really is a matter of life and death: if you pick the wrong ones and actually eat them, you may will die. Period. Hence knowing your oracles for recognizing poisonous mushrooms is really important.

The first thing I was taught as a child was that a poisonous mushroom has a ring around its stalk. This is true for fly amanita which also has a very easily recognizable cap) and other types of amanitas. Hence, it’s generally a good heuristic to use to avoid such mushrooms.

However, one of my favorite mushrooms which is not poisonous and can simply be pickled or frozen (after heating them on the pan to get rid of the water) has a ring around its stalk. It’s called gypsy mushroom (Cortinarius caperatus). Many a times I’ve seen other types of mushrooms being picked but gypsy mushrooms are left behind. Well, I don’t mind, really…

The “ring means poison” heuristic is typically helpful for survival but if it’s the only oracle used, it doesn’t really help us fully evaluate the situation. There are more characteristics for identifying bugs (and poisonous mushrooms) than just one (cap color, type, stalk type, etc). Therefore, we need to ask ourselves if we have more than one piece of information that could be useful when recognizing a bug, and what could be missing that would change the evaluation.

On the one hand, sticking to this specific heuristic means minimizing risks. On the other hand, sticking to very few oracles, I am able to recognize a limited amount of edible mushrooms. I know that I, too, walk past numerous perfectly edible mushrooms but since I haven’t taken the time to learn more and taken the risk of venturing into new territories, I leave these mushrooms behind.

We need to take more risks in testing, question and learn and push ourselves. Because it may be a matter of “life and death” in a different way.

Quantity and quality

So if you walk out of the forest with the biggest bucket of mushrooms, are you the most successful mushroom whisperer? Well, it depends. It depends on the quality of mushrooms you’ve picked. If it’s not the best season and worms have wreaked havoc, you may not find many mushrooms in acceptable condition, so you may start to lower your notion of quality. Maybe you’re not inspecting the mushrooms very carefully and trying not to see the worm holes… Maybe you’re trying to tell yourself this one would still be good… But when you get home, you may end up discarding a large percentage of mushrooms when you finally sit down and evaluate them. Also, if the quantity is large, it is takes many hours to clean, chop and prepare. Different mushrooms need different treatment (some need to be boiled, some need to be soaked before boiling, etc).

Similarly, it doesn’t mean that you’re a great bug whisperer when you find a lot of bugs. Some may not be bugs at all. Some may be low risk and unimportant. Some may not be properly described. And it may happen that if you bring too many issues on the table and ask people to inspect them one by one, you bring the team’s progress to a halt.

When mushroom picking, you will need to inspect the condition of the cap (mushrooms age differently but you can tell if a mushroom is old), the stalk for worm holes (sometimes only the lower portion is bad but you need to cut it shorter to see if the rest is fine; sometimes you may also need to cut the cap). And sometimes you need to leave that mushroom behind.

While testing you also need to continuously evaluate what you find, dig deeper, not take things at face value, and sift through the information to take back what is really valuable.

What to do when you’ve found one

The heuristic to use when you’ve found a mushroom is “look for others of the same kind” nearby. Chanterelles are a good example of this. You also need to know they typically hide themselves in the moss, so you need to peel your eyes and dig around a little. But sometimes you can’t find the others because you’re standing on top of them. Or you placed your bucket on them when bending over to pick the one that caught your eye. Then I use the “glance over your shoulder” heuristic when I walk away from a spot. Sometimes sunlight changes what is visible and what is not, so you need to check the place under slightly different angles.

If you’re a beginner mushroom whisperer, take note of the location where you found the mushroom. Observe the surroundings. What is the undergrowth like? Is it taller, does it contain moss or also grass? Is it on the higher ground or lower? Is it close to certain types of trees? This is how you develop a nose for mushrooming, picking up on characteristics and their combinations which is what will help you navigate even an unknown forest in the future.

In testing we pay attention to similar things. Where are the bugs most likely to occur? What characterizes the location in the product? What kind of issues cluster together? Over time you develop a nose for bugs, too. But only when you pay attention and develop your observation skills.


And finally, I find that it is difficult for me to explain everything about how I find mushrooms (and how I find bugs). There’s tacit knowledge that I’ve accumulated but that I can start to make sense of when I observe myself. Also, it’s sometimes hard to explain why two people go into the same forest and come out with a different amount of mushrooms. Is it luck? Or is it the skills developed throughout the years?

Teaching Testing to High School Girls

This Saturday I participated at a Digigirls event where I taught a testing workshop. Digigirls is an event series in Estonia founded by Tech Sisters – a non-profit organization – to provide girls (9th to 12th grade) encouraging practical experience with roles in software development. Tech Sisters is a community that aims to inspire, educate and encourage women and girls in technology and IT. I was the mentor for testing workshop alongside other professional women who led workshops in project management, programming, design, and UX.

I’m not new to participating at community events be them casual discussions, panels, peer conferences, etc. However, I am new to designing and running a workshop for teenage girls. I must admit that back in the day I sometimes had a hard time relating to girls of my age since I always had my nose in the book, and I genuinely liked to study. Hence, designing a workshop spanning 2×45 minutes that would keep them engaged but also teaches them about the testing profession was a challenge that kept me anxious the few weeks leading up to the event, and also the night before.

Well, I actually did not have to worry so much as the girls were engaged, seemed to have fun, and some were making good observations and were able to describe what they saw quite succinctly.

I wanted to outline the workshop design and my thoughts afterward. Maybe someone will find it helpful when they participate in such events.

Structure and setup

  • 2×45 minute sessions with a break in between
  • girls in pairs or trios (depending on group size)
  • introduction by me – why testing is needed, what it is, what a good tester is like
  • 3 rounds of testing using different test objects for each round
  • overall direction: moving from simple bug hunt to test design
  • around 10-15 minutes of pair testing followed by discussing their findings (why these are issues or not, asking for their evaluation of the program they tested, expanding on and adding context to their observations, adding snippets about tester’s role, etc)
  • each bug/issue written on a post-it note and put on whiteboard


Since I loved Kevlin Henney’s idea that “software is executable fiction”, I mulled over it and figured that I could extend it into a metaphor to explain the role of testing.

Imagine writing a book alone. It’s quite a bit of work, you need to edit and rewrite bits of it, then proof-read and edit again. Now imagine if  10 people are trying to write the book at the same time while trying to keep to a deadline. There are many stories customers tell the team, and the team also has their own interpretations and ideas about what this book is supposed to become.

Someone switches around paragraphs. Someone switches around chapters. Someone cuts out some sentences and words and pastes them elsewhere. Someone rips out some chapters and starts rewriting from scratch. On top of that, there’s the telephone game of passing on information and you know quite well what happens in that game.

So a software tester is someone who is working alongside the team: he or she tries to read this book, find the missing chapters, pages, typos, weird characters, abrupt endings of stories that are supposed to continue. They help the team to bring coherence to the stories and the book in various ways. After all, we want all this to make sense to the readers.

I compared testing as an activity to research where hypotheses are constructed, experiments are carried out, and results are produced that need to be interpreted on someone’s behalf. I talked about quick learning, prioritizing, analytic and systems thinking. I emphasized curiosity, asking questions, being a great communicator.

Exercise 1

Assignment: test this game and find what’s broken in it (thanks for the reference goes to James Bach whom I’ve seen use it albeit for a different purpose).

Learning objectives:

  • a simple program can be broken in complex ways
  • something that seems to be an obvious issue may be interpreted in many different ways, hence you need to know your sources


It was a good introduction and warm-up exercises as the game is conceptually simple. You may start from different places in the game but if you apply some systematic thinking, you will start finding issues quickly (or at least something that looks like an issue).

There were some observations and comments made by the girls that provided some good segues into

  • testability – having a one-way path to click through states is annoying and wastes time, so this would be something a tester would take to their team to get it solved (build a secret feature to get into different states instantly)
  • states and modeling – you can draw up what you see and see if the model can expose more than just clicking in the game
  • domain knowledge – horses can switch between the states in a different order in the real life and knowing this may help you ask useful questions
  • purpose – for whom was this game made, why would anyone play it
  • context – to understand the difference in states between different horses, you could do more research and find out whether some things were intentional (it’s just an early version vs it’s a supposedly fully-functional game that has some flaws)
  • configuration – you might not discover a feature of the game if your computer was muted


Exercise 2

Assignment: you need to evaluate and make recommendations for improving the customer’s self-service account tied to their customer card at a particular department store.

Unfortunately, I cannot provide that piece of software (yet)as it’s part of a test assignment I’m developing at my workplace. I can say that it was a simple piece of front-end prototype with 3 pages, a table where to enter other users to your account, shopping history, and some configuration options (limits, discount codes, etc). There were plenty of issues planted in there, also some in the background.

Learning objectives

  • test for potential business problems in a close-to-real-life business context
  • there is more than meets the eye if you know how to use tools
  • it’s possible to test things that aren’t fully functional


This piece of software looked more like something one would encounter in real life, so it’s easier to relate to. I think they found most of the planted issues as the large majority of them were visual or easy-to-find functional problems.

Once we’d discussed the issues they found, I also opened up Developer Tools and showed console errors and other broken things you can detect. The key point that I think landed quite well was that you have limited view when just looking at the obvious GUI elements. So you can pop open the hood and see what else is there: how much stuff is loaded when a page is loaded, how much time it takes to load something in a web page (so if it’s slow, you can see what makes it slow), etc.

After they had found that some things didn’t work, I also said that well, these can’t work yet as there is no back-end, no database where to save the data. Testers can and should try to get their hands on early prototypes (even if on paper) to test those. It’s possible to test raw things that are semi-functional and it’s valuable. You know it’s possible because you just did it, didn’t you?

One thing I’d change next time is to request them to write down all issues they found but pick 3 that must be fixed above others. That would provide input for a discussion around prioritization and why it matters, why not all issues have the same weight, etc. I did ask them for THE most important bug they thought they found but I think this topic can be built in a bit better.


Exercise 3

This exercise was about testing tours. I figured we could do a few tours so that we can touch upon some test design. I explained the testing tours via being a tourist in a foreign city and how your experience and overview of the city will vary greatly depending on whether you only visit one museum, many museums, only Starbucks, only use underground, etc. The moral of it is that variety will help you learn faster but sometimes you need to focus on one aspect to go deeper or wider.

Assignment: I used Task Coach for this (thanks goes to Huib Schoots who used it in his testopsy at European Testing Conference, so that’s where I got the idea) to get more variety in terms of software.

  1. Create a task (specified the criteria)
  2. Do a menu tour in task coach


Since in this exercise I gave some more specific instructions to follow, there was less of freestyle exploring (which kind of made the group a bit more quiet – note to myself, though it was the end of the workshop already). Probably I could have asked them how this exercise felt different from others to get some more specific feedback.

My main goal with the tours was to contrast the reach of the menu tours with just creating one task. Both are valid starting points. Yet one yields more information about what this program is capable of than the other.So this kind of information will help you make decisions about what to test next (and you want to make good decisions not just about one feature but in the context of the entire program).

As a bonus exercise, we also looked at some configuration options and how to combine some of them into one test. That was a dip into test design on a detailed level and provided an opportunity to show how combining can be a good way to get initial input (see if anything “smokes”, then look for fire). On the other hand, it also showed that some of the things in the combination of settings were difficult to observe, so that may justify trying less options together.


Overall, I feel like the workshop was a success. I received some compliments from them that I will hold dear. It seemed to me that I was able to keep the pace fast enough and content interesting enough that noone gave up or switched off. That was one of my main concerns as I was not sure how quickly we were going to burn through my prepared exercises.

With the second group, we also ended up discussing business and team model differences as they had different testing workshop experience from previous Digigirls events.

All in all, I’d do it again. I hope other testing ladies out there would also jump on similar opportunities – there’s so much we can share. The smart comments and happy faces and light in their eyes made it worthwhile (even my sleepless nights).


How do you do, Head of Testing? vol. 3

Quite OK, thanks for asking. I feel like this writing exercise helps me clear my head a bit.

Many have learned by now that “all models are wrong but some are useful” – a saying attributed to the statistician George Box. In my quest for trying to understand an organization and its dynamics, I’ve come across a few models that help me in various ways. At the end of my last blog post I said I will cover “change and a context model for affecting behavior” next. That was a bit premature as I developed another idea of what to show you before I talk about a specific model. Sorry about that but I’m sure you can wait for that one.

I have created an “inspiration board” that contains images, drawings, and schemas that relate to change one way or another. I am fascinated about the nature of change, how difficult it can be and yet how transformative, how mind boggling can be its effect yet how inspiring its results. I am fascinated by how some people seem to be good at nurturing change for others while some struggle with changing their lives for themselves. We cope and struggle with change in very different ways. I believe a great manager is sensitive to this and develops empathy towards a variety of reactions to change. Because after all, managing nurturing change is the managers’ business.


The title of the board reads “Restart heals all wounds” – written by a colleague who handles our Atlassian stack. The drawings at the bottom and the PFUDOR are not related to the inspiration board.

Satir’s change model

satir change model

Satir’s change model is a classic, I think. I find the model helpful when discussing the perception of the same situation. Once during a team retrospective, I asked people to mark  on the model where they thought we were: in late status quo, in deep chaos or in our way up. It was a way to bring awareness of each others perspective: some felt we were in the middle of chaotic change and were not approaching the end of the tunnel while others felt like it was no big deal and things were stable. That will lead to discussing how your team members react to the same events differently.

When I look at Satir change model, it reminds me to ask myself where I think I am in the process and where others are. Maybe we should have a chat about this? Also, the model reminds me that it may be hard to know when the chaos will come to an end, and that may strain me and others if we pile more change on top of what we’re already going through.

However, I think there’s something that this specific representation of change biases us towards: when you look at it, it hints at the outcome of the change to be “higher” and “better” than what was there before. It may be so. It may not be so. Sometimes people can’t stand to go through the process and quit, there’s a lot of resistance, or the change really hurts the people and the system, so you may end up with lower productivity, satisfaction, what not.

Managing complex change

complex change

This table reminds me to think of components of change. Sure, there could be more added to the list. But conceptually, it helps me analyze what pieces I have already thought of, and which I’ve forgotten about. I can say that I have personally experienced many of the emotions or effects and can very much relate to this matrix. For instance, I remember in great detail how I felt when I was trying to “make testing happen” at my previous job. I had the vision, incentives for myself, and some kind of a plan but I felt anxious because I knew I didn’t have much skills or knowledge about testing. I also had very little resources, so I felt like David taking on Goliath. At the same time, there were many who were to be influenced by the change of bringing testing into the organization and they lacked the incentives to seriously give it a try. Hence the resistance and belief in the “good old ways”.

And of course, I’ve had my share of false starts when I haven’t considered putting together a useful plan.

When I look at this model, I also think of the extra stuff in each box that is not spelled out but is there. For example, a vision in my head or on a page is not enough. It needs to be communicated, stories need to be told, discussions need to take place. Skills are not simple matter either: awareness of skills is a big step, teaching and learning new skills is a whole big process on its own.

Interestingly, the matrix is from a book about climate change. Inspiration can be found anywhere!

Process improvement


It’s curious that the top right-hand picture looks similar to Satir change model. They both have the same bias in common, though: that once change is carried through, things will be better.

I realize that the pictures about process change are really simplified. However, I think they illustrate many people’s experience well, even if using anecdotal graphs. When I look at these graphs, they remind me to ask the question “why”. Why have the previous attempts failed? What did the people do and how they approached change? Why did they approach it this way?

I’ve observed that some unwanted effects of change are blamed on the “object of the change” (i.e. the new technology that was introduced “sucked” and wasn’t the right one) while nobody seems to ask if the people introducing the change were skilled at helping to make it happen, nurturing it, and supporting the process.

Through a few more or less painful lessons (and thanks to PSL), I’ve come to understand that just knowing how some new stuff is supposed to be like is not enough. You need to understand process, how to facilitate it, how to support the process to successfully change something. So these graphs also remind me to focus on the process and not “abandon” it.

Just to add another twist, it also reminds me to consider when it might be the time to let go and not be the victim of sunk cost bias.


So there ya go. I will cover more pieces of this board in later posts.

What do you use to help you be aware of different aspects of change?


How do you do, Head of Testing? Vol. 2

Thanks for asking. Still tired but I made some good progress on a project I’ve been working on for about 6 months. I actually slept better after finishing this draft. But that’s not today’s topic…

Today I want to reflect on jam and systems. Jam systems. Jammed systems. Systemed jam. Systems of jam. I need to stop before I can no longer say “jam” and still understand what it means.


I really like raspberry jam. Even if the seeds get stuck in my teeth, it tastes like summer. If you are a manager and have not heard of the Law of Raspberry Jam, I’m sure you have felt it. Jerry Weinberg writes about the Law of Raspberry Jam in his book The Secrets of Consulting: the more you spread it, the thinner it gets. I learned this as a team lead at my previous job where I tried to build up testing and fix everything and anything that was wrong with quality.

As a competence head, I’ve now learned that the higher you go vertically, the more you are in danger in spreading yourself too thin. It’s an asymmetry that probably can’t be helped and that C-level executives probably know better than I do: the higher you go, the more there are people who place demands on your time so that your time becomes extremely fractured. It’s just there’s still one (jar) of you (to spread across those demands).

Managing one’s energy is important to be able to work on and achieve stuff continuously and in sustainable manner. Managing your energy well also means you know your priorities as you will spend it only on important things. If only it was easy to identify priorities when all opinions are different and contradicting… But there’s always something you can observe that can tell you what’s important to achieve and whom to get support from to succeed.

Being a manager, I’ve found systems thinking to become increasingly important for uncovering organizational dynamics. Combined with sketching it allows to visualize the system and its parts which is a great exercise for clarifying your own thinking. So after a couple of months, I was able to sketch the organizational dynamics that I considered detrimental to its strategy and goals (and to my goals), and the visualizations helped me discuss it with others. Making changes to the organization at this level was not in my scope or under my influence but at least I could express my concerns, and point at potential obstacles.

There was a downside. Once I understood (or thought I understood) how the organizational dynamics worked, I also saw how all the things I thought of trying to bring about change would fail. It’s entirely possible that someone with a different mindset would have only seen opportunities not obstacles.

I see opportunities too. I’m simply pessimistic about overcoming some of them. Especially those that get perpetuated in the organizational structure and behaviors. This is quite abstract but here are a few heuristics to use:

  • what gets talked about in all-hands-on-deck meetings held by C-level management
  • how managers handle requests for collaboration coming to their team or going out of their team
  • how much awareness and knowledge there is in one team about what kind of practices are used in other teams
  • how much slack people have to attend trainings (either internal or external)
  • how much energy and time are people willing to spend on internal educational or social activities
  • how much time do people spend talking about each other’s work (and in what tone) at the coffee machine
  • what kind of objectives are managers driven by primarily
  • how many spontaneous initiatives you see popping up without having to use a stick to get someone do something

The list could go on. Some of these observations can be made within a couple of weeks if you’re new to the organization. Some of them may take longer. Eventually, by observing these behaviors and then investigating the sources of them, you start getting the hang of the system.

For example, if you keep hearing how people “don’t have time” for this or that, then in addition to listening their stories (and justifications), it’s worthwhile looking at how the system structures the use of time, what kind of actions control it, and what kind of other behaviors are related to it. I find that “lack of time” mostly comes down to priorities and preference, hence, it’s informative to learn what drives those.

In systems thinking language:

  • where are the flows and what is their direction;
  • where are the stocks and what’s their relation to each other and the flows
  • what are the feedback loops like in terms of strength
  • and so on

In case you’re new to systems thinking, check out Thinking in Systems: A Primer by Donna H. Meadows. I think it’s a wonderful introduction to systems thinking.

I had an understanding of how the structure of a system affects the behavior of the system and its parts. I mean.. .isn’t it obvious when you observe any kind of system, especially one that contains humans? Reading Meadows and getting more into systems thinking in general, has helped to make my thinking clearer and has helped me understood the system within which I am operating.

As Meadows has elegantly put it, “System structure is the source of system behavior. System behavior reveals itself as a series of events over time.

This sums up quite well why I think it’s essential for managers to be skilled systems thinkers. Surely you can affect the behavior of some humans in the system who may then help you change the system (maybe you end up being a bunch of revolutionaries on the chopping block, who knows). However, without understanding the system, it will be hard to direct your efforts. Remember the spread jam? Exactly.

I see my work mostly to be about affecting the system and some of its structures so as to help change in behaviors to emerge that would improve the system, and that would make it sustainable and resilient. (It could also be that I attempt to remove jam from systems or get them “un-jammed”).

But how to do that? I don’t have a magic bullet but I’ve tried and failed and tried, so that is what I’ll cover in my next post: change and a context model for affecting behavior.


Now where’s my raspberry jam…


How do you do, Head of Testing? vol. 1

Thanks for asking. To be honest, the answer depends on the type of day I’m having. Downward spiraling direction can be found in my days more often than not recently. Ah, I should finally take the lesson from Benjamin Zander about those spirals and ditch them. And also remember rule #6.

A bit more than a year ago I started out with the goal to help testers in the company do a great job. It looks broad and sounds idealistic but it was as good as any mission to take as a starting point. I believed I could find a way to rally people behind initiatives, ignite their belief in change, and help them engage and drive testing and its development. At the same time, I had to figure out what the role of a competence head means…

I couldn’t be everywhere and put my hands into and onto everything, I knew as much. Some groundwork needed to be done or so I thought. Understanding the people and the system, its influences and boundaries was my first goal. So I set out to have long discussions with almost all testers in the company. I didn’t believe in picking and choosing just a few because, naive as I am, I believe all people matter and play some sort of role. So I did that, spent quite a lot of energy and didn’t get much of it back.

I gathered a lot of information in notes, impressions, memories, stories by doing so but then trying to analyze and systematize it ended up being a painful and somewhat confusing process. So I learned that sometimes there can be too many different answers to a limited set of questions. Some patterns may emerge but it may only be your own biases that make you see what is not there. But interpretations you make are very real in your mind. Piecing the information together to describe a big picture of the status of testing in the company through the eyes of testers took me a lot of energy and didn’t give much back.

It also made me think of how I chose the questions. I tried to cover a vast spectrum: understand the person’s history in testing, their knowledge and habits in testing, evolution of testing in this company through their eyes, problems and issues they see, changes they hope for, etc. Looking back, I’d still talk to every single person but be more careful and selective in the questions. I could easily come up with a convincing rationalization why I needed each and every one of those questions, what kind of information they would yield and why I deed this information… My mind is really great at such things. However, excess of stories will make the load heavier than it needs to be. This takes energy and it’s difficult to get it back.

As a systems thinker, I’d been noticing signs of barriers between people. Detecting barriers in an invisible system of people in an organization and off the org chart goes like this: you run face first into something, scream in pain, curse and cuss, and try to determine the culprit. It’s just that you can’t see it but can trace your fingers across air and lightly touch… something. Imagine navigating a glass labyrinth where there are people almost within your reach, only to brush your fingertips against the glass. Some cold drafts sweep past you. Eventually you determine that the actual office layout has an uncanny similarity to the labyrinth. Why, both have glass walls through which you can see people but not interact with them very well. So you turn your back to others and mind your own business with the people next to you. Observing this is emotionally taxing and takes energy that I can’t get back.

So I sketched the organization on paper to visualize issues, and discussed it with some people. In addition to all the other things on my plate, I started trying different things to see what could increases osmosis…

More on that in the future. I’m starting to ramble and wax lyrical about glass walls. That ain’t gonna give me the energy back.


Takeaways from a Pitching Masterclass

Pitching is 95% practice and 5% inspiration. -Annette Kramer

A couple of weeks ago I attended a pitching masterclass by Annette Kramer. It’s part of this strange habit I’ve developed that I watch masterclasses on Youtube to pick up ideas for my own coaching sessions. Now I decided to watch a live one. But more on that in some other blog post…

I learned things about conducting a live masterclass including interaction with the audience. In addition, I also picked up a few ideas that I believe would be useful for testers. Communication skills are among the core skills of testers and I’d say pitching is a subset that could be useful to have in your toolkit.


This masterclass focused on the 3-5 minute pitch format which, according to Annette, is the hardest pitch to do. This is because talking for longer is easier – you have time to expand on your points (though you may water down the content this way…). The masterclass was mostly targeted at different business representatives who need to do pitching to investors or potential business partners. We had a good mix of people: from a startup pitch to looking for partners for school software to a showroom rep. Therefore, it was great to see exampels of pitches for different audiences.

Annette live-coached the pitch makers starting from how they walked to take their spot, to how they stood, how they expressed themselves, and she also re-engineered the flow of their pitch on the fly while also including the audience in giving feedback. It was inspiring to look at how she worked with people.

Selective Hearing: Lessons in Communication

Annette says that when an investor listens to a pitch, they hear “blah blah blah will I make any money on this blah blah blah when will I earn money back blah blah”. When a potential business partner listens to a pitch, they hear “blah blah blah will this make me look good blah blah will I make/lose money with this blah blah”.

How does your audience listen and what is important to them?

Do you focus more on what you want to say or what they want to hear?

During the past year I’ve dealt with C-level and other directors and managers more than I previously have, so this one hits home and is a good reminder. I know I frequently fall into the trap of thinking more about what I want to say and what I hope the effect to be, rather than doing more listening to be able to target the message better. And understand the people I work with better.

From observing a number of testers over the past year, I think there is an important takeaway/reminder here: when talking to your manager (or some other stakeholder/decision maker) about testing, don’t focus so much on your specific testing problem but on the impact of the problem. When focusing on the impact of the problem, you can think of what that manager/stakeholder sees and what they’d like to hear. I bet it won’t be some testing-specific talk about the issue you want to address or the idea you want to introduce. I bet it would help you if they heard “blah blah solving this will make us look good blah blah blah solving this will mitigate the risk of not fulfilling financial goals that  a director somewhere set me blah blah blah”.

That being said, I really liked and I agree with Annette’s proposition that you can’t sell or push ideas on people – it’s much more liberating to think of pitching as a way to offer people an opportunity. So this is why you need to know what your audience cares about and is interested in.

I don’t do pitching on the stage to my managers but I treat some hallway conversations or situations at meetings as micro-pitching opportunities. I have a lot of ideas and I keep looking for ways to get buy-in or traction to take the ideas further. I don’t always know which ones get better traction which means I need to pitch them several times to different people. Which takes me to the next takeaway…

Process. Process Everywhere.

Annette emphasized that it’s useful to think of pitching as a step in the process, and to keep the process in mind (not the result). The goal of the pitch isn’t to close the deal because hey, that hardly happens so easily (or right after a pitch).

The goal of the pitch is to get people asking questions, to keep the conversation going. When I later talked to Annette about it, she said that we do this outside work all the time. And I said, “Oh, this is why we have friends… because we keep the conversation going and this is a process”.

When trying to approach a decision maker with an idea (which probably will take up time (=money) and money, so there are considerations in their mind you may not know about), don’t think of it as a “make or break” situation at the first try. I find myself sometimes doing this exact thing and then getting frustrated. Well, that’s not helpful, is it? And it’s not helpful because if I focus too much on the result, I forget about the process of getting the result. Introducing new ideas in organizations can be difficult, so focusing on the process, focusing on starting and keeping the conversation going is helpful.

Mean It. Clearly.

It was fascinating to watch Annette pick up the difference between when the speaker really meant and believed in what they said and when they didn’t because they focused on what to say (or remembering what to say…). I observed a significant difference in this person’s body language and facial expressions in the pitch after Annette had made some adjustments and asked them some questions to help them discover what they actually meant.

And I mean if I could pick it up, so can you. And other people will pick it up about you. Here’s another takeaway: don’t be abstract, be specific. It will be hard for you to say it like you mean it if the concepts you use are too abstract (and it will be hard to grasp).

Annette did a great job helping people to be specific and get the real meaning out from behind the words. I think this is also a process: you start with an idea, and through practicing a pitch for it, you peel away layers and arrive at the core that will be specific and clear.

We also addressed the issue of using jargon and how this makes attempts at being specific revert to being abstract (“What do you mean by [this thing]?”). I’ve also observed in testers that they tend to use testing jargon when talking to stakeholders who don’t know anything about testing (or don’t care about it…). It’s a similar point to that above: think about the audience you have and what they can understand and want to hear (probably not jargon). Focus on the problem that they feel related to. Annette pointed out why TED talks are so great: among other things, the speakers avoid jargon (so that everyone can understand what they’re talking about).

And don’t talk on autopilot!

Autopilot leads to not really putting yourself in the words you’re saying and you end up “just talking” not delivering a message.

Annette had a great tip for this: remember why you care about what you do/say BEFORE you say it.


I’ll blog a bit more on the pitch structure and other takeaways in another post.