Over the past few weeks I’ve had several coffee catch-ups with colleagues that don’t know what software testers do. I’ve responded that, in general, I put my hands on the features under development and investigate if something bad will happen if we were to release the feature to production.

The response is typically something like: well, how do you do that? What kind of risks do you care about? What testing techniques do you use?

I’ve been embarrassed by my inability to deliver a clean, convincing answer to this question. And I’ve generated more questions of my own: how do I find risks? How do I build confidence that I’ve found the right ones?

Below I dive into my thinking, then I’ll compare my thinking to these two PDFs out of Rapid Software Testing (RST):

• How to Talk About Software Testing
• Heuristic Strategy Test Model

My Thinking

Me: “I identify and investigate risks in the features under development.” Colleague: “How do you do that? What techniques do you use?”

mmm…

Let’s back up, because a technique implies that I have a question to answer. Often when I’m starting my work I don’t yet have a question to answer. I show up to test a feature more or less blind to how it works. Yes, I have some context from planning and grooming meetings, but I have to be very careful about basing my testing work on assumptions I make about the product from previous experience. The ultimate oracle for how the product behaves is the product, not how I think the product behaves.

So, instead of running down a list of activities it’s more interesting to interrogate the product and ask it questions about things that might go wrong. Sometimes the questions I’m asking the product are very familiar to questions I asked it in a previous testing session. If I’m asking the same question, I’ll likely use the same technique to answer it.

Let’s take another step back. How do I know what questions to ask the product? How do I differentiate between what matters and what doesn’t and to whom?

I need three things before I can start doing good testing work:

• An understanding of the context that this software lives in
• The ability to send inputs into the system
• The ability to read outputs from the system

tl;dr – I rely on my mental model of context to generate my list of questions to ask the product.

This doesn’t feel very satisfying, so how about a five-minute thought experiment:

You’ve been asked to test the checkout process in a brick and mortar store. Go.

Here’s where I would start:

Do I have my wallet? What payment methods can I use? What hardware does the store have for payments? What billing service do they use? Are we in space? How many checkout lanes are there? Can I check myself out? What am I buying? How long is the line? How much change is in the drawer? What country are we in? Do I have access to a test setup? Do people use carts? How long have the checkout folks been working here? When was the last time the software/hardware for check out was updated? Does it timeout? How much weight can I put on the scale before it breaks? Is there a scale? What if I have an oversized item? What methods of scanning exist?

Context is all about system decomposition – drawing boundaries around parts of the system and mapping how inputs move through the system, and then putting in breakpoints and playing with the inputs of each subsystem independently.

In the checkout example:

I assume:

• a person
• is carrying/rolling some stuff
• that will get rung up
• and the customer will pay
• then the customer will leave with their stuff

That’s a naive and fairly rigid mental model. It’s a starting place. Now I get ask the system questions regarding each assumption and iteratively update my mental model, which will generate new questions.

So, I think I’m left without a satisfying answer:

Me: “I identify and investigate risks in the features under development.” Colleague: “How do you do that? What techniques do you use?” Me: “It depends.”

An anecdote

Recently I was interviewing a developer. I introduced myself as a tester. He said “Oh, that’s great. I’m going to be best friends with you because you’ll do crazy stuff and look for ways to break my code.”

This made me happy, because somebody wanted to be my friend. But something about the answer made me feel a little bit uneasy. A couple of weeks later it came to me. I don’t just look for ways to break code. I try to break code while focusing on someone who matters. My goal isn’t to find errors. My goal is to boost our confidence that our customers and internal users won’t perceive anything as broken after we ship our changes.

Some ideas from RST