% Confidence: a small intervention that can go a long way

I experienced, or discussed, each of the following three scenarios in the last week. I’ve outlined them leaving an asterisk where I suspect an intervention would have prevented large and unnecessary expenditures of energy.

Scenario 1: “I need an estimate!”

We’re sitting in a Zoom call for a planning meeting. We’ve got a list of user stories and the PO is trying to get a handle on how much work we can fit in to the next two weeks. This makes sense because they need to forecast where we’ll be so they can communicate what will ship so that we can prepare or pivot if we need to.

PO: “What’s the estimate for this story?”

Devs: “We’re not really sure."*

PO: “Come on, guys. We really need to get a plan together.”

Devs: “It’s really hard for us to estimate without better defined AC.”

PO: “Okay. Assume we’re working down Path A. Let’s estimate based on that.”

Devs: “Hmm. It’s a 3.”

The team misses the delivery.

Everybody wonders what happened and assumes we’re just bad at estimating.

Scenario 2: “When will this be done?”

Most of the team is away for the holiday, it’s relatively quiet around the “office.” There’s a relatively big chunk of work outstanding and nobody’s quite sure how close we are to delivery. After estimating the relatively complex body of work the devs put their heads down.

PO: “Guys, gimme an update on Project X.”

Devs: “We’re working on it and making good progress. Should be done tomorrow.”

8 working hours later…

PO: “Guys, did we make it?”

Devs: “Something came up. We’re still cranking on it. Should be done in just a couple more hours.”

Tester thinks to self: Gee…this is supposed to land in 12 working hours and I haven’t seen it yet…*

The next morning…

PO: “Guys, did we make it?”

Devs: “Yea. We’re code complete. Just working on getting it into a testable environment now. Should be ready to go for release.”

PO: “Great! Thanks for all your work!”

Tester: “Uh… I guess I’m working late tonight.”

Scenario 3: “Hm… we’ve got a problem.”

A team member posts something in passing that might be the tip of the iceberg for a fundamental disagreement.

I’m reading Slack and see a comment in passing.

I infer the comment is informed by my teammate’s mental model of our work.*

I recognize that my teammate is representing a mental model drastically differnt than my own.

I decide to confront my teammate about our different mental models.

We blab back and forth for a while. The only thing we accomplished was to delay the completion of meaningful work.

The Intervention

*Whoever is answering, open with: “With X% confidence, it’s ___”

In Scenario 1, if the devs say “With 40% confidence, it’s a 3.” Then they’ve set the team up to have a productive conversation about the work. What about it is confusing? What are the options? What are the unknowns? What would increase the “% confidence”? Perhaps writing better AC would clear up the problem, but the PO actively chose not to write better AC. Maybe the PO is only 60% confident in their understanding of the thing they’re asking the team to build. That would be valuable for the team to know.

Takeaway: To build the right things well at speed, we must acknowledge that comprehension is a continuum.

In Scenario 2, instead of the tester thinking about the sticky situation they could have said, “Team, right now I’m only 20% confident this will make the release. How are you all feeling?” My guess is that the developers would respond by asking why the “% confidence” is so low and what the concerns are. The tester would probably have a well-reasoned reply and the devs would conclude the feature is indeed not going to make the release. The PO could see all of this and report out to the business exactly what the hang up is and when they can expect the high quality feature to land.

Takeaway: When we allow “% confidence” to hide in the background, we assume 100% confidence. That’s dangerous, especially for the people between the feature and the release.

In Scenario 3, I assumed with 100% confidence that my colleague’s comment is informed by their deeper mental model of our work. I imbued a passing comment with deeper meaning. When considering my “% confidence” that this comment is indeed reflective of my colleague’s mental model there’s not much I can say. My confidence is 10-20%.

Takeaway: I wasted a lot of time and energy on a 10% bet. Instead, I could have invested that energy in raising my “% confidence” so that I could get maximum ROI on my attempts to improve our performance as a team.

A couple quick notes

  • The % confidence thinking is heavily inspired by Ray Dalio’s description of the believability score in Principles: Life & Work. Maybe we can make our estimates more believable and our actions more reliable by broadcasting our % confidence.

  • The mental process described in Scenario 3 is framed in Virginia Satir’s model of communication, whereby any interaction can be decomposed into:

    • an observation
    • some meaning derived from the observation
    • some significance derived from the meaning
    • some response derived from the significance

  • In Thinking, Fast and Slow, Kahneman outlines the many ways that our brains jumpt to conclusions. We’re typically very confident in the conclusions we jump to. Considering “% confidence” is a way for us to get away from gut feelings driving our work and move towards data driven decision making.