Glossary

Disclaimer: Much of this content is pulled directly from the authors of the Rapid Software Testing methodology, James Bach and Michael Bolton. I’ve linked to their related content where I feel its appropriate and used quotation marks when using their language instead of my own. For the entirety of my short software testing career Mike Duskis and the Software Quality Association of Denver have been major influences on my thinking and development.

Words mean different things to different people, as they should. What I’m trying to do here is build a comprehensive list of the words I use and how I use them. Ideally, my teams will find this helpful.

I fully expect this list to change over time. If our language doesn’t evolve, it’s unlikely that our thinking will.

---

• Bug

• Check

  • The comparison of a measured output against an expected output.

• Check (Automated)

  • The comparison of a measured output against an expected output, executed by a computer.

• Coverage

  • The breadth and depth of the testing effort (e.g. the user interfaces, apis, and data stores that will be touched in the experiments).

• Defect

• Definition of Done

  • A fuzzy, highly contextual concept that a team might use to frame a constructive conversation about what is “good enough” for a given feature at a certain time (see Bolton’s Relative Rule)

• Error

  • The human decision, oversight, or miss that results in faults in the system (e.g. Zoom’s test team was very stressed when they wrote the automated checks for the audio channels; no one ever checked that the check failures were being detected by the CI pipeline. Their checks are failing, but they didn’t know it until the failure was reported in production).

• Failure

  • A discrete instance of something not working as expected, in the opinion of someone whose opinion matters (e.g. I enter a Zoom meeting and can hear audio from another call, but see no participants in my virtual room).

• Fault

  • A characteristic of the system that results in failures (e.g. Zoom’s code dictates that the audio in my meeeting room should include the audio channels for my meeting id and my meeting id less 1).

• Procedure

  • The methods, and sequence of those methods, employed to yield a measured output for a check.

• Motivating Risk

  • The driver behind a targeted experiment.

• Quality

  • Measuring company-wide output in an attempt to make it faster, better, and cheaper.

• Rapid Software Testing

  • “A mindset and a skill set."
  • “A way to understand testing.”
  • “A set of things a tester knows how to do.”

• Risk

  • Something that could go wrong, in the opinion of someone who matters.

• Test

  • A performance comprised of five parts: tester, procedure, coverage, motivating risk, oracle.

• Tester (Primary)

• Tester (Supporting)

• Testing

  • The identification and investigation of risks.

• Testing (Automated)

• Testing (Exploratory)

• Testing (Performance)