Visualize the Test Strategy

There are plenty of models/frameworks that list what you could have of testing activities – but there seems to be little assistance in WHAT to do in which situations. I have two visual ideas – let’s explore how to visualize the test strategy. And yes, there’s no size fits all – context matters.

It all started with one of my colleagues asking: “Do we have any test framework / strategy document that describe the purpose and which type of customers need which testing: Unit testing, Functional testing, Data Handling testing, Integrity testing,” … +16 other testing activities.

It was clear to me that while ISTQB lists and describes the above testing activities, and Heuristic Test Strategy Model lists heuristics – there’s little guidance as to when these activities are more or less valuable – and in what contexts they apply best. We should obviously not do everything and the kitchen sink in every single testing project in the world. The value of any practice depends on its context – even the heuristics.

What we are looking for, is a way to discuss and visualize the overall test approach. While this can be put in a test strategy document (I often do) – a test strategy is only a written narrative of the test strategy selected. Test coverage strategy and test automation strategy are parts of the test strategy – but we have first to see the relationship between the parts.

One way to go about it could be to visualize the pipeline:

Continuous Delivery Pipeline Example
Visualize the pipeline!

Visualizing the pipeline, as described by @lisacrispin & @aahunsberger, can help you find all the places from idea to deploy, where the story can be tested. I think this model could work even for non-DevOps deliveries too – testing can add value everywhere and there’s more to testing than gatekeeping.

Though if you are not clear of when in the SDLC/pipeline to apply the different testing approaches – you need to have a discussion around value and visibility on hand, and relevance of test activities on the other. That discussion could also help elaborate the boxes on the above visualization.

Another way to go about it could be to make a Wardley map

A Wardley map is an illustration that has the characteristics of a landscape map, and can be used for orientation in an IT setting. Wardley maps have two dimensions: Visibility and Evolution.

Visibility to the stakeholder could be business need or perceived value, as used in “No Code, No Test“. Evolution is mostly about the relative position from unknown/uncharted to embedded/industrialized. For instance looking at IT systems, it matters how evolved the system under test is:

It seems obvious that the more novel the SUT is there more exploratory testing is relevant, and similarly the more standardized the stack is the more continuous testing is valuable. …. Relatively valuable, is probably the better wording. Relative position of the elements is a key output of Wardley maps. (And more about the relative relationship between ET and CT later)

Add more exploratory testing to uncharted systems

So first of all we can position the test activities based on characteristics of the underlying IT structure. Secondly, as characteristics change, we can map the visibility and evolution of each testing activity. Continuous Testing, it self, can be made more or less visible and more or less embedded and industrialized.

As mentioned with ET and CT, we can now use the map to discuss why we need both CT and ET for a specific project. Continuous Testing relates upwards in the value chain to continuous Delivery, while exploratory testing ties more into a more visible end user goal of building the right thing, especially in a context of implicit and tacit knowledge.

To conclude on my colleagues question – to plan a test strategy we need to understand the pipeline, the relative value of the test activities and the relative evolution of the test activities.

[Wardley Mapping by Simon Wardley; template by @HiredThought]

8 thoughts on “Visualize the Test Strategy

  1. Very nice post Jesper, it maps nicely with some of the ideas I’ve been developing around Cynefin, explorability, fail-safe versus safe to fail etc. The core question is are we building to learn(high uncertainty) or building to earn(lower uncertainty). When building to learn we optimise for speed & quality of feedback while when building to earn we are more structured and methodical. I’d love to explore it further sometime?

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.