Visualize the Test Strategy

There are plenty of models/frameworks that list what you could have of testing activities – but there seems to be little assistance in WHAT to do in which situations. I have two visual ideas – let’s explore how to visualize the test strategy. And yes, there’s no size fits all – context matters.

It all started with one of my colleagues asking: “Do we have any test framework / strategy document that describe the purpose and which type of customers need which testing: Unit testing, Functional testing, Data Handling testing, Integrity testing,” … +16 other testing activities.

It was clear to me that while ISTQB lists and describes the above testing activities, and Heuristic Test Strategy Model lists heuristics – there’s little guidance as to when these activities are more or less valuable – and in what contexts they apply best. We should obviously not do everything and the kitchen sink in every single testing project in the world. The value of any practice depends on its context – even the heuristics.

What we are looking for, is a way to discuss and visualize the overall test approach. While this can be put in a test strategy document (I often do) – a test strategy is only a written narrative of the test strategy selected. Test coverage strategy and test automation strategy are parts of the test strategy – but we have first to see the relationship between the parts.

One way to go about it could be to visualize the pipeline:

Visualize the pipeline!

Visualizing the pipeline, as described by @lisacrispin & @aahunsberger, can help you find all the places from idea to deploy, where the story can be tested. I think this model could work even for non-DevOps deliveries too – testing can add value everywhere and there’s more to testing than gatekeeping.

Though if you are not clear of when in the SDLC/pipeline to apply the different testing approaches – you need to have a discussion around value and visibility on hand, and relevance of test activities on the other. That discussion could also help elaborate the boxes on the above visualization.

Another way to go about it could be to make a Wardley map

A Wardley map is an illustration that has the characteristics of a landscape map, and can be used for orientation in an IT setting. Wardley maps have two dimensions: Visibility and Evolution.

Visibility to the stakeholder could be business need or perceived value, as used in “No Code, No Test“. Evolution is mostly about the relative position from unknown/uncharted to embedded/industrialized. For instance looking at IT systems, it matters how evolved the system under test is:

It seems obvious that the more novel the SUT is there more exploratory testing is relevant, and similarly the more standardized the stack is the more continuous testing is valuable. …. Relatively valuable, is probably the better wording. Relative position of the elements is a key output of Wardley maps. (And more about the relative relationship between ET and CT later)

Add more exploratory testing to uncharted systems

So first of all we can position the test activities based on characteristics of the underlying IT structure. Secondly, as characteristics change, we can map the visibility and evolution of each testing activity. Continuous Testing, it self, can be made more or less visible and more or less embedded and industrialized.

As mentioned with ET and CT, we can now use the map to discuss why we need both CT and ET for a specific project. Continuous Testing relates upwards in the value chain to continuous Delivery, while exploratory testing ties more into a more visible end user goal of building the right thing, especially in a context of implicit and tacit knowledge.

To conclude on my colleagues question – to plan a test strategy we need to understand the pipeline, the relative value of the test activities and the relative evolution of the test activities.

[Wardley Mapping by Simon Wardley; template by @HiredThought]