There are plenty of models/frameworks that list what you could have of testing activities – but there seems to be little assistance in WHAT to do in which situations. I have two visual ideas – let’s explore how to visualize the test strategy. And yes, there’s no size fits all – context matters.
It all started with one of my colleagues asking: “Do we have any test framework / strategy document that describe the purpose and which type of customers need which testing: Unit testing, Functional testing, Data Handling testing, Integrity testing,” … +16 other testing activities.
It was clear to me that while ISTQB lists and describes the above testing activities, and Heuristic Test Strategy Model lists heuristics – there’s little guidance as to when these activities are more or less valuable – and in what contexts they apply best. We should obviously not do everything and the kitchen sink in every single testing project in the world. The value of any practice depends on its context – even the heuristics.
What we are looking for, is a way to discuss and visualize the overall test approach. While this can be put in a test strategy document (I often do) – a test strategy is only a written narrative of the test strategy selected. Test coverage strategy and test automation strategy are parts of the test strategy – but we have first to see the relationship between the parts.
One way to go about it could be to visualize the pipeline:
Visualizing the pipeline, as described by @lisacrispin & @aahunsberger, can help you find all the places from idea to deploy, where the story can be tested. I think this model could work even for non-DevOps deliveries too – testing can add value everywhere and there’s more to testing than gatekeeping.
Though if you are not clear of when in the SDLC/pipeline to apply the different testing approaches – you need to have a discussion around value and visibility on hand, and relevance of test activities on the other. That discussion could also help elaborate the boxes on the above visualization.
Another way to go about it could be to make a Wardley map
A Wardley map is an illustration that has the characteristics of a landscape map, and can be used for orientation in an IT setting. Wardley maps have two dimensions: Visibility and Evolution.
Visibility to the stakeholder could be business need or perceived value, as used in “No Code, No Test“. Evolution is mostly about the relative position from unknown/uncharted to embedded/industrialized. For instance looking at IT systems, it matters how evolved the system under test is:
It seems obvious that the more novel the SUT is there more exploratory testing is relevant, and similarly the more standardized the stack is the more continuous testing is valuable. …. Relatively valuable, is probably the better wording. Relative position of the elements is a key output of Wardley maps. (And more about the relative relationship between ET and CT later)
So first of all we can position the test activities based on characteristics of the underlying IT structure. Secondly, as characteristics change, we can map the visibility and evolution of each testing activity. Continuous Testing, it self, can be made more or less visible and more or less embedded and industrialized.
As mentioned with ET and CT, we can now use the map to discuss why we need both CT and ET for a specific project. Continuous Testing relates upwards in the value chain to continuous Delivery, while exploratory testing ties more into a more visible end user goal of building the right thing, especially in a context of implicit and tacit knowledge.
To conclude on my colleagues question – to plan a test strategy we need to understand the pipeline, the relative value of the test activities and the relative evolution of the test activities.
[Wardley Mapping by Simon Wardley; template by @HiredThought]
11 thoughts on “Visualize the Test Strategy”
Very nice post Jesper, it maps nicely with some of the ideas I’ve been developing around Cynefin, explorability, fail-safe versus safe to fail etc. The core question is are we building to learn(high uncertainty) or building to earn(lower uncertainty). When building to learn we optimise for speed & quality of feedback while when building to earn we are more structured and methodical. I’d love to explore it further sometime?
LikeLiked by 1 person
1st most popular article of the 35th issue of Software Testing Weekly #35 @QANewsletter Sep 10 https://softwaretestingweekly.com/issues/35#start
Mentioned in the Ministry of Testing weekly newsletter, Community News section, Issue 350 Monday 7th September 2020
[…] the visualization of the delivery pipeline it can be a good idea to visualize your quality model. There’s more to it than V-model, […]
[…] currently exploring how “Wardley maps” can be applied in strategies for testing. Here is an example of how I used a map to understand and work with […]
[…] than to pick test charters, test coverage strategy, test automation strategy and the visualization of pipelines. .. it’s a path towards a business […]
[…] It’s a general map, so it’s probably missing some trends and weak signals I have yet to spot. The map is not specific for a certain delivery team, but it seems to me that it would be valuable to map in detail using Team Topologies or Wardley maps with a team – to visualize the test strategy. […]
[…] somewhere new. While a fixed test strategy document can help establish expectations – the visualization and communication around the strategy is the key […]
[…] mentioned in the blog post about visualization, we can now use the map to discuss why we need CT and ET for the project. Based on the […]
[…] balance between exploratory and continuous testing, Visualize the Test Strategy, […]
[…] much great content out there already, my focus was on writing about my experience and my vision of better test strategies. But to do that I needed to stand on the shoulders of others to set the scene and describe the […]