Using Wardley Maps to visualize test strategies and align the test strategies with business goals.
-> Available on LeanPub: https://leanpub.com/goatsbook
strategy
On the Shoulders of 112 Giants
In my book “Goal-Aligned Test Strategies,” I draw extensively on the work of others. This is a reference post to further credit my reviewers and to list the 112 references in the book.
Special thank you to Lonnie for my writing evenings and to Simon Wardley for finding a path.
Reviewers and Feedback
- Tristan Lombard, a guardian angel for the project since the start in 2021.
- Praise from Simon Prior, Head of Quality Engineering, EasyJet.
- Feedback from Anne-Marie Charret, Quality Coaching
- Feedback from Anders Dinsen, Senior Test manager
- Feedback from Rikke Simonsen, Test Automation Consultant
- Feedback from Dan Ashby, Director of Quality Services
Focus on Goals over Risks
Looking into the discussion on what goes into a Test Plan and what goes into a Test Strategy – it’s my personal opinion that we can improve our business alignment. Risk-based testing and Product Risk Analysis have been around for long – but better models have emerged to address what will be more impactful.
Continue readingConway’s Law for Test Automation?
Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.
Melvin E. Conway via Wikipedia
Conway’s law continues to be one of the hard truths in IT deliveries. It tells us that solutions are shaped – by the shape of the delivery organization. While Conway’s law is usually seen on a macro level (Just Eat UK is an example) – it also applies to smaller units (see the Angel of North anti-pattern).
As software test automation becomes more and more like a software development project – I would hypothesize that Conway’s law indeed predicts the shape of the (test) automation solution. So in other words, if the shape of your automation is a pyramid/triangle, so is your team structure heavy on development tests.
Continue readingStrategy is about Making a Map
Strategy is not where you are heading, but how you’re getting somewhere in the long run. That goes for all strategies, and even for test strategies. Though for test strategies we often get caught up in mechanics of selector strategies, testing types and techniques that we lose track of the higher purpose: Moving the business towards a vision.
Continue readingAlign you Test Strategy to your Business Strategy
Obviously! – But often where we fail to do this as testing professionals. We get caught up in terminology discussions, application of standards and obligations and who gets to do the work – that we forget to align with the business side of things. And thus the beatings continue until morale improves – if you don’t align you test strategy to the business*.
The business side can be hard to read. Also coming from the back story that testers long for objectivity – and “just” want to state the facts for the decision makers. I know, I’ve fallen into that trap many times.
We need to be able to read the business strategy and prepare the test strategy accordingly... and business decisions first.
Continue readingInnovation in Testing
Let’s look at testing and test management as something you can build expertise in, thus it can be placed in various places on a wardley map. Similarly innovation activities in the field of testing can be modelled by “Pioneers, Settlers, Town Planners” [also originally swardley, article by Itamar Goldminz].
The model has three types of talent: those that experiment, those that build products and those that optimize the products/commodity. Shortly put:
Each group innovates, but there is also an built-in drive from experiment to product, to optimal commodity and back again as components to experiment on. As stated in the original article (from 2015) all three kinds are brilliant people. We can relate the model both to what value the customer looks for and what kind of activity the organization strives for. We can apply it for the broader testing field as not all testing is pure play experiments and not all testing is a commodity.
Examples of Pioneer experiments could be all the fuzz around RPA, AI and ML.. and square lashings on the System Under Test – on the technology side. On the practise side, it could be emerging practices of how to test in the space of infrastructure or IT service transition. It’s the “Pippi Longstocking” of – “I have never done that before, so I probably can“.
The settler activities are all about taking the emerging practises mature them and make them repeatable. Shortcutting the time to learn something or repeat some novel practise in a new setting. Some examples could be: A Practical Guide to Testing in DevOps, the shifts of testing (at their time of writing) as ways to codify emerging practices.
Example: In 2018 I did management of testing of a large enterprise IT transition of 700 servers, running 100+ applications – it was a novel first time, so we put together some testing practices that seemed to work (for that context). In 2019 I’m doing a similar transition of similar size, where we try to repeat the practices and approaches.
The brilliant quest of the settlers is to take ideas and built innovative and established solutions for the broader audience. Most settlers are probably framework (and content) creators .. not framework maintainers.
As soon as a practise has been established it’s up to the Town Planners to maintain and optimize the practices. To me, examples in this space includes:
- Using Selenium to test web applications with
- Using BDD/Gherkins for collaboration
- Using agile practices and embed testing in the agile teams
- … following the ISTQB cook book
You mind find it harsh that I group all of those practices together. To me, they are so established by now that they can be purchased. It’s a commodity market and it’s frowned upon if you don’t use it. But still – innovation happens and town planners do a brilliant job. It’s about faster, better, smarter – and especially about building more effective teams.
Also the Town Planners build the components that the Pioneers stand upon for their next novel idea. One example could be that to test web applications with code-less test an RDA tool utilizes the Selenium framework.
Broaden the scope of the SUT
When testers talk about SUT (System Under Test) there seem to be an implied context of it being software, developed, bespoke software to be specific. Let me broaden the notion of a SUT using Wardley Maps and with that illustrate how testing can add value across the board.
Bespoke software (aka Custom built) is where the solution (SUT) is built and maintained tailormade to a specific company by a specific team that answers to order giver. When you build and maintain an app or web site for a company and is embedded in the team delivering the code base – it’s usually in a bespoke context.
Experiment/emerging example: Built an internal web site to do some simple public service case management. Write it based on MicroSoft .Net and IIS technology. The solution is new and novel, so interaction with the user is important.
All the commotion on buiding MVP experiments and interaction with the Product Owner are usual symptoms of a genesis situation. As the processes mature and products emerge, the solution development becomes more an customization activity.
Customization example: Implement Dynamics 365, SiteCore, SalesForce etc – but taylor and code them to your specific purpose. I have worked in a project taking Dynamics 365O and creating custom forms to handle public sector health events.
The last class of software “development” is the pureplay configurations of standard solutions. This is the context of SaaS – pay the license and get started. Think SAP or Office Applications, anything that is so accepted that it’s almost free (OpenOffice) and kills the IT department.
Let me draw this on the axis of Wardley Maps Evolutions:
Similarly we can add the underlying infrastructure to the drawing. As solutions move to the cloud and infrastructure becomes code, the system under test could very well be code around infrastructure. Initially bespoke infrastructure experiments (in perl?), and as time moves – even infrastructure becomes a commodity in the form of Amazon S3.
So where is your SUT? – what is the path down the stack? Because there is a huge difference between testing custom code for cloud services, as compared to product customization on actual physical owned hardware.
Let’s think testing outside the bespoke areas on the map too. Some current examples I am working on are:
- Infrastructure transition of 700 serves from being owned to being hosted
- Application transition of 50+ applications from being owned to being outsourced
- Transformation of a standard form management solution
- Implementing a standard system for ITIL case management
These projects have no code, the SUT is either a server, an environment (collection of servers), a form, a process or something else. While we do know a lot about testing in bespoke software contexts, the practices for testing in transition and transformation are emerging practices! This gives us and extra layer. And this is where it gets interesting.
There are plenty of standard practices (SAFe, agile..) but the practices for testing in the context of transition is yet to materialize.
The same model can be applied to IT as a whole. IT support and end user computing (devices, desktop operation) are to the very far right as commodity services. While on the far left is the constant experimentation and tinkering (of AI, ML and RPA) to become actual products.
If we only see testing as part of building bespoke software we fail – we fail to see the horizontal and vertical contexts, where the testing disciplines can add similar value and impact.
Complexity is a Matter of Perspective 