On the Shoulders of 112 Giants

In my book “Goal-Aligned Test Strategies,” I draw extensively on the work of others. This is a reference post to further credit my reviewers and to list the 112 references in the book.

Special thank you to Lonnie for my writing evenings and to Simon Wardley for finding a path.

Reviewers and Feedback

• Tristan Lombard, a guardian angel for the project since the start in 2021.
• Praise from Simon Prior, Head of Quality Engineering, EasyJet.
• Feedback from Anne-Marie Charret, Quality Coaching
• Feedback from Anders Dinsen, Senior Test manager
• Feedback from Rikke Simonsen, Test Automation Consultant
• Feedback from Dan Ashby, Director of Quality Services

Referenced Quotes

“Making work visible is one of the most fundamental things we can do to improve our work because the human brain is designed to find meaningful patterns and structures in what is perceived through vision. Thus, it makes sense that we have difficulty managing our work when we can’t see it.”

Dominica DeGrandis, Making Work Visible: Exposing Time Theft to Optimize Work & Flow, 2017

“Today, we are offered a bewildering variety of tools and concepts to aid in analyzing and constructing strategies. Each of these tools envisions the challenge slightly differently. For some, it recognizes advantages; for others, it understands industry structure. For some, it identifies important trends; for others, it erects barriers to imitation. Yet, there is a more fundamental challenge common to all contexts. That is the challenge of working around one’s own cognitive limitations and biases—one’s myopia. Our myopia is the obstacle common to all strategic situations.“

“Good Strategy/Bad Strategy: The difference and why it matters” by Richard Rumelt, 2017

“Take your strategy. Apply it to 3 random companies. If it sounds like it would be something they would want (“more money,” “more customers,” etc.), then it’s not a strategy,”

Adrian Howard, 2022

“Instead of well-designed strategies plotted out in sealed boardrooms, executives needed strategies dynamic enough to receive market feedback and flexible enough to evolve.”

How to Make Big Old Companies Act Fast, Pia Lauritzen 2021

“Anticipate your boss’ needs before they do. Help them with prioritization as much as possible. In the same way, they shield you from upper management, shield them from details that are not important – a boss that wants insights into every detail needs micromanagement coaching, and you can help too.”

Tristan Lombard, 2022

“The way we have always thought about strategy is very centralized and top-down. That’s controlling and directive – and leads to very detailed strategic plans. We need to move towards something lighter driven by a shared purpose.”

Rachel Happe, Control Is For Amateurs: Empower The Community To Change The Culture In Your Organization, 2018

“When you learn Wardley Mapping, you may build better situational awareness than your boss has and a better one than anyone in your time. Unfortunately, you are doomed unless you know how to sell your strategy to other people and whether you can sell it in the first place. Nobody will act on your thinking.”

Krzysztof ‘Chris’ Daniel: You need to sell your strategy! 2022

Referenced Links

• “The key purpose of your business can easily be watered down by the hierarchy of your organization and teams,” “here may be dragons,” John Cutler, 2021
•  “The test strategies are often a document facing into the team” Anne-Marie Charrett, https://twitter.com/charrett/status/1543419664952205313.
• “Prediction is challenging, especially about the future” The original author remains unknown (although Mark Twain is often suggested)
• Infographic: “The Dangerous Animals of Product Management,” Product Board 2021,
• “Our business objectives and management bets overrule even the best test results,” TBM 19/52: A Portfolio of Bets, John Cutler 2022,
• “It comes down to the relationship between the organizations,” The undisputed facts about outsourcing, Part 6: Europeans love money, but hate change, 2011, Phil Fersht,
• “the underlying assumption that we will think about testing in the future the same way we did in the past,” James Christie, 2022, https://twitter.com/james_christie/status/1484600368805953552
• “the CIO’s favorite teddy bear was on the line,” Teddy bear methods, James Christie, 2014,
• “There might be many reasons to write a test strategy” Reasons why you should write a test strategy, Mike Harris, 2022,
• “provide evidence of your thinking,” Anne-Marie Charrett, https://twitter.com/charrett/status/1543419664952205313.
• “Questions can be a powerful catalyst for collaborating on strategies,” Choose Your Own Test Adventure: A Strategy Template, Meg Mackay, 2022,
• “pipelines are essential for achieving high organizational performance in software delivery.”, 5 reasons why CI/CD is vital to your organization’s value stream, Matthew Skelton 2022, 
• “Test Automation in DevOps” by Lisa Crispin, Chapter 2: What’s In Your Pipeline?, Test Automation in DevOps, Test Automation University 
• Authors’ Cut—No More Pipeline Blues: Accelerate CI/CD with Observability, Liz Fong-Jones, 2022
• “OKRs can be perfect, and the company can still struggle,” John Cutler, 2021, https://twitter.com/johncutlefish/status/1426655427001872386.
• “When a measure becomes a target, it ceases to be a good measure,” from “‘Improving ratings’: audit in the British University system,” 1997, Marilyn Strathern.
• You can follow the work of Satir: Satir Change Model described by Dhaval Panchal, 2018,
• Dan Ashby’s “A New Model for Test Strategies,” Dan Ashby 2017,
• The Contemporary Quality Engineering model by Anne-Marie Charrett, 2018
• Testing From A Holistic Point Of View, Janet Gregory, 2021
• 8 Perspectives of Quality: A Model, Dan Ashby 2022
• “The questions we return to are the inherent ones around problem-solving.” REMEMBERING JERRY: ARE YOUR LIGHTS ON?, 2022 Markus Gärtner,
• “For a sports team, a strategy is how to win, not the scores,” Stu C, @StooCrock, https://twitter.com/StooCrock/status/1535601111591178242.
• “The beating from stakeholders continues until the results improve,” The beatings will continue until morale improves, Keith Klain, 2020
• “Vernon Richards understood what made his test strategies successful,” How Do You Make A Test Strategy That’s Actually Helpful? Make Sure It Has These 3 Characteristics, Vernon Richards, 2022,
• What one thing could C-level people do to help with testing? https://twitter.com/ministryoftest/status/1486504229602828289.
• “How aligned is our test approach to our business strategy?” Keith Klain, How can software testing support business and digital transformation?
• “Global enterprises predominantly focus on improving the digital fluency of the workforce and migrating the technology to the cloud,” The Five Fundamental Changes that have Reset how we Work, July 2021,
• “And sometimes, SaaS implementations are about a fresh start – for political reasons,” John Cutler, 2022, https://twitter.com/johncutlefish/status/1525149722838593536.
• “Organizations are learning that non-monetary benefits are becoming a key differentiator,” Feeling Demotivated? Consider How Your Job Helps Others, John Coleman, 2022
• Every organization has a culture. The more important questions are things like: how much are they deliberately created, and how well do they serve their purpose? Inspired by Bob, https://twitter.com/BobTechThoughts/status/1491324011728764929.
• Organizational models like Team Topologies enable strategic decisions about organizations, StooCrock, https://twitter.com/StooCrock/status/1535612806615244800.
• “…it is the willingness to learn, and willingness to learn requires two conditions: a mismatch between mental models and reality and enough of free processing power to notice that mismatch.” Wardley Mapping is not for everyone“, Krzysztof ‘Chris’ Daniel 2021,
• Wardley Mapping is a model to visualize strategies developed by Simon Wardley 2016.
• Architecture for Flow with Wardley Mapping, DDD, and Team Topologies, Susanne Kaiser 2022,
• ANZ AWS Summit 2022, https://twitter.com/JamesBromberger/status/1526766941926346754.
• Mapping the Evolution of Socio-Technical Systems, Cat Swetel, 2019
• Building Adaptive Systems For a Fast Flow of Change, YOW 2020,
• Mike Haber labels the Evolution axis A way to figure out what someone is doing & how they might act in new situations, Mike Haber 2022,
•  Emotion -> Language -> Patterns and Metrics from The role of Communities in Innovation, 2020,
• Simon Wardley’s research showed that the attributes of evolution are consistent across practices, data, and knowledge, Finding a Path, Simon Wardley 2016,
• Characteristics Cheat Sheet, Finding a Path, Simon Wardley 2016,
• Helena Marie Meyer, 2019, “In the old days, before cloud and Kubernetes, we treated our services as pets. Nowadays, we treat our services as cattle.”
• “Architecture Modernization: Product, Domain, & Team-Oriented,” Nick Tune 2022.
• Netflix’s 1st competitive advantage was a streaming infrastructure, Chris @wardleymaps 2022,
• A review of Autodesk Inc., maps, numbers and evolution, 2022 by joapen,
• How to not sabotage your transformation, Jonathan Allen, 2022,
• Exploring clinical trial management with the healthcare research group …, Simon Wardley 2022,
• “What is Wardley Mapping doing for us here? It lets us explore a more nuanced view of the problem space. Instead of treating things as one problem, we break the problem apart into a bunch of capabilities.” “TBM 18/52: We Need Someone Who Has Done “It” Before,” John Cutler 2022,
• Learn Wardley Mapping Operated by Ben Mosior, https://learnwardleymapping.com/introduction.
• It will help to illustrate that your teams move differently, as Conrad Braam observes
• end user goal of building the right thing, Mapping Maturity: create context-specific maturity models with Wardley Maps informed by Cynefin, 2018, Chris McDermott
• Wardley Maps have the same properties as landscape maps,
• On the distinction between a map and a diagram, Simon Wardley 2020,
• https://twitter.com/swardley/status/1280891193128337409.
• we can visualize the changes that can happen to an element, Based on “Using Wardley Maps and Cynefin to create context-specific maturity models.” Lean Agile Scotland 2019.
• Susanne Kaiser elaborates On how characteristics change over time: “Preparing for a future Microservices journey using DDD & Wardley Maps,” Susanne Kaiser 2020,

Referenced Books

• “The testing has to align with the quality narrative of the originating customer,” “LEADING QUALITY: How Great Leaders Deliver High-Quality Software & Accelerate Growth” by Ronald Cummings-John & Owais Peer, 2019, https://www.leadingqualitybook.com/.
• “A test strategy document is often the only place to establish the testing narrative, “LEADING QUALITY: How Great Leaders Deliver High-Quality Software & Accelerate Growth” by Ronald Cummings-John & Owais Peer, 2019, https://www.leadingqualitybook.com/.
• “Research from Accelerate demonstrates that pipelines are essential” ACCELERATE, The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations, by Forsgren et al., 2018,
• “Agile Testing Condensed,” Janet Gregory and Lisa Crispin, 2019
• Team Guide to Software Testability, Ash Winter and Rob Meaney, 2021,
• “Simply being ambitious is not a strategy either,” Good Strategy/Bad Strategy: The difference and why it matters” by Richard Rumelt, 2017
• “the chronic conflict of pursuing both: responding to the rapidly changing competitive landscape for features and platforms while maintaining business as usual,” The DevOps Handbook, How to Create World-Class Agility, Reliability, & Security in Technology Organizations, 2016,
• “The research from Accelerate is apparent,” ACCELERATE, The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations, by Forsgren et al., 2018,
• The Team Topologies model, “Team Topologies: Organizing Business and Technology Teams for Fast Flow” by Matthew Skelton and Manuel Pais, 2019,
• If you are the manager of a testing team, I can recommend the book Leading Quality. “LEADING QUALITY: How Great Leaders Deliver High-Quality Software & Accelerate Growth” by Ronald Cummings-John & Owais Peer, 2019, https://www.leadingqualitybook.com/.
•  “Staff Engineer: Leadership beyond the management track by Will Larson.”

Blog posts

• Management strategies are often fortune-telling and hearsay compared to the rigor in template-based test strategies, Stop writing overdone Test Plans, 2021
• The explicit activities in a test strategy document often become a pointy cog in the wheel, The Pointy Cog in the Wheel, 2014
• We must write better test strategies and stop writing overdone test documents. Stop writing overdone Test Plans, 2021
• I have counted test cases many times and even wrote an old guide on tracking them. A Little track history goes a long way, 2011
• “The sooner the next opportunity comes around, the less fuzz there is on solving everything right the first time,” #265 – Using MTTR to Understand When to Test, 2021
• “I have seen a 100-page test process document.” Tester Aided in Two-Digit Million Dollar Deal, 2022
• “Mean Time To Repair (MTTR) plays a massive part in why testing is essential,” #265 – Using MTTR to Understand When to Test, 2021
• “My experience is that you get more organizational traction by aligning with goals,” Focus on Goals over Risks, 2021
• “I’m no longer sure that narration is the best format.” – oh, there’s always a (test) strategy, https://twitter.com/jlottosen/status/1491311405009862657.
• “If there is a misalignment between the OKRs and the rewarded behaviors – the behaviors win.”, 4 Reasons Why Culture Is More Important than Strategy, 2012,
• On Jerry Weinberg, You don’t have to be a boss to be a leader, 2018,
• “Always work with the first visible steps,” Implementing Change, 2021
• There are numerous models for quality, 3 Visual Models on Quality, 2020
• Go Read Accelerate, 2020
• The whole team approach: 9 blog posts at https://jlottosen.wordpress.com/tag/wholeteamapproach/
• Often, I work on strategic (staff) programs, A More Advisory Role, 2021,
• Do you experiment to implement novel ideas and actively seek information? Shoot, Neglect, or Train? 2020
• We often discuss where the “testing people” fit into the organization, Where Does Testers Fit In? 2020
• Test- and agile coaches can focus on enabling self-reliance in the delivery team, Something About Leadership, 2021
• You don’t have to be a boss to be a leader, 2018
• Managers should instead focus on servant leadership and let their testing pros aid in closing the deal, Tester Aided in Two-Digit Million Dollar Deal, 2022
• I often articulate it as the continuum between hunches and hard truths, 2021
•  I found the mnemonic “Darlings, Pets, Cattle, GUID’s” more memorable, 2021
• Once, I participated in a tender, Tester Aided in a Two-digit million dollar deal, 2022
• 30 blog posts here https://jlottosen.wordpress.com/category/wardleys-theories/.
• We need the cattle mindset with structure and standards for factory solutions and stable operations. “Darlings, Pets, Cattle, GUIDs.”
• How to consider the broader system under test, 2019
• Wardley maps have helped me explore the testing and test management craft, 2020.
• observing the market for test tools, Robot Process Automation As A Power Tool For Testing, 2018.
• observing other trends in the IT field, We Have Outsmarted Ourselves Again, 2021,
• I have used Wardley mapping to understand the trends and guide decision-makers, No Code, No Test? 2020.
• the balance between exploratory and continuous testing, Visualize the Test Strategy, 2020,
• context of implicit and tacit knowledge, A Ratio between tests, 2019
• I have used Wardley Maps to understand “Calculating Time to Information,” 2021.
• we can illustrate there are at least the following driving forces in the test strategy: Test Automation for Software-as-a-Service Solutions, Synapse QA 2021,
• Making a Map of Risks, 2022,
• Situational Aware Test Plan, 2021, CC BY-SA 4.0 Jesper Ottosen, 2021 based on Wardley Map Characteristics
• Taking My Own Medicine, 2022,