Making a Map of Risks

I’m currently exploring how “Wardley maps” can be applied in strategies for testing. Here is an example of how I used a map to understand and work with risks.

In my current projects the system under test is not software development, but rather a end-2-end transition of IT services. The technology is part reinstallation, part spin up of containers – in other words, some elements are more or less commoditized. Also some elements are more or less visible to the stakeholder. It seems like a good fit for a Wardley map….

How to use Wardley Mapping to understand how you deliver customer value

Read the intro by Stephan Willemse on Medium

First I need to tailor the horizontal axis a bit. I have read up on the articles by Simon Wardley and Ben Mosior with regards to the framing of “evolution”. For a quick intro read Erik Schön describes the four stages, like this:

  • Genesis: the unique, the very rare, the uncertain, the constantly changing and the newly discovered. Our focus is on exploration. 
  • Custom Built: representing the very uncommon and that which we are still learning about. It is individually made and tailored for a specific environment. It is bespoke. It frequently changes. It is an artisan skill. 
  • Product (+ rental): the increasingly common, the manufactured through a repeatable process, the more defined, the better understood. Change becomes slower here. Whilst there exists differentiation particularly in the early stages there is increasing stability and sameness. 
  • Commodity (+ utility): scale and volume operations of production, the highly standardized, the defined, the fixed, the undifferentiated, the fit for a specific known purpose and repetition, repetition and more repetition. 

There is also a relevant article, where Evolution is the level of ambiguity – which is just one of the relationships Wardley Mapping have with the Cynefin Framework.

The key legend to the horizontal axis is the relative position from unknown/uncharted to embedded/industrialized – to you or the organization being mapped. It’s possible to tailor the labels specifically to the components being mapped – like the legends of altitude lines, buildings and roads on a map. Here’s one for the specific risk context above:

OriginalConceptHypothesisTheoryAccepted
Risk legendUnchartedKnownStableStandard
ElaborationWe don’t know the risks yetWe see these risks.Risks are stabile.Risks are reduced to standard components
Experimental tailored legend

First I map the system landscape: relative to the business vertically and relative to Risk stage horizontally. It gives me this anonymized map:

It’s probably only half-way right, but with this I can consider:

  • How far can I move the elements to the right?
  • What inertia blocks movement?
  • Which ones can I move the furthest?

With the map (position and movement) I can consider what happens when risks change and evolves – how does evolution affect the test strategy?

If we automate Configuration, the risk is significantly reduced.

Reinstallation can only get us to a certain point, due to

One Test Case is All You Need

[Originally on Ministry of Testing, November 20, 2013: One Test Case is All You Need. Reposted from the web archives]

  • Does it work?
  • Do the old features work as before?
  • Does it work on the CEO smart phone?
  • Does it look good on the owners CRT monitor?
  • Does it make management look good?
  • Does it make a difference to the customer?

When you start your work of identifying testcases and test ideas – consider it an exercise to identify the fewest testcases needed for the application. What would be the one thing you want to confirm and explore – to build confidence for the customer?

Many times I have experienced that, in an enterprise setting and due to market pressures, the most important test cases where the regression tests and a few daily smoke-tests. Often these were the tests that major releases depended on also.

“No problem Mr D – the dinosaur count is defined as a global constant.”

What testcases matter?

Really – it’s a trap to consider that the individual testcases matter … for the business. When we look into the business purposes it’s not even the app being built either. It is what the app enables the business to do: the business features and transactions between the users and the business. And remember that there will very likely be a difference between what the business wants and what the users want.  Let us look at an example here:

The local LEGOLAND theme park has a park app (iOS and Android) [In 2013]. They are proud of it, and have printed ads on the physical map and mentioned everywhere. The theme park management probably has a business case for the investment in app development and advertising, that pays off beyond “we need one of those app things” (true management quote).

The smart phone app enabled you to do the things that the physical map had: Offers and advertisements for restaurants and season ticket upgrades, a simple list of performance shows and similar push information from the park to the visitors. To ease navigation the in-app map always started at the entrance…

[which is not where I am all the time]

Look at the User Needs

But when I visit the park I want these things:

  • Given that I open the map, I want to see where I am per the phone GPS
    • – and how to navigate to a specific ride
    • – and mark where I have parked the card
     
  • Given that I open the show program,
    • I want to see it compared to the actual time
    • I want to compare it to the times where I have booked rides[1]
     
  • Given that I open the list of rides
    • I want to mark them individually as “go see”
    • I want to know the estimated wait time
    • I want to add a reminder if I book a ride at a specific time
     
  • Given that I am in the park with friends and family, I want to communicate with them
    • Where are you, where are you going next
    • When and where are we eating

Compare that to the existing functions mentioned above and you can see why I use text and phone messages – and the old-school solution of a physical map. Imagine the app development having just one requirement to justify the quality of the app: Given that the person is in our park, help her to make it enjoyable.

If you can come up with just one business transaction – that crystallizes why the customer will be kicking and screaming to want to use your application, then you have a very good understanding of your customer and all you need is that one testcase.


[1] For the LEGOLAND ride ”Traffic School” you book a specific time-slot.

Test Master S1:E1

In Taskmaster (the TV show) celebreties are rated on solving challenges. At Test Bash Manchester 2020 the participants got a challenge too. A good interactive challenge, where you had to think outside the box. Encore MoT Team!

I’m not usually a fan of celebrity shows, but puzzle solving activities in Task Master show taps right into creative thinking and thinking outside the box. I get a kick of challenges that are not by the book… 😉

In the TV show the celebreties and comedians are tasked with challenges like: Stack things – without being able to see, you have 10 minutes – Time starts now.

Snippet of DK version of Taskmaster.

International versions of the programme have been made in Belgium, Sweden, Spain, Denmark (as above), Norway, Finland and the US (according to Wikipedia). The Guardian has this article about it: ‘Rage, clumsiness and brains’: the joy of Taskmaster, TV’s funniest show.

Initially you could think it’s all about the requirements, or the brief, i.e. You’ve been challenged to record the most creative way to get a piece of paper in a bin. While you have to solve the challenge – the key actually to test the brief. When you analyse what’s not there, and bend the words that are – that’s when you have the innovative edge case that impress the most. Go Bruce!

While the award was not for a golden head of BossBoss Richard and no-one misused/involved the assistant DojoBoss Mark this online conference interactive activity is hopefully on for another season.

I was presenting “When Subject Matter Experts test” at Test Bash Manchester 2020. As I mentioned the FDF activity of making fireballs in my talk, my submission had to involve setting the paper in fire:

3 Visual Models on Quality

Besides the visualization of the delivery pipeline it can be a good idea to visualize your quality model. There’s more to it than V-model, Agile/Scrum and other old models. The following models are at least as established and practice based as any old book

While numerous test strategy document templates and document formats exists, the focus of this article is visual models. From what I have seen with regards to test strategy document templates the Ashby model, below, illustrates most of the practical content in such templates.

Dan Ashby’s Perceived Quality Model

Dan Ashby’s “A new model for test strategies” illustrates that  a test strategy needs to consider the context, the approaches, the structure, the test ideas and the test execution, including continuous testing – to gather the knowledge about the perceived quality for the system under test.

Dan Ashby: A model of test strategies 

Heuristic Test Strategy Model

Besides the Ashby model the Heuristic Test Strategy Model (HTSM) also comes into mind. The HTSM list properties around the Project Environment, Quality Criteria, Product Elements and Test Techniques to define the Testing and Quality Story.

Heuristic Test Strategy Model

Quality Ice Cream Truck

Other more recent visual models of testing exist – based on the works of Jerry Weinberg, Rich Rogers developed a visual model than Sergio Freire has recently refined in the “Quality Ice Cream Truck”:

Freire: Quality Ice Cream Truck

The Ice Cream Truck model is a graph of the of the relationship between testing, risks, quality and other elements. It frames the perception of the role of testing, which is the corner stone of every test strategy.

Review

The three models (Ashby, HTSM and Ice Cream Truck) states generic properties of test strategies . You need to draw your own model or list your own properties to visualize the test strategy based on any one of them. The models are diagrams, or directional graphs, that illustrate links between items. You can rearrange the components and keep the directional links and it would still be the same meaning. Well, besides the ice cream truck…

While I find the models above useful – I still lack a visualization in the discussions and decisions needed to derive at a test strategy. While listing generic properties can get me started, I still need to visualize:

  • Who does what parts of the testing – and why?
  • What to automate?
  • What tools and frameworks to utilize?

And even more: What is the value for the decision maker for each part of the test strategy? ? As Wardley maps are drawn from decision maker needs and enable discussions and decisions around maturity and value – there seems to be a match to the challenges above.

My findings are:

Episode V: Return of the ESN

In which our hero must revive old wisdom to battle empires and where the band of front runners will be supported by fuzzy friends to bring down legacy structures.

Enterprise Social Networking (ESN) tools are a rage these days with the global pandemic of Covid-19 sending all staff to work and collaborate from home. Microsoft, and work place after work place, pushes the Teams platform for collaboration, calls and file sharing. As if the cargo cult ceremonies will actually have an impact – this time around. So the empires and old ways of working remains.

The drive towards a learning organization is not new – it has been around at least 10 years. Remember the battle of Ya.. Yammer vs Jive? We should dig our x-wing into the swamps of ancient wisdom (all from 2012):

And if you want something more recent:

A common misconception is that if a company upgrades its technology, digital transformation is done. The reality, however, is that digital transformation isn’t about software or technology — it’s about organizational adaptability. To keep pace with the change driven by digital transformation, organizations must be agile and adaptable, and organizational culture is crucial to the success of any digital initiative.

Why Culture Change Is Essential for Digital Transformation, 2020

… as always culture eats strategy for breakfast, and the cobbler’s children runs barefooted. … yet.. the Ewoks managed.

The Ewoks manage to overwhelm and completely defeat a technologically superior force simply by using conventional military tactics.

Wired: Ewoks Are the Most Tactically Advanced Fighting Force in Star Wars

So who are our fuzzy friends, that can help? First of all there is the millenium trend/evolution of the digital natives, as collaboration tools becomes more and more prevalent, – companies have to catch up to retain the employees that desire to be in a learning & generative organisation. As well as they have to keep up with providing a culture of WFH and trust.

Secondly the Jedi team that we need are the Community Managers and leaders of Community of Practices like Rosie and Rachel:

They can remind us that for the enterprise collaboration to succeed we need to consider the four types of interactions: the fans, the followers, the fringe and those further away. The fans needs to involve the next levels for the empires and legacy structures to fall.

Visualize the Test Strategy

There are plenty of models/frameworks that list what you could have of testing activities – but there seems to be little assistance in WHAT to do in which situations. I have two visual ideas – let’s explore how to visualize the test strategy. And yes, there’s no size fits all – context matters.

It all started with one of my colleagues asking: “Do we have any test framework / strategy document that describe the purpose and which type of customers need which testing: Unit testing, Functional testing, Data Handling testing, Integrity testing,” … +16 other testing activities.

It was clear to me that while ISTQB lists and describes the above testing activities, and Heuristic Test Strategy Model lists heuristics – there’s little guidance as to when these activities are more or less valuable – and in what contexts they apply best. We should obviously not do everything and the kitchen sink in every single testing project in the world. The value of any practice depends on its context – even the heuristics.

What we are looking for, is a way to discuss and visualize the overall test approach. While this can be put in a test strategy document (I often do) – a test strategy is only a written narrative of the test strategy selected. Test coverage strategy and test automation strategy are parts of the test strategy – but we have first to see the relationship between the parts.

One way to go about it could be to visualize the pipeline:

Continuous Delivery Pipeline Example
Visualize the pipeline!

Visualizing the pipeline, as described by @lisacrispin & @aahunsberger, can help you find all the places from idea to deploy, where the story can be tested. I think this model could work even for non-DevOps deliveries too – testing can add value everywhere and there’s more to testing than gatekeeping.

Though if you are not clear of when in the SDLC/pipeline to apply the different testing approaches – you need to have a discussion around value and visibility on hand, and relevance of test activities on the other. That discussion could also help elaborate the boxes on the above visualization.

Another way to go about it could be to make a Wardley map

A Wardley map is an illustration that has the characteristics of a landscape map, and can be used for orientation in an IT setting. Wardley maps have two dimensions: Visibility and Evolution.

Visibility to the stakeholder could be business need or perceived value, as used in “No Code, No Test“. Evolution is mostly about the relative position from unknown/uncharted to embedded/industrialized. For instance looking at IT systems, it matters how evolved the system under test is:

It seems obvious that the more novel the SUT is there more exploratory testing is relevant, and similarly the more standardized the stack is the more continuous testing is valuable. …. Relatively valuable, is probably the better wording. Relative position of the elements is a key output of Wardley maps. (And more about the relative relationship between ET and CT later)

Add more exploratory testing to uncharted systems

So first of all we can position the test activities based on characteristics of the underlying IT structure. Secondly, as characteristics change, we can map the visibility and evolution of each testing activity. Continuous Testing, it self, can be made more or less visible and more or less embedded and industrialized.

As mentioned with ET and CT, we can now use the map to discuss why we need both CT and ET for a specific project. Continuous Testing relates upwards in the value chain to continuous Delivery, while exploratory testing ties more into a more visible end user goal of building the right thing, especially in a context of implicit and tacit knowledge.

To conclude on my colleagues question – to plan a test strategy we need to understand the pipeline, the relative value of the test activities and the relative evolution of the test activities.

[Wardley Mapping by Simon Wardley; template by @HiredThought]

Take Time for Time Off

It’s important to find time to log off – completely. I used to say it’s been a good vacation, if you forgot your company password. But these days it takes a bit more to wind down.

  • Go somewhere else than home
  • Leave your smart watch
  • Take of the neck chain that helps with computer neck pain
  • Make “ipad time slots” for checking news and SoMe – goes for the adults too

Have days where you forget what week day it is – and what time it is exactly. Right out there past the trees is a wonderful beach. Read fiction books and comics, explore the area, go swimming even if it’s a bit cold. And follow my rule of summer: have an ice cream a day

Keep the work bosses, game bosses and bad thoughts, at bay – with an ice cream a day.

Test Bash Manchester 2020

“How to coach subject matter experts to do testing” link post:

Talk replay included in Ministry of Testing Pro subscription.

Previously scheduled for Test Bash Brighton – full text here.

Register for Test Bash Manchester 2020 Online

For RPA to succeed …

TLDR: For RPA tools to succeed for automation you need to add engineering practices.

BTW: All automation projects require good software engineering practices.

Yes, you can use RPA tools to do test automation. And yes, you can have business persons designing the flows. BUT in order to succeed you need to apply some software engineering practices.

Robot Process Automation (RPA) tools like tools like LeapWork , Blue Prism  and UiPath  can be used to build business automation – it’s their core job. It’s the enterprise version of the Apple iOS shortcuts, “If This, Then That” or home automation. And they can be used for test automation – in some contexts.

The RPA tools are interesting because they seem to have a low barrier of entry. Some let you design the flows and robots visually, others using flowcharts. Either way it’s a low-code way of developing solutions. It seems compelling to let business end users prepare the flows and bots, it’s all plug & play.

Until it’s not.

Until the RPA robots design is only kept in the head of one person or until the flows/bots is a mess of interdependencies and cross links, that somehow the business still relies heavily on. And you have yet again created a spaghetti solution to add to the pile…

via DevHumor

The bigger the visual flowcharts in your RPA designer the more the project is a coding project. And you need to apply software development practices like version control, BDD style documentation and designing solutions loosely coupled and highly cohesive.

Sometimes you can teach the people the details, sometimes you can have a guide (tool smith) to enable them – sometimes it’s best to let the engineers tackle the automation. That depends on your gameplay and map of the world. What end users are good at is intrinsic domain knowledge the novel and the uncharted – not coding nor town planning.

Loosely coupled and highly cohesive.

The terms loosely coupled and highly cohesive is from the world of Computer Science, and most examples are around coding in object oriented languages like Java. In java, and in the flowcharts and visual scripts of RPA tools, you can group things into reusable “building bricks” or functions or sub flows. The Computer Science word for this is encapsulation.

It turns out it’s important how you group the bricks, and what the inputs and outputs are. Vladimir Khorikov summarizes with the following (my emphasis):

  • Cohesion represents the degree to which a part of a code base forms a logically single, atomic unit.
  • Coupling represents the degree to which a single unit is independent from others.
  • It’s impossible to achieve full decoupling without damaging cohesion, and vise versa.
  • Try to adhere to the “high cohesion and low coupling” guideline on all levels of your code base.
  • Don’t fall into the trap of destructive decoupling.

Without good engineering practices any automation initiative is, at best, just a smartass expensive one trick pony.