Focus on Goals over Risks

Looking into the discussion on what goes into a Test Plan and what goes into a Test Strategy – it’s my personal opinion that we can improve our business alignment. Risk-based testing and Product Risk Analysis have been around for long – but better models have emerged to address what will be more impactful.

I find the risk analysis I participate it – predictable. All projects comes with at least three risks:

1. We run out of time
2. Our staffing is not sufficient
3. We find more issues than we expected

What can we do about these as a team – not much. We can ask management to give us time and focused staff. And thus it comes down to which goals management have and what bets they are willing to make. Over the years I have been on big transformational programs where the deadline was less important because it was the CIO’s #¤! that was on the line.

Similarly when we try to perform a risk analysis of the system under test. We sit around the table and discuss for each item X two seemingly random numbers I and P, that we multiply and then base the testing effort on. At best it’s predicting the unknown, at worst it’s gameable. Who ever is in the discussion and has the most influence can sway the numbers simply by insisting.

The Dangerous Animals of Product Management by Product Board

I have seen individual contributors insisting so hard and having so much informal power that everything was equally important and everything had to be tested to the highest degree – as pr. understanding of the risk-based testing approach: The higher risk – the more testing… and thus the the risk analysis, was all a lot of work for nothing really.

What to do instead

Analyze your challenge based on Cynefin. Read Liz Keogh’s awesome primer and consider:

“Who’s done this before?”

5. Nobody’s done this before.

4. Someone’s done this, but not in this context.

3. Someone in our organization has done this (or we have access to expertise some other way).

2. Someone in our team’s done this.

1. We all know how to do this.

Cynefin for Everyone! by Liz Keogh

While still “fuzzy” I find the questions more objective than random numbers picked out of the air. Secondly Cynefin has a method to acting according to the problem space, whether is “act–sense–respond” or “sense–categorize–respond”.

Understand the business goals

Look at the business case for the project or the business goals driving the imitative. Recently I participated in a €50 million bid that had the following business goals – in prioritized order.

1. Stable operations
2. High end-user usability
3. Security
4. Features and functionality

Notice how stable operations are prioritized over features. We would fail to align with the business goals if all we could focus on would be functional testing. So while operations, security and usability might not be our usual game – we need to focus on being able to test all the things.

Test ALL the things

Lastly, my experience is that you get more organizational traction by aligning with goals rather than risks and issues. It’s a behavioral trick simply to talk about the thing you want to give attention. And at the end of the day the CIO wants to talk about goals rather than risks.