Applying Accelerate as a Strategy Legend

TL;DR: What I learned from applying the research from the book Accelerate (2018) by Forsgren et.al. as a strategy legend on my Wardley Strategy Map. Spoiler alert – it’s more about people than technology.

I use Wardely Maps to read my projects’ context and identify a test strategy that aligns with stakeholder needs. Currently, I am doing yet another project – this one is primarily based on Infrastructure as Code (IaC) and Functions as Services (FaaS) and creating an API engine/event-driven broker. My usual starting point was my Situational Aware Test Plan template – below with the headings from my DPCG mnemonic.

	Darlings	Pets	Cattle	GUID’s
Situation	X
Delivery / Release frequency			X
– Staffing	X
– – Collaboration	X
– Environments			X
– – Tools				X
– Test scope / Verification			X
– – Test Data		X

This is not a map

This, of course, is not a map, as I have not considered the user need, but more given each element a score based on my initial reading of the project. It’s a new project team that hasn’t worked together before – building a new system, that does not yet exist similarly. That we know of – so it’s Cynefin Complex domain. The tech stack and testing challenges are well-known and can be mostly automated.

There is probably a natural order to the items above: It’s a new situation (for the customer too) and a release is needed. the delivery needs staffing – staffing needs collaboration and tools. Delivery (releases) needs environments and verification and verification needs test data. The table is neither a strategy as I have not considered which items to move – but I had an idea.

Research From Accelerate

ACCELERATE – The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations by NICOLE FORSGREN, JEZ HUMBLE, and GENE KIM (2018) identified a key set of practices to optimize for organizational performance and a great delivery culture:

Saved you a click: https://itrevolution.com/book/accelerate/

If we zoom in, we can see the prerequisites for Continuous Delivery to be successful:

• Test Automation
• Deployment Automation
• Trunk-based Development
• Shift Left on security
• Loosely Coupled Architecture
• Empowered teams
• Continuous Integration
• Version Control
• Test Data management
• Monitoring
• Proactive system notifications

These items are just in the order from the book – but I can similarly score our ability and tools. Because of the platform, most are a commodity or best practices: Loosely Coupled Architecture, version control, CI/CD, trunk-based development, and deployment automation. A few require a bit of a push from possibility to active use (proactive system notifications, monitoring). The sore spots are (as detailed in 24 Key Capabilities to Drive Improvement in Software Delivery):

• for the development team to own test automation: Test automation is a practice where software tests are run automatically (not manually) continuously throughout the development process. Effective test suites are reliable—that is, tests find real failures and only pass releasable code. Note that developers should be primarily responsible for the creation and maintenance of automated test suites. 
• establish test data management: Test data requires careful maintenance, and test data management is becoming an increasingly important part of automated testing. Effective practices include having adequate data to run your test suite, the ability to acquire necessary data on demand, the ability to condition your test data in your pipeline, and the data not limiting the number of tests you can run. We do caution, however, that teams should minimize, whenever possible, the amount of test data needed to run automated tests. 
• Empowered teams: Research shows that teams that can choose which tools to use do better at continuous delivery and, in turn, drive better software development and delivery performance. No one knows better than practitioners what they need to be effective.

The Test Strategy

Hence my focus as a test manager would obviously be the first two. Let me be the first to move for test automation by the developers (the domain for the test is mostly API endpoints) and the first to ask for killing the test environment so that we can learn to bring it up again – and again.

But because I care – and because I have read in the book what is a requirement for the listed team capabilities above – I should be mindful to consider also empowering teams and encouraging transformational leadership: Vision, Inspirational Communication, Supportive Leadership, Intellectual Stimulation, and Personal Recognition.

Using the research from Accelerate as a mapping legend for the team’s Continuous Delivery Practices I can see that even with state-of-the-art tooling and a modern high-level stack – it’s a people problem first of all – and thus the test strategy is threefold:

1. Encourage testing by the developers (evolve test automation)
2. Prioritize the construction of stubs and drivers for test data (evolve test data)
3. Encourage collaboration, learning, and a supporting culture (evolve culture)

Calculating Time To Information

The key metric for any knowledge work – IT deliveries and testing in particular – is more than Mean Time to Repair (MTTR). While fixing fast matters – timing is everything. Timing in getting information to the people who needs it to make decisions. It’s no use if you can turn the ship around on a plate now, if you needed it yesterday. Key elements in calculating time to information is how far away the information is and how evolved the information is.

Focus on Goals over Risks

Looking into the discussion on what goes into a Test Plan and what goes into a Test Strategy – it’s my personal opinion that we can improve our business alignment. Risk-based testing and Product Risk Analysis have been around for long – but better models have emerged to address what will be more impactful.

Relations – are about half of IT

You can’t have IT projects without relations. Relations matter more than it seem.

Recently I had an online call with the adults around the 14yo: An autism psychiatrist, the school guide/trainer, their mom and I. Online on teams, obviously. The other adults in the call were struggling with all the online schooling and interaction, and no face-to-face with the young student. Besides all the stress of the pandemic they had a hard time working at all.

To me online meetings are a natural way of working. I have been working primarily through online interactions the last five years or so. The people I work with are distributed away from me, they are in Copenhagen, China, Philippines and other places. It seems the teaching industry broadly has to catch up with the lessons and tricks from the IT industry. Besides the stress of the pandemic and lockdowns IT people can actually do their work. So I mentioned that I only work online.

And the psychiatrist said, but for IT people relationships aren’t so important.

… oh …

Relations matter in IT projects

The psychiatrist probably doesn’t know the inside of an IT project, and are probably only aware of the City IT staff when something is not working or they are rolling out yet another upgrade of their systems. So to them it seems that IT is all about the deliveries – not so much about the relationships. (to bad really)

On the inside of IT projects, relationships are about half of the story – the actual IT thing is the other half. For the clear and simple things relations matter less (the commoditized, as Wardley would point out). When you think about it, IT have a huge range of models and frameworks around the relationship part: Cynefin, Agile Manifesto, Scrum, Team Topologies etc. The more you move around on Cynefin the more relations matters.

3D version from Rob England Cognitive Edge

Relations are obviously important to line managers, coaches, lead roles and people in enablement roles. But for the individual contributors in the teams – relations matter too. First of all, time has run out for the expert douche bags and hero culture. Everyone talks about the whole team approach to quality.

Secondly the research from the Accelerate is clear – besides all the mechanics of delivering IT – the culture of the workplace impacts the business performance. Lastly, when you look at incentives for the individual, the non-monetary matters are becoming a key differentiator. Especially given the pandemic – gone are the bro culture of foosball tables. What’s hot is work from home, team diversity, moral practices and a decent team culture.

There will be a trend in the future, that changes the IT office landscape yet again. The best teams will meet online for some of the work – and join in collaboration spaces or getaways for alignment. Because even for IT work relationships matter – and at the end of the pandemic even we will adjourn in our teams.

To Transform Testing

There is no doubt that our long lived testing narrative is under pressure. To continue to bring business value, both the IT field and testing is transforming to be about proactive business enabling.

The IT domain is currently buzzing with the word “IT transformation” – the idea that IT services should be more about “inspire, challenge and transform the digital businesses“. That it should be less about delivering IT products and artifacts and more about enabling digital business outcomes. Even for testing – it should be less about a product/service, and more about business necessity.

Stop focusing on the things that bug people, and dare to do both IT and testing smarter and more business focused from the start. Build Quality in – smarter from start. That goes for IT services as a whole, and definitely also for the testing activities.

What can you do to transform your testing? I have three areas:

Discuss Business Strategy

Learn Wardley mapping – and use it like Chris McDermott to create context specific maturity models with Wardley Maps informed by Cynefin. Use the mapping to Broaden the scope of the system under test.

Align with the Business Strategy

Leading Quality [Cummings-John, Peer 2019] has a whole chapter on “Align your team to your company growth metric“. Consider if the company you work for is driven by Attention, Transaction or Productivity metrics, and arrange your test activities accordingly.

Dare to Deliver in New Ways

We are usually talk so much about optimizing the (IT and testing) delivery, that we forget other ways to be innovative and provide business enabling. One way could be to dare use new technology like RPA or a HoloLens to support tedious tasks in testing – to use an existing product to something new. Another approach to actually test “all the things” that matter or to apply testing to IT outside the realm of application delivery.

To Transform Testing I will discuss, align and dare so that test solutions can be proactive business enablers – (not only achieve shippable quality).

Mapping Mondays – Pioneers, Settlers, Town Planners

Test ALL the things

TL;DR: We can add testing to all requirements and all business risks. Testing to document requirements and to debunk risks provides valuable information for the business. Let us not limit testing to things that can be coded. The intellectual activity of trial and learning is happening anyways, we might as well pitch in with ways to find important evidence for the decision makers.

Test all the requirements

Traditionally testing was all about testing the functional requirements that could be coded. Non-functional requirements was left for the specialists, or plainly disregarded. I know I have done my share of test planning, with a range of requirements left “N/A” with regards to testing. Especially performance scope, batch jobs, hardware specs, data base table expansions and virus scanning have been left out of my functional test plans…

When I look at a list of requirements now – I see that we can indeed test all the things, or we can at least work on how to document that the requirement is fulfilled. Some of the requirements are actually quite easy to document. If it’s on a screen somewhere, take a screen shot and attach it to a simple test case. Done deal really. Additionally with a testing mind-set I can think of ways to challenge the details. But do we really, really need to fill up a disk to establish if it’s exactly a 1 Gb allocation – probably not. Do we really really need to document all requirements – yes in some contracts/contexts it’s important for the customer to know that everything has indeed been established. Sometimes the customer doesn’t trust you otherwise, sometimes the tests are more about your ability to deliver and provide evidence that matters.

Test all the business risks

Look into the business case of your project and find the business risks. Sometimes they are explicitly stated and prioritized. A a recent Ministry of Testing Meetup we looked into a case for a large national health system. We looked at the tangible benefits, intangible benefits and on the scored business risks.  What worried the business and management most was budget, time and whether the new system would be used in a standardized way. There is an opportunity for testing here to help address, document and challenge the most important business  risks. Traditional testing would usually look at functional requirements that can be coded or configured, and miss totally to address what matters most to the business.

OK, how do we test the project costs? How about frequent checkpoints of expected spending – would that be similar to tracking test progress. Perhaps – let’s find out. Testing is all about asking questions for the stakeholders and solving the most important problems first. Can we help to analyse risks and setup mitigation activities – sure we can. We just have to step out of our traditional “software only bubble”.

Meme ALL the things

 

 

 

 

 

 

 

Read also: Many Bits under the Bridge, Less Software, more Testing, Test Criteria for Outsourced Software, The Expected, Fell in the trap of total coverage.

Links: “A Context-Driven Approach to Delivering Business Value”, Cynefin In Software Testing, Testing during Application Transition Trials

 

Less Software, more Testing

I rarely test software these days. I mostly lead testing of IT solutions.

Testing in the context of:

• Updating all corporate PC’s from windows 7/8 to Windows 10
• Consolidating network equipment from more devices to one box, on 80 global locations *
• Move 40 live business applications from one data center to another *
• Take over application maintenance for a specialized public organization
• Implement track and trace for pharma products from production to shops
• Migrate HR data for 2500 people from one system platform to another

Yes, it happens that I participate in a project that is about developing a new business application, but my activities are less about testing software and more about testing in IT solutions in general.

Mostly I manage test activities and describe testing in these contexts. My preferred way of working is in setting and implementing test strategies. I prefer complex and non-ordered projects (Complex and Chaotic – I’m looking at you), it fits well with my context-driven approach of finding the “test solution that fits the context”.

Testing is in it self a solution, that must solve a business problem. Great testing is all about providing information to the stakeholders. I don’t care especially if this is done by someone TESTING or a TESTER. It is my responsibility to setup the testing activities (information gathering) that supports the team, faces the business & technology and challenges the product “sufficiently“.

Sometimes “sufficiently” is merely confirming and going through the motions of explicit requirement coverage. This is a special challenge to me, as I know of many effective and Rapid approaches, that could add valuable information. When I face this challenge, I try to look at the full picture of the project, and what the business want’s to achieve.

The business of the business is business. What matters is not software or projects, but the solutions to the challenges the business have. And the context of testing is similarly so much more than the software.

*: As mentioned in “How to test in IT Operations” at Nordic Testing Days 2016.

The Expected

Many test processes and test tools mention that you have to establish the expected results to every test. It is at best risk ignorance not to take the notion of expected results with a kilo of salt #YMMV.

If you can establish the result of your solution in a deterministic, algorithmic and consistent way in a non-trivial problem, then you can solve the halting problem. But I doubt your requirements are trivial.. or even always right. Even the best business analyst or subject matter expert may be wrong. Your best oracle may fail too.  Or you may be focussing on only getting what you measure.

Nailed it

When working with validation in seemingly very controlled environments changes and rework happens a lot, as every new finding needs the document trail back to square one.. Stuff happens. Validation is not testing, but looking to show that the program did work as requested at some time. It is a race towards the lowest common denominator. IT suppliers can do better that just to look for “as designed” [1].

Still the Cynefin framework illustrates that there are repeatable and known projects, and in those contexts you should probably focus on looking to check as many as the simple binary questions in a tool supported way, and work on the open questions for your testing.

Speaking of open ends – every time I see an explicit expected result I tend to add the following disclaimer  (song to the tune of nothing else matters [2]):

And nothing odd happens … that I know of … on my machine, in this situation [3]

And odd is something that may harm my user, business or program result

Significantly…

But I’d rather skip this test step  and work on the description of the test and guidelines to mention:

• Think! Use your head – use what you know.
• You are the expert.
• If it smells fishy, there is something fishy going on
• Wonder, Wander, Galumph if you may
• Analyse, consider,.. if you get an idea, try it!
• Learn, Explore, and by all means test

See also: The unknown unknown unexpressed expectations, Eating wicked problems for breakfast

1: Anyone can beat us, unless we are the best, todays innovation becomes tomorrows commodity

2: https://www.youtube.com/watch?v=tAGnKpE4NCI 

3: I’ve heard that somewhere…