Factor in the Ripple Effects

TL;DR: Investing in basic tooling and automation improves your team besides expected metrics.

I work mostly with the implementation of enterprise SaaS systems these days. Large global companies are consolidating custom-built applications and on-premise applications with web-based standard solutions in the cloud aiming for “one standardized source of information to enable digital transformation”.

Yet the testing tooling hasn’t caught up. One company with €5000 million in sales is still using Word documents for test cases and “party like it’s 1999“. They are reluctantly considering tooling to support more agile ways of working. The whole “automate the knowns-knowns” is still pending an evaluation of return on investment (ROI) into technology from 2015. As of writing, Anno Domini 2022.

Assumptions

  • Writing test cases in documents takes about as long as writing automation
  • Maintaining automation is a more explicit task, humans can more easily apply a bit of fuzziness
  • When automation is in place, the execution requires limited efforts to run
  • The alternative to automated test execution is hours of people following and filling out the documents

With the investment in the tool, there’s a break-even around XX hours of document-based testing a month. That is if we plan for more than XX hours of document-based testing a month, the investment pays off. Your Mileage May Vary

But there’s more to it

First of all, when automated test execution is at limited costs to run and it can run independently at night, you will get the same effects as Continuous Integration and nightly builds have had in software development: you tend to run them more and more often.

This enables faster feedback both with regards to confirming new features and sums up to more effective regression testing. I have seen this happen in both custom application development and configuration of web-based standard solutions. In one project where I added automation, we have run nearly 8000 automated runs in a year (and 200 SME-based). We actually run the tests more often, and we cover the important things every day – and everything often enough. We do in fact get more testing, and broader coverage than any document-supported testing could ever scale to do.

Believe the experts

While there is some vendor basis in the following two webinars, the story is the same: Test automation can accelerate IT deliveries:

Alternatively, look into the research from Accelerate – and the DevOps handbooks. The ripple effects of automated test execution are plenty and go beyond the math of the testing effort. One thing to keep in mind is that test automation itself is not enough. At first, you need transformational leadership.

#263: There is a Model for your Trouble

Often directors, managers and other decision makers talks about an advanced challenge they have: What can we automate, who should automate and what tools to pick. There more and more I listen, the more I hear – they have not applied any models of the problem at hand. And there usually is a model of the problem space already. Any old model is preferable over no model at all. But it can be hard to see in the middle of chaos.

Continue reading

Calculating Time To Information

The key metric for any knowledge work – IT deliveries and testing in particular – is more than Mean Time to Repair (MTTR). While fixing fast matters – timing is everything. Timing in getting information to the people who needs it to make decisions. It’s no use if you can turn the ship around on a plate now, if you needed it yesterday. Key elements in calculating time to information is how far away the information is and how evolved the information is.

Continue reading

Imagine That Things Can Be Different

One of the key skills of a knowledge worker – and testing people are knowledge workers – is to imagine that things can be different. I have written previously how to recruit for curiosity – and contributed to the book of “21st-century skills for testers“. But apparently I have missed to highlight the key skill of imagining that things can be different.

Continue reading

Darlings, Pets, Cattle and GUID’s

Kill your darlings and treat your tests more like cattle than pets, are among some of the heuristics currently around for managing your environments and automation test suites. These heuristics tells me that the environments and automation are in a state of product or even commodity, while previously the tests and environments where like darlings and pets – named and nurtured.

Continue reading

It’s a Model – not the Truth

Usually when we discuss Observability, Testability, Modern Testing Principles, it should be with the disclaimer from what context the lessons originate and that Your Mileage Will Vary.

There are so many different IT projects out there – that assuming every IT project is about source code is quite a blind spot. Projects that deal with commercial standard systems or outsourced software might have source code underneath – but many teams does not have access to the source. Additionally, many legacy systems from the 90’es and older does not have the same automation capabilities as we have now.

Not all software projects are about consumer facing native apps and websites. While they are numerous there’s still plenty of systems out there for internal and business-to-business use. While the trends from CI/CD are picking up for B2B and internal systems, things doesn’t move so fast.

Continue reading

Go read Accelerate!

ACCELERATE – The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations

by NICOLE FORSGREN, JEZ HUMBLE, GENE KIM

The authors have “multiple examples of applying these practices within mainframe environments, traditional packaged software application delivery teams, and product teams“. It’s not just for business-to-consumer web-based organizations.

The book is a tour the force into software delivery performance – the research and statistics shows a clear correlation from DevOps and Lean principles to high achieving organisations. Every arrow on the below model is backed with research. Read the arrow clearly as “drives”, “improves” and “lead to”. E.g. Continuous Delivery leads to Less Burn out.

Saved you a click: https://itrevolution.com/book/accelerate/

A last thing to highlight: High performing organisations have lower manual work percentages in areas like: configuration management, testing, deployments and in the (ITIL) change approval process.

So – if you want to increase the boxes on the right, go do the stuff on the left.

Read the book and act on it.