Factor in the Ripple Effects

TL;DR: Investing in basic tooling and automation improves your team besides expected metrics.

I work mostly with the implementation of enterprise SaaS systems these days. Large global companies are consolidating custom-built applications and on-premise applications with web-based standard solutions in the cloud aiming for “one standardized source of information to enable digital transformation”.

Yet the testing tooling hasn’t caught up. One company with €5000 million in sales is still using Word documents for test cases and “party like it’s 1999“. They are reluctantly considering tooling to support more agile ways of working. The whole “automate the knowns-knowns” is still pending an evaluation of return on investment (ROI) into technology from 2015. As of writing, Anno Domini 2022.

Assumptions

• Writing test cases in documents takes about as long as writing automation
• Maintaining automation is a more explicit task, humans can more easily apply a bit of fuzziness
• When automation is in place, the execution requires limited efforts to run
• The alternative to automated test execution is hours of people following and filling out the documents

With the investment in the tool, there’s a break-even around XX hours of document-based testing a month. That is if we plan for more than XX hours of document-based testing a month, the investment pays off. Your Mileage May Vary

But there’s more to it

First of all, when automated test execution is at limited costs to run and it can run independently at night, you will get the same effects as Continuous Integration and nightly builds have had in software development: you tend to run them more and more often.

This enables faster feedback both with regards to confirming new features and sums up to more effective regression testing. I have seen this happen in both custom application development and configuration of web-based standard solutions. In one project where I added automation, we have run nearly 8000 automated runs in a year (and 200 SME-based). We actually run the tests more often, and we cover the important things every day – and everything often enough. We do in fact get more testing, and broader coverage than any document-supported testing could ever scale to do.

Believe the experts

While there is some vendor basis in the following two webinars, the story is the same: Test automation can accelerate IT deliveries:

• We Must Address Testing in Low-Code Development
• Best Practices: Measuring the Business Value of Test Automation

Alternatively, look into the research from Accelerate – and the DevOps handbooks. The ripple effects of automated test execution are plenty and go beyond the math of the testing effort. One thing to keep in mind is that test automation itself is not enough. At first, you need transformational leadership.