Continuous Customer Feedback

There are many ways for continuous customer feedback. In coding deliverables, you can demonstrate how the solution works so far and monitor the usage to get actual data on the end-user experience. It’s proven over and over again that faster feedback loops are essential to optimal deliverables – and to improve business performance. Go read Accelerate.. again! Here are some of my tricks for the non-coding deliverables as a staff-level testing consultant.

Provide Continuous Training

One task I have is to help others manage the testing activities. It’s usually people fresh out of Uni – as long as there is curiosity the rest can be learned. The way I usually do it is to have an intro session, break down the first few steps and then have recurring meetings to align and dig even further into the tasks. My outcome as a leader is to make them self-reliable in doing the tasks. The usual struggle is for me to sit on my hands and to pause more.

Simon Koerner suggests these five ways to lead when not in charge:

  • Be proactive
  • Motivate those around you
  • Look beyond your job role
  • Recognize others
  • Share your knowledge

Continuous Test Plan Alignment

I’m currently leading a testing activity to deliver some IT services to company A. There’s a huge contract that stipulates all the mechanics of delivering these services. And for each testing activity, there is a good old-fashioned test plan document that needs to be delivered and approved by company A. (sigh)

The old-fashioned way would be for me to prepare the document in detail and forward it formally. If you have to write too much down and debate the documents over and over – it might be an organizational maturity issue, but it’s most likely a people problem and a trust issue.

Recently I have started stating that the test plan will be presented continuously. At my weekly meetings with the Customer A test responsible persons I present the current draft of the document. This way they see it being developed step-by-step and we can align – and the approval will be a formality in the end. I prefer this way of working – but have also failed where the customer expected detailed hand-overs and I preferred to work entrepreneurially to set things in motion.

Limiting Contracts Replies to a Few Pages

When working with large IT services contracts it’s unfortunately customary that there are requirements for the content and headlines of test plans and other test documents. These content requirements are often straight out of old templates listing testing deliverables, testing types, and physical conditions for the testing activities. No test documents will be approved if there are sections missing, as the list items are contractual minimum requirements. No one wants to be non-compliant with a contract based on a missing headline in a document somewhere.

The contract authors and their lawyers would probably argue that it provides business value – at least in the sense of comparing competing companies submitting for the contract. That best-practice templates are market-leading. Yet, the case is over and over again that best practices will only give you standard solutions that are not tailored to you.

In the year 2022, it’s so awkward that you have to address physical rooms and equipment as a required element. Sure if it’s special in this case, by all means. But usually, it probably isn’t that important. Preferably we should make situational aware test plans – but if your content isn’t as innovative I recently wrote a reply in a big contract tender for customer B reply that solved this in a good way.

For the contract, customer B requested a maximum of 4 pages – where we had to elaborate on what we would advise the customer and our experience in doing so. See we don’t need 100 pages to convey a quality narrative. And based on our reply they could still evaluate and score us on how well we understood their situation.

Continuous customer feedback succeeds when we show that we understand the customer.

Factor in the Ripple Effects

TL;DR: Investing in basic tooling and automation improves your team besides expected metrics.

I work mostly with the implementation of enterprise SaaS systems these days. Large global companies are consolidating custom-built applications and on-premise applications with web-based standard solutions in the cloud aiming for “one standardized source of information to enable digital transformation”.

Yet the testing tooling hasn’t caught up. One company with €5000 million in sales is still using Word documents for test cases and “party like it’s 1999“. They are reluctantly considering tooling to support more agile ways of working. The whole “automate the knowns-knowns” is still pending an evaluation of return on investment (ROI) into technology from 2015. As of writing, Anno Domini 2022.

Assumptions

  • Writing test cases in documents takes about as long as writing automation
  • Maintaining automation is a more explicit task, humans can more easily apply a bit of fuzziness
  • When automation is in place, the execution requires limited efforts to run
  • The alternative to automated test execution is hours of people following and filling out the documents

With the investment in the tool, there’s a break-even around XX hours of document-based testing a month. That is if we plan for more than XX hours of document-based testing a month, the investment pays off. Your Mileage May Vary

But there’s more to it

First of all, when automated test execution is at limited costs to run and it can run independently at night, you will get the same effects as Continuous Integration and nightly builds have had in software development: you tend to run them more and more often.

This enables faster feedback both with regards to confirming new features and sums up to more effective regression testing. I have seen this happen in both custom application development and configuration of web-based standard solutions. In one project where I added automation, we have run nearly 8000 automated runs in a year (and 200 SME-based). We actually run the tests more often, and we cover the important things every day – and everything often enough. We do in fact get more testing, and broader coverage than any document-supported testing could ever scale to do.

Believe the experts

While there is some vendor basis in the following two webinars, the story is the same: Test automation can accelerate IT deliveries:

Alternatively, look into the research from Accelerate – and the DevOps handbooks. The ripple effects of automated test execution are plenty and go beyond the math of the testing effort. One thing to keep in mind is that test automation itself is not enough. At first, you need transformational leadership.

#263: There is a Model for your Trouble

Often directors, managers and other decision makers talks about an advanced challenge they have: What can we automate, who should automate and what tools to pick. There more and more I listen, the more I hear – they have not applied any models of the problem at hand. And there usually is a model of the problem space already. Any old model is preferable over no model at all. But it can be hard to see in the middle of chaos.

Continue reading

Calculating Time To Information

The key metric for any knowledge work – IT deliveries and testing in particular – is more than Mean Time to Repair (MTTR). While fixing fast matters – timing is everything. Timing in getting information to the people who needs it to make decisions. It’s no use if you can turn the ship around on a plate now, if you needed it yesterday. Key elements in calculating time to information is how far away the information is and how evolved the information is.

Continue reading

Imagine That Things Can Be Different

One of the key skills of a knowledge worker – and testing people are knowledge workers – is to imagine that things can be different. I have written previously how to recruit for curiosity – and contributed to the book of “21st-century skills for testers“. But apparently I have missed to highlight the key skill of imagining that things can be different.

Continue reading

Darlings, Pets, Cattle and GUID’s

Kill your darlings and treat your tests more like cattle than pets, are among some of the heuristics currently around for managing your environments and automation test suites. These heuristics tells me that the environments and automation are in a state of product or even commodity, while previously the tests and environments where like darlings and pets – named and nurtured.

Continue reading

It’s a Model – not the Truth

Usually when we discuss Observability, Testability, Modern Testing Principles, it should be with the disclaimer from what context the lessons originate and that Your Mileage Will Vary.

There are so many different IT projects out there – that assuming every IT project is about source code is quite a blind spot. Projects that deal with commercial standard systems or outsourced software might have source code underneath – but many teams does not have access to the source. Additionally, many legacy systems from the 90’es and older does not have the same automation capabilities as we have now.

Not all software projects are about consumer facing native apps and websites. While they are numerous there’s still plenty of systems out there for internal and business-to-business use. While the trends from CI/CD are picking up for B2B and internal systems, things doesn’t move so fast.

Continue reading

Go read Accelerate!

ACCELERATE – The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations

by NICOLE FORSGREN, JEZ HUMBLE, GENE KIM

The authors have “multiple examples of applying these practices within mainframe environments, traditional packaged software application delivery teams, and product teams“. It’s not just for business-to-consumer web-based organizations.

The book is a tour the force into software delivery performance – the research and statistics shows a clear correlation from DevOps and Lean principles to high achieving organisations. Every arrow on the below model is backed with research. Read the arrow clearly as “drives”, “improves” and “lead to”. E.g. Continuous Delivery leads to Less Burn out.

Saved you a click: https://itrevolution.com/book/accelerate/

A last thing to highlight: High performing organisations have lower manual work percentages in areas like: configuration management, testing, deployments and in the (ITIL) change approval process.

So – if you want to increase the boxes on the right, go do the stuff on the left.

Read the book and act on it.