Where I Have Changed

The Ministry of Testing Bloggers Club suggested that I write a post based on “In testing, I have changed my mind about ________”. As this blog dates back to 2012 with consistent (220) articles about testing, and my career in the field dates back to 2002 – it seems a 20-year experience should give me a few things. Testing is still not dead – and it’s still about the context (lower-case context, not CDT).

It’s not about: Testers being the only ones doing Testing

yeah, not so much these days. Testing is an act that any role can do in context. It’s about the testing – not so much the testers. And I have realized that even classic test management tasks can be done by someone else. Testing is not owned by the testers – it might be stewarded/facilitated by us, but it’s performed by a team member (who could be a tester).

It’s not about: Perfect Requirements

After decades in IT, it’s clear that even requirements are never perfect. When we look closer we see the business requirements can vary from a profound idea to a rudimentary feature of the system under test. Even in regulated industries requirements can be both about a specific configuration in a SaaS system or a loose idea of a relevant dashboard. Sometimes a requirement can be by design of an underlying commodity product – there doesn’t need to be a test case for everything.

The more rigor you add to the requirements management – the more fragile it becomes. It’s key to understand the risks and bets of the person paying for your solution. – in that lies the true borders of the delivery. Much can go informally along if it aligns with stakeholder values.

It’s not about: Defects

Back in the day defects needed to be accounted for, tracked, and distributed. Besides testing documents – defects were the only tangible delivery of the testers. The defects needed to be raised and closed. I recently wrote a guideline that stated that only observations that couldn’t be fixed within a day should be raised to the project manager for shared handling. In that context fixing things is within the same team. If it’s for another team to fix, defects are simply something communicated between the teams (check team topologies for team interactions). Sure you can still find a blocker or a P1 – what matters is how fast you can fix things.

It’s not about: Month-Long Testing Phases

The more time there is from idea to implementation – the more the requirement risks not addressing an up-to-date business objective. Timing is key. Some tools provide epics and user stories – but the structure is often misused to be a simple work aggregation – and not goal aligned.

The counter-intuitive trick is not to add formality, and more time between releases – but less. Less time between feedback between idea and implementation, and less time between implementation and test. Less time between the various forms of feedback adds to better results faster.

It still happens, I’m sure, that a business needs a month-long testing phase before a release; having a range of business staff to participate in testing the latest release of the enterprise ERP or CRM. More often the testing phase is one sprint behind the development activity. I have pondered this a lot.

At best testing is an integrated activity in the team and in the sprint. But if testing is a more separate activity – it can be both agile and context-relevant. So I have changed my mind about this anti-pattern.

It’s ok for testing to be in the next sprint –

if that adds consistency and less stress to the team*

* Seperate boards needed. Your Mileage Will Vary

Low-code – the Bigger Picture

Low-code test automation is part of a bigger trend in IT. Forresters and Gartner call it the “Citizen Developer” – the general idea that many business activities can be achieved by business users and citizens directly without the need for big IT projects… initially.

For the last 10 years we have mockingly called Robot Process Automation (RPA) a “poor man’s integration”, in the sense that instead of building sic “real” integration, we build an RPA robot to handle the interface. But it’s equally low-code when your Apple Shortcuts trigger application events or you use Airtables or SmartSheets instead of MS Office tools.

In the mocking from IT teams, we do tend to forget that low-code tools are a short-term efficient and user-friendly way for organizations without a big IT budget to solve some common problems. That it can very well make business sense to RPA data between systems until the last silo has been cemented over.

There is a clear trend that the business units of large enterprises are getting more tech-savvy and can do more IT things on their own: order a new OS, configure a new form, populate tables, and configure collaborative work products. Previously these actions would have mockingly been called “shadow IT” when outside the realm of the IT units. Now it’s more out in the open – and where the IT spend money is.

Low code is, when you squint at it, all about the visualization and abstraction of something that previously took coding in IDEs, tinkering in Excel sheets, and similarly skilled IT labor to configure. It’s really nothing new in the history of IT. For large global enterprise companies, it has always been about consolidating business IT systems and redefining new coherent ways of working.

Replace the existing system suite of 10+ tools with one Software-as-a-Service Solution to create and maintain product information, so that it can be kept in one place and inspire additional digital transformation.

The strategic objective for a large global company in 2022

The current journey for these global enterprises is to move the IT savvy-ness into the business units and make the business units more autonomous in their IT spending. There’s no need to hire an external outsourcing company to maintain the IT operations when most can be done by a few internal staff inside the cloud dashboards or similar admin modules of Salesforce and ServiceNow.

Give it some time, though.

While low-code and RPA can in some cases be effectively coded by business experts – they will soon need some good old computer science techniques to maintain the RPA and low-code shoe strings. At the end of the day, visual code is still code. And low-code test automation is just part of a bigger picture.

Trends for the Tester Role

YMWV – this is a model for reflection not to a 1:1 scale of everything in the universe. It might be useful.

The space for the testing professional is under pressure – for my own role and even more for the “traditional” testing professional. At least since 2017 there has been a shift and ongoing disruption. I finally have a form to visualize some of the trends that puts the role of the tester under pressure:

  • SIT / UAT debate
  • Low-code trend
  • Modern Testing
  • Quality Engineering and whole team approach

I still see two key areas (stars below) for the classic tester to move into: exploratory testing based on weak signals and supporting the end-users low-code activities (test tool smith). For the more managerial and coordinating role I will have to get back to you in a future blog post.

Continue reading

The Testing, not the Testers

Occasionally I see posts and discussions, where testers are indignated that this and this company has no testers. How could they! Or similarly when a product is released publicly with significant issues: See – it’s because they have no testers! Or that the testers are not taken seriously. OMG!

Continue reading

Someone else will do it

The testing activity has been under change for long. And it’s clear that the testing activity has shifted. Even the test managers have to re-calibrate – as other roles will be doing the test management activity. Be prepared, as someone else will do your testing job. Work on building self-reliance in others and be prepared to hand-over what you can do.

There is more to testing than testing specialists punching test cases. The testing activity as such, has shifted (both left and right), and testing is being done by more roles than “testing people”. Depending on the context, the explicit testing activity is done by a mix of developers, testing specialists, end users and others.

I often find myself as the only testing person on the project. The testing activity is done by automation specialists and end users in one project, and by technical operations staff and end users in another. In these projects either the technology or the business knowledge is paramount, and not so much exploration, flaws and edge cases for specialized testers to explore.

me, 2020. YMMV

Similarly for the test managers – there’s a trend/shift, that sometimes the test management activity is shifting away from the test managers. Even to me – even if I’m sometimes more an a “project manager of the testing activity“, a “Test coach” or similar. The trend is already there – coined sometimes as “whole team approach to quality“. Yes, most of the test management activity can be done by scrum masters, Release Train Engineers and even project managers ….

Recently I was asked to assist a large transition project for a holding company with many brands. Each brand had their own applications and technology stack, but the holding company had decided to move the hosting. So the holding company’s Project Management Office (PMO) was put in charge of facilitating the brand’s testing activities – an activity they had never considered nor done before. My role would only be to provide guidance, not do the actual facilitation.

Which got me thinking….

And after some deep thinking. – I do have the privilege to be able to adapt. I don’t need to hoard knowledge or make power moves (anymore) or worry about health-coverage or any of the lower Maslow pyramid terms (anymore).

It’s very natural for me to hand over project approaches to my co-workers. I’m often on the “blue team” to outline the strategy, My best field of work is to bring clarity and consistency, not scalability or repeatability to the practise.

I naturally hand over learning anyways, so why not re-calibrate when the thing I do has reached a stage, where it’s repeatable. And then focus on building the skills in others, work myself out of the test management role as we know it.

And don’t worry that someone else will eventually do my (testing and test management) job. The first step is to acknowledge the trend/pattern, second to redefine and bring clarity! Let’s explore and see what we find!

Someone else will do the building, not Emmet. His task is repeatable.
Someone else will do the building, not Emmet. Their task is repeatable.

Shifting is more than Shift Left

Links on the topic: Shifting is more than Shift Left as presented at the OnlineTestConf Fall 2017.

My own writings:

by @KatjaBudnikov #katjasays
by @KatjaBudnikov #katjasays

OpsDev – more dev work by ops

The hyped mnemonic “DevOps” is equally true the other way around: OpsDev – that is, more and more work in the operations and infrastructure departments happens as development activities with scripts, code repositories and build managers. OpsDev is as tool-heavy as DevOps, and test involvement similarly pipeline focussed.

Guest blog post at http://www.plutora.com/blog/opsdev-test-environments-management 

Teaser for Online Test Conf 2017

I’m speaking at the Fall 2017 Online Test Conference on the topic: Shifting is more than Shift Left 

Change is happening to the testing activities. Shift-left automates and codifies the testing activities. Shift-right does it for production.

This session will be about a couple of other trends, changes and shifts that’s happening to testers and test managers.

– Shift-Coach, where It’s more about coaching teams.

– Shift-SME, where it’s more about business savvy.

– Shift-Deliver, where it’s more about the road to production



Less Test Managers, More Test Coaches

One of the trends/shifts I experience in testing & test management in particular is the Test Coach as discussed initially here: The Shift-Coach Testing Trend (Oct, 2016). Recently (Aug 2017) it came up again in a Twitter thread, where Stephen Janaway stated the inspiration to the title of this blog post.

Less Test Managers and more coaches. That’s how I see it going.

Fittingly as they inspired the first post with the talk “How I Lost My Job As a Test Manager” presented at Test Bash 2015. This post is a further elaboration of the Shift-Coach test management trend. Here are some of my experiences:

  • I have been assigned to an agile development team to introduce them to 3 Amigos, Test data driven test automation and such things. The purpose of my involvement was to enable the team to continue the practices without me, and without testers besides the business analyst / product owner (See The domain expert is the tester) as they are doing Shift-left. Similar to an agile or scrum coach, my approach was to look at it as a change in the way of working.
  • Another project is an infrastructure project, there are no testers only technicians configuring Cisco routers that by software can replace firewalls, iron ports, VM servers and other network equipment. The project has to implement 80+ of these, so I setup both a test process and an ITIL change request process acting as a test and release manager – another quite frequent trend. I could continue in the project for the duration, but instead I setup guidance and leave when it’s sufficiently in place.

This might be similar to a test architect, a (internal) test consultant activity. It has nothing to do with diminishing testing. Rather I see it as more testing happening, something that would not have been done without the coaching from a test manager. It’s all about finding a test approach that is fit for the context.

Here are some things others have written:

The competence of the test coach is to have enough change management expertise (people skills) and test management expertise (domain skills) to know how to coach and facilitate the change. Should test coaches test too, perhaps when required, but not necessarily. The activity is primarily to up-skill the team to continue on their own.

The “Test Coach” is a trend similar to “shift-left” and all the other shifts in testing and test management. I see it as a pattern, and what I read from the threads and discussions is that many test managers gradually shift towards test coaches.

2017-07-03 13.57.42

The domain expert is the tester

Sometimes the best tester is the domain expert, the person that knows all the in’s and out’s and corners of the system. I have worked with testers that have had hands-on on a system since the late 1970’es, but I also know testers of mobile app’s that marvel in being the subject matter expert of the domain. Sometimes the professional tester doubles as agile Product Owner(1) too given their vast knowledge. The tester becomes the the SME …

The subject matter expert, though, is usually a business analyst, or perhaps a User Experience expert. Those persons might have a better stand to be testing the system, than testers with no prior knowledge. Often the SME is the best tester available. I see this happening in a shift-left setting – but also in settings with a heavy user and business involvement. Like SAP releases to enterprise systems – where the business users and SAP architects still spend a month off their “actual” work (user acceptance) testing corporate configurations and customization.

The UAT is not dead, but the classic role of the tester testing on behalf of the business is declining. The business would rather test their own, with in-house subject matter experts. The field is active, as there is tool support for this activity. Panaya(2) is a tool that specializes in managing the UAT of a corporate system like SAP, and one of the key elements is that test cases can be broken up in steps and handed over between persons. Not even classic HP ALM’s handle hand-over between testers well. While ALM’s support that the tester does the testing, Panaya supports that tests are distributed across many people. People that have other (“real business”) tasks during the work day.

Testing can also be pushed even further out to the users with crowd-based testing, beta releases etc. In both crowd-based and UAT-based testing, the role of the pro’ tester is missing but the testing is still happening. IT’s being done by the most skilled – most valuable for the task.

So what can we as testers do when our tasks are gone – skill up, go with the change and become the expert – or move out to other skills: Coaches, delivery leads etc.

2015-12-18 13.49.28
The expert says 5 pieces should be in the build – though the customer is OK.

  1. If they double as scrum master, she’s probably more a Test Coach
  2. this post is not sponsored. I’m just making observations – not recommendations.