In the recent years I have been working on projects with no dedicated testers but plenty of testing. The testing has primarily been performed by subject matter experts. This is where it gets interesting, as my role on these projects has been to lead the testing being performed by people that have limited experience in testing. They also have no desire to be testing specialists, after all they are already specialists in their own subjects, however, everyone agrees and insist that the testing needs doing. So how do we ensure that the testing being done is done well? After having worked on several very different projects, yet still with subject matter experts doing the testing, I have been able to get both the public process clerks and the technology specialists to perform excellent testing. This talk is about the approaches that I have found work well:
One of the approaches is for me to prepare the test cases and prepare them only as headlines. Sometimes preparing the tests as open questions helps too.
Another approach is to lead them as if they are doing the project participation voluntarily. They probably are, but still it helps to respect where they are coming from.
The lessons though (good and bad) is relevant to many testers in other situations, especially being the only “tester” on the team. The story applies equally to developers and business end users doing most of the testing and you will have them contributing with great testing in no time!
What you will know after the talk:
An understanding of how testing looks when done by subject matter experts
How to lead a testing activity with an appreciative and motivating style
Examples of how teams can do great testing without dedicated testers
As a Test Manager I oversee the testing in a project or program – I am usually the only testing specialist in the project, so, I need the right leadership skills and the right tools to succeed. I have to own the data about the testing and quality activities.
As the test manager I need to facilitate a quite a range of testing activities:
Business Analysts  or similar within the project
End users or end user representatives
I need to balance that I need to know what’s going on (with regards to testing) but without micromanaging the people being involved in testing and quality activities. My role is to facilitate that testing things happen – like the project manager making project things happen. I cannot own the activities without owning the data about it. I need to cover the full spectrum of tests – from engineered (RDA and CI/CD) to people-based (scripts and exploration).
The most practical tool for a test manager with this scope is PractiTest, as there is more to testing than just the test cases . The old term “ALM”  comes to mind – it is still relevant when I look for a full test management tool. I need to cover both the “inputs” to testing (requirements, tickets and user stories) and the “outputs” (bugs) in one location. I need the requirements and user stories in my tool, as I need to base my test analysis and planning on the delivery model (that may not always be agile). I need the bugs in the testing tool too, as bugs can happen in any work product of the project: documents, code base and even the tests. PractiTest acknowledge that there is more to IT projects than code.
I appreciate the key driver of PractiTest – that all activities happen in-flow. You don’t have to change window, stack pop-ups or go to another tool in order to run the tests or create bugs. Creating bugs happens in context of the test case and seamlessly moves all data about the run to the bug. Everything you need to do is context-based, and available to you on screen. And it has some cool features of read-only links to graphs for management reporting, and a smart built-in “rapid reporter” for exploratory testing notes.
It can be a challenge to switch to PractiTest if you are in a compliance setting, if you need on-Premise or if your team generally uses Azure DevOps (the tool formerly known as TFS). To get the full potential of Azure DevOps, though, you need the full Microsoft Test Pro licenses, so it’s not a free tool either – nor is DevOps intuitive for testing things doesn’t have the code available. As with Azure DevOps PractiTest is also SaaS only, with multiple data centers for regional data compliance. As there is always inertia towards a commodity it won’t be long before there is no good arguments to have test management tools on-Premise and for the tool vendors to provide the compliance certificates (ISO/SOC really should be sufficient, IMO).
Out of the box PractiTest supports the categories of testing above (engineered, scripted, exploratory) and has the necessary integrations too: Surefire for unit testing, Maven for CI/CD, Jira, ServiceNow or any other ITSM for requirement input. There is even a two way integration to Azue DevOps. As the web design is “responsive” it could probably run off a tablet. That would enable easier test documentation for field tests. It would be even better to have a small version of it on a phone and be able to use the camera for “screen shots”.
And I need
to own the data around all of this, if I want to in in charge of the testing
(and not only the testers). We are very few software testing specialists on the
project team, but as the manager of testing I need to cover many other people
performing the testing. This transforms my role from test management to one
about leadership, coaching, and facilitation of testing being performed by the
SMEs – and anyone else really.
Recruiting for testing roles these days should be mostly about curiosity, problem-solving and less about productivity and text book knowledge. Recruit for right brain skills – not so much operational process jockeys.
Similarly HFresearch have compiled an analysis that even on a management level the trend is to hire for creative thinkers over “operational experts that improve business performance and productivity”. Talent focus should be on right brain thinkers over – The wonks who spend all day staring at spreadsheets, focused on execution “left-brained” activities are less in demand .
But where do we find curiosity training?
If that is the skills we are looking for perhaps we should stop looking at university programs in computer science or engineering, when we want to recruit testers. I have a computer science master degree, and that was really theoretical and while it somewhat focused on problem solving, the lesson was rarely about thinking outside the box.
I think I would rather higher with business domain skills and train testing theory, than hire a process jockey with no experiences in besides textbook examples. That’s also how I came into testing myself, practical activities first – formal training later.
Perhaps it’s not as such important to have an university degree to get into testing. Though it helps 🙂 A diverse background is important, I know of librarians, laboratory technicians and humanities majors that bring good competencies to the testing field.
Finding one higher education that focuses on building curiosity, whole picture thinkers is hard – perhaps dungeons and dragons, as also discussed at the conference?
The Shift-Right label is that more and more testing (and checking) can happen on the live application in production. Some call it monitoring, some call it Testing in the Wild. It is a very wild idea for some people and some contexts #YMMV. It may very well be the best way of testing in some contexts.
Once I consulted on a network stabilization and delivery optimization project for a consumer bank. They had many issues in their production environment… I strongly advocated that they did test controlled and structured inproduction on the network changes and other operational activities. (I have talked about “How to Test in IT operations“ at Nordic Testing Days 2016). More on testing during IT deliveries in Shift-Deliver.
Shift-Right is trend that people have covered well before me, here are some pointers:
The key is really as Alan puts it “testers should try to learn more from the product in use” and with that comes the tools of Google Canary builds, NetFlix Chaos Monkeys etc.
This trend goes along with Shift-Coach,Shift-Left and Shift-Deliver discussed separately. Initially I considered shift-right to be regarding consulting, but after hearing Declan O’Riordan at DSTB 2016 I realized that shift-right was the right label for test in production, testing in the wild etc.