When Subject Matter Experts test

Teaser for [Test Bash Brighton 2020] : How to Coach Subject Matter Experts to Do Testing

In the recent years I have been working on projects with no dedicated testers but plenty of testing. The testing has primarily been performed by subject matter experts. This is where it gets interesting, as my role on these projects has been to lead the testing being performed by people that have limited experience in testing. They also have no desire to be testing specialists, after all they are already specialists in their own subjects, however, everyone agrees and insist that the testing needs doing. So how do we ensure that the testing being done is done well? 
 After having worked on several very different projects, yet still with subject matter experts doing the testing, I have been able to get both the public process clerks and the technology specialists to perform excellent testing. This talk is about the approaches that I have found work well: 

  • One of the approaches is for me to prepare the test cases and prepare them only as headlines. Sometimes preparing the tests as open questions helps too. 
  • Another approach is to lead them as if they are doing the project participation voluntarily. They probably are, but still it helps to respect where they are coming from.

 The lessons though (good and bad) is relevant to many testers in other situations, especially being the only “tester” on the team. The story applies equally to developers and business end users doing most of the testing and you will have them contributing with great testing in no time!

What you will know after the talk:

  • An understanding of how testing looks when done by subject matter experts
  • How to lead a testing activity with an appreciative and motivating style
  • Examples of how teams can do great testing without dedicated testers
Test Bash Brighton, March 2020

A Track down History

Currently I am doing a large V-model waterfall project with a three month testing period and 500+ requirements. To track the progress I want to dust off my old “test progress tracking” method that I matured and described in 2011 and 2014.

The approach was documented in two articles for “The Testing Planet” a paper magazine by the Software Testing Club. The “STC” later became the now world famous Ministry of Testing. Unfortunately the original articles are no longer available on the MoT site – they are on the WayBackMachine. So not to track down that path, here’s a recap of:

A Little Track History that goes a Long Way

For large (enterprise, waterfall) projects tracking test progress is important, as it helps us understand if we will finish a given test scope on time. Tracking many of projects have given me one key performance indicator: daily Percent Passed tests as compared to an s-curve. The data in the S-curve is based on the following data points, based on numerous projects:

Time progressExpected Passed Progress
10%0%
20%5%
30%10%
40%30%
50%45%
60%60%
70%75%
80%90%
90%95%
100%100%
Adding a 3rd order polynomial trend-curve gives the s-curve.

If the “Actual” curve is flat over more days or is below the blue trend line, then investigate what is blocking the test progress… defects perhaps:

The Defect Count and Camel Image

During a large project like this the active bug count goes up and down. No one can tell what we find tomorrow or how many we will find. In my experience tracking the daily count of active defects (i.e. not resolved) is key, and will oscillate like the humps on a camel:

Camel background is optional

If the curve doesn’t bend down after a few days there are bottlenecks in the timely resolution of defects found. When the count goes up – testing a new (buggy) area is usually happening. Over time the tops of the humps should be lower and lower and by the end of the project, steep down to 0.

And thus a little track history has come a long way.

The Testing Planet, July 2011

A 30 Days Agile Experience

In September 2017 the Ministry of Testing had a crowd-based knowledge sharing event called “30 Days of Agile Testing” with a small learning activity for each day of the month. As with the similar security event I set up a weekly schedule at work to meet for an time-boxed hour and discuss 3-5 selected topics each time.

Our score was 17 topics discussed – some more discussed than actually tried out. Hence the half marks on the poster on the window below. Me and my coworkers work on many different teams – so to dig into specific team tools and processes was out of scope.

Here is a few of our findings:

IMG_0007

Links to “the Club” on some of the topics we selected:

 

 

Testing roles are shifting

New ways of delivering software and solutions challenge existing perceptions of what roles and activities testers and test managers have. Some are getting more into development others into people skills. Sometimes forcefully, other times as part of self-organizing teams.

This post is a link collection for the talk: Testing roles are shifting, but where to? for the Ministry of Testing Meetup Copenhagen chapter, march 2017.

Introduction

Shift Left:

Shift Right:

Shift Coach

Shift Deliver:

 

Additional links

Continue the discussion Where are testing roles shifting to?

Co-creating Smarter Tester
Co-creating Smarter Tester

3 Sessions of Security Testing

One way to collaborate in a team is to achieve shared knowledge together. An example of this is the online activity of “30 days of testing” that The Ministry Of Testing has been putting out to the online community to participate it. My test team has a “Work Group / Special Interest Group” with regards to security testing, so when a 30 day challenge for security testing came up, we scheduled sessions to learn from the topics provided (see below).

As we are testing consultants doing work for our customers, we scheduled 3 sessions – initially for an hour. At the start of the hour we picked 4-5 topics from the list, and worked our way through them in a prioritized order – within the time box of the hour. Come to think of it we might as well have used the Lean Coffee format. As we have team members two places in DK and one place in PH, it was a skype call using screen sharing. After the call I  summarized sending out a “link mail” to all in the testing group (DK and PH). Evaluating the sessions we extend our ordinary scheduled WG meetings to make room for collaboratively investigate additional security testing topics.

12 From the list: ZAP, Google Gruyere, threat models, HTTP proxies, posture assessments, tiger boxes, recent hacks (elaborated by Troy Hunt), OWASP top 10, OWASP SQL injections, adding data integrity testing into a test plan, share ideas for security testing internally and externally, discuss security testing with regards to EU GDPR compliance.

7 Not on the listNaughty Strings form GitHub, Bug Magnet plugin, How real persons names trick IT systems, how to be careful with custom license plates, DDoS attacks, IoT privacy failures, Chaos monkeys/Siamese army and little Bobby Tables:

exploits_of_a_mom
XKCD: Exploits of a mom

To sum up, we have learned about: what tools that can make testing easier, where to read about vulnerabilities and and simple exploits, understand how personal data and logins are used and stored, how to pitch security testing based on fear of breaches and safety concerns, testing the requirements for “by design” security.

30 Days of Security Testing
30 Days of Security Testing

Many Bits under the Bridge

I’ve been in the testing business for 14 years – when I started in late 2002 it was all about using HP Test Director 7.6 – in a browser… There was only one model of testing, v-model, and only one book of testing the ISEB (later ISTQB) vocabulary. And only one expected output of testing: Testers designed test cases, executed and perhaps wrote both a test plan document and test report document. Test process improvement was a thing, but even so testing was often a pointy cog…  

Many Bits under the Bridge Later

It is not about the test cases any more, it’s about being part of a team – that delivers an IT solution to the business. First of all, if it’s just about the test cases then it is a race to the lowest paid off-shore location, a run to the bottom in repetitiveness and mechanic activity. Checking! with more focus on crossing the t’s and dotting the i’s. We have tools for that now – the plates are shifting.

When testing professionals puts “writing test cases” on their LinkedIn description. It seems to me that they are stuck in the testing world of 10 years ago. Standing still and not seeing that Testers are Knowledge Workers – not workers of producing artifacts. It is much more important to see beyond the visibleUncovering better ways and seeing testing as an activity to provide information to the stakeholders, based on experiments and observations.

Skill up and be smarter! And don’t listen to old tapes – it’s not worth it :).

See also this from QA Symphony & Ministry of Testing:

The Software Tester: Modern vs. Traditional [Infographic]
The Software Tester: Modern vs. Traditional [Infographic]

Co-creating smarter testers

Co-creating smarter testers” is the byline of the Ministry of Testing, a small company with a great impact that I have been following and supporting for 7 years* now . I have attended TestBash’es, webinars, challenges, discussions and memes. And now for the first time in Denmark – Anders Dinsen and I are bringing the world known Meetups to Copenhagen (Aarhus 2017 you’re next).

Ministry of Testing – Copenhagen

Copenhagen, DK
224 Members

The Ministry of Testing exists to advance the software testing industry in a fun, safe, professional and forward thinking way.Our meetups exist as a way to bring people toget…

Check out this Meetup Group →

The topics so far are:

At the first meetup we split into three groups, discussed risks and how to TEST THEM RISKS. Dearest to me was the discussion of stakeholders and new places to test. Great to see that even with very little information, we can still do a rapid testing based on business objectives. There is so much more to testing these days.

600_457785753

1: since a EuroStar 2010 t-shirt competition 🙂