The Shift-Coach Testing Trend

Shift-Coach is when testers and test managers trends towards being coaches and facilitators of the testing activities. Shift-Coach is more about leading the testing than leading the testers to paraphrase from @DevToTest Joe DeMeyers blog post.

The ground breaker for this trend, is to me, the talk “How I Lost My Job As a Test Manager” presented at Test Bash 2015 by Stephen Janaway. Stephen explains how reorganization of the test manager role forced them to be more a facilitator than embedded in the teams. Similarly many other great test managers talk more and more about people skills and coaching, especially in agile projects. I want to define shift-coach around the facilitation testing activities, and place testers that doubles as scrum masters in the Shift-Deliver trend.

In traditional (v-model) projects testing has often included people that were not professional testers; – in user acceptance tests this has often been business subject matter experts. The testing was done by someone with the best knowledge of the topic, and that may not have been the professional tester. That more and more projects do this – more and more, is a big challenge for many testing folks. But it is a significant trend in testing world of 2016.

Shift-Coach trend is visible when Alan Page  talks at Test Bash Philly 2016:

You’ve heard the rumors, and you’ve seen it happen. An organization or development team decides they don’t need testers, and you have big questions and massive concerns. Is quality not important anymore? Are they irresponsible or idiotic? Are their hats on too tight? Do testers still have jobs?

Alan Page is a career tester who has not only gone through the “no-tester” transition, he’s taking it head on and embracing it. Alan will share experiences, stories, strategies, and tactics (and failures) on how he’s taken everything he’s learned in over twenty years of software testing, and used those skills to have an impact on software engineering teams at Microsoft. Whether you’re going through this transition yourself, think it may be coming, or just want to tell someone what an absurd idea this is, this is the talk for you.

This trend goes along with Shift-Right, Shift-Left and Shift-Deliver discussed separately. I discussed these trend labels at Nordic Testing Days 2016 during the talk “How to Test in IT operations“ and coined the labels on the EuroStar Test Huddle forum.

Drive the Testing – Coach!

It all starts with an odd piece

One of my coworkers had gotten himself a LEGO 10242 MINI Cooper, and by the help of the other consultants it had been build (to spec?). We look over the remaining pieces and discuss how come. All the 1×1 plates are quite expected, there are extras of these because the weights aren’t that precise and the pieces are cheap. Also customers easily loose them, so it’s cheaper to send some extra than to handle through customer service. On BrickLink inventory there is even a fan made list of the usual extra items….

But an extra black 2×4 plate – naa. That’s odd.. And surely it missed on the bottom of the car. I had prior knowledge .. but have not built this exact set.

Now I have another hunch that the two gray 1×3 tiles and 1×1 dark green bricks in to the rear are missing somewhere. A good thing those consultants have a test department, one could say…  Still the pieces seem not to be 1-CRITICALLY missing, so the model is DONE and accepted. So even if the LEGO tester gets to ask “what if” – we have to remember to hear the answer to “does it matter” – even if it is our favorite hobbyhorse

The Expected

Many test processes and test tools mention that you have to establish the expected results to every test. It is at best risk ignorance not to take the notion of expected results with a kilo of salt #YMMV.

If you can establish the result of your solution in a deterministic, algorithmic and consistent way in a non-trivial problem, then you can solve the halting problem. But I doubt your requirements are trivial.. or even always right. Even the best business analyst or subject matter expert may be wrong. Your best oracle may fail too.  Or you may be focussing on only getting what you measure.

Nailed it

When working with validation in seemingly very controlled environments changes and rework happens a lot, as every new finding needs the document trail back to square one.. Stuff happens. Validation is not testing, but looking to show that the program did work as requested at some time. It is a race towards the lowest common denominator. IT suppliers can do better that just to look for “as designed” [1].

Still the Cynefin framework illustrates that there are repeatable and known projects, and in those contexts you should probably focus on looking to check as many as the simple binary questions in a tool supported way, and work on the open questions for your testing.

Speaking of open ends – every time I see an explicit expected result I tend to add the following disclaimer  (song to the tune of nothing else matters [2]):

And nothing odd happens … that I know of … on my machine, in this situation [3]

And odd is something that may harm my user, business or program result

Significantly…

But I’d rather skip this test step  and work on the description of the test and guidelines to mention:

• Think! Use your head – use what you know.
• You are the expert.
• If it smells fishy, there is something fishy going on
• Wonder, Wander, Galumph if you may
• Analyse, consider,.. if you get an idea, try it!
• Learn, Explore, and by all means test

See also: The unknown unknown unexpressed expectations, Eating wicked problems for breakfast

1: Anyone can beat us, unless we are the best, todays innovation becomes tomorrows commodity

2: https://www.youtube.com/watch?v=tAGnKpE4NCI 

3: I’ve heard that somewhere…

Diversity is important for testing, prejudice isn’t

I want the field of testing to have high diversity

• Different personality types:
  • we need people to get ideas, and people to finish them
  • We need people to see the strategic view, and people to get into the details
• Different backgrounds
  • We need people that can code
  • We need people that understand the business domain
• Different business domains
  • We need testers in the field of software development
  • We need testers in the field of IT / ITIL service delivery
  • We need device testers, embedded software testers….
  • We need testers that understand the GxP regulations
  • We need testers that understand rapid and agile delivery
• Different people
  • Parents, singles, people with kids and without
  • Young people, experienced people
  • People who take it as a lifestyle, and people to whom it’s just a job

…most of all people. People who knows that things can be done in many ways. Let’s get rid of the prejudices that testing is for the detailed and i-dotting only. Testing is about bringing information to the stakeholders about what works and what doesn’t – it’s never about “failure is not an option”.

Recently I was required to do a Cubiks Problem solving test. It’s a 12 minute online test in word patterns, calculations and geometric patterns. Apparently I “failed” to complete all in time, but had a high degree of right answers, so my score was “average” #whatever. That apparently made me perfect to the testing area… OH NO – it only tells you that I put pride in my own work. Everything else is pure speculation and prejudice, as mentioned by Gerry Weinberg in Psychology of Intelligent Problem Solving there is a challenge with these kinds of tests for problem solving – they test, but not for problem solving.

Testing is about solving problems – business problems. Like can we ship?

See also:

• More Diversity in testing
• Being different is key
• The Build-A-Tester Workshop
• HOW DID YOU GET INVOLVED IN SOFTWARE TESTING?
• Diversity in software testing webinar
• What’s the Most Common Career path for Testers?
• More than connecting the dots

Read for your kids – special interest edition

If you are a parent to (early) school children you should know that it is important to read  to your kids. Reading the words out trains vocabulary, recognition, imagination, wondering etc etc. So I read subtitles from movies… because

The kids currently have Star Wars as their special interest [1], and wanted to see the “people” movies. The have played the scenes via the LEGO Video Games (GC) and have a range of the LEGO sets – so they had the basic plot already. Feature movies like Star Wars are usually subtitled in Denmark – while animation movies are dubbed [2]. So in order both to keep up with “PG” [3] and helping them read the titles – I get to watch the movies and read the subtitles…

Poor daddy, it’s almost as hard as when they has to finish the ice cream they can’t 😉

In the last months the (soon to be) 9yo have cracked the reading code and have gone from LIX11 books to the shorter subtitles. The 11yo have rest covered, but some of the longer texts are tricky (I’m looking at you – opening Scroll).

I tried reading Harry Potter (in Danish) but even if the story was very elaborate and detailed it didn’t catch their interest. Neither did classics from when I was a kid (Sorry Bjarne Reuter), so I had to rethink the acceptance criteria for “read for your kids“.

See these two kids are not as easily motivated – it has to tie into something they can see a direct interest in. Their autism makes them very picky on the choice of subject. What I try is to meet them where they are, expand their competencies and give them a lot of positive feedback until they master it on their own.

Links: People are people – despite their labels, They are just people, The yardstick of mythical normality, Acceptance is more than what can be measured

1. special interest, as in overly dedicated into the topic and cannot talk about anything else.
2. The Danish “dubbers” are usually world class, luckily.
3. Episode 3 is still to come, though.

The yardstick of mythical normality

Those that accepted me, worked with my neurology not against it. Their yardstick was not a mythical normality but the potential that they felt that I could achieve. They recognized that my way of doing and learning certain things was different. Instead of proceeding from the assumption that was somehow wrong, they worked with it and helped me to find the place where my neurology and the world could safely mix. 

[ What Acceptance Means to Me | Published on April 20, 2013 by Lynne Soraya in Asperger’s Diary on http://www.psychologytoday.com ]

Acceptance is more than what can be measured

A typical acceptance criteria is that a specific percentage of test cases passed and/or a specific number of critical defects unresolved. Yet the expressions have to be recognized as a way of the stakeholder to express the level of expected confidence. In other words “what are you OK with?” How can we help you gain confidence in the solution? What trends should we look for?

I have experienced situations where the solution was delivered, even though the criteria where not met. The so-called Go-NoGo meeting, was usually a Go-Go, and jokingly called so. I have also experienced that the business deferred a fix for a critical defect in production. That even if a planned release fulfilled the elaborate acceptance criteria – the delivery was cancelled. It was a major enterprise release that was technically to spec – yet because of other business risks, it was postponed.

 As a tester you may experience this where all the automatic Factory Acceptance tests (FAT) pass, but the system still fails to react. I had to text “FAT failed” to someone recently, but it auto-corrected to “FART failed“. Indeed if all the requirements pass and but the system fails to deliver – it is a fart.

The challenge is that not everything that can be measured counts, and not everything that counts can measured. 

And that’s not even considering that 100% testcases passed is a metric that makes no sense. Quality is not only the amount of specific attributes, as ISO/IEEE standards may lead you to measure, but a relationship “something that matters – to someone who matters – at sometime“. If you only look at the measurable – you miss half of the story.

The business needs

• fit for purpose (requirements) and fit for use (business context) [ITIL v3]
• to solve a business problem – ff the problem isn’t solved, the product doesn’t work. (Even of all the tests are green). [Ben Simo]
• information from testing to aid in making business decisions