Teaser for Experiences in Testing Infrastructure Projects

I’m presenting at UKSTAR 2018 on the Topic: Experiences in Testing Infrastructure Projects. The content is a continuation of my materials and talks about testing outside the SDLC, in this context operations and infrastructure. It’s additional problems and examples compared to “How to test in IT Operations” at Nordic Testing Days 2016.

UKSTAR 10% OFF ANY TICKET WITH CODE

Relevant blog posts, but not talk content:

• https://jlottosen.wordpress.com/2017/08/23/andthekitchensinktoo/
• https://jlottosen.wordpress.com/2017/05/24/testing-across-the-it-landscape/
• https://jlottosen.wordpress.com/2016/10/04/less-software-more-testing/
• https://jlottosen.wordpress.com/2017/07/27/test-all-the-things/

The talk is in the same field as this talk – by Mike Talks @testsheepnz although with no applications on top… and other stories 😉

 

Less Test Managers, More Test Coaches

One of the trends/shifts I experience in testing & test management in particular is the Test Coach as discussed initially here: The Shift-Coach Testing Trend (Oct, 2016). Recently (Aug 2017) it came up again in a Twitter thread, where Stephen Janaway stated the inspiration to the title of this blog post.

Less Test Managers and more coaches. That’s how I see it going.

Fittingly as they inspired the first post with the talk “How I Lost My Job As a Test Manager” presented at Test Bash 2015. This post is a further elaboration of the Shift-Coach test management trend. Here are some of my experiences:

• I have been assigned to an agile development team to introduce them to 3 Amigos, Test data driven test automation and such things. The purpose of my involvement was to enable the team to continue the practices without me, and without testers besides the business analyst / product owner (See The domain expert is the tester) as they are doing Shift-left. Similar to an agile or scrum coach, my approach was to look at it as a change in the way of working.
• Another project is an infrastructure project, there are no testers only technicians configuring Cisco routers that by software can replace firewalls, iron ports, VM servers and other network equipment. The project has to implement 80+ of these, so I setup both a test process and an ITIL change request process acting as a test and release manager – another quite frequent trend. I could continue in the project for the duration, but instead I setup guidance and leave when it’s sufficiently in place.

This might be similar to a test architect, a (internal) test consultant activity. It has nothing to do with diminishing testing. Rather I see it as more testing happening, something that would not have been done without the coaching from a test manager. It’s all about finding a test approach that is fit for the context.

Here are some things others have written:

• Katrina has written an excellent article on Test Manager vs. Test Coach that compares the traditional hierarchical test manager with a test coach of agile practices.
• “Gus” Evangelisti: writes about Test coach versus test specialist, impact on queues  comparing the differences in work output based on either a test coach guiding the team or a test specialist in the team
•  Paul Gerrard | Will The Test Leaders Stand Up (2013) pointed out that a future for the test manager could be in Help agile projects to recognize and react to risks, coach and mentor and manage testing (See Wither the test manager)
• Alan Page writes about having only feature teams, and no practice teams in The matrix organisation

The competence of the test coach is to have enough change management expertise (people skills) and test management expertise (domain skills) to know how to coach and facilitate the change. Should test coaches test too, perhaps when required, but not necessarily. The activity is primarily to up-skill the team to continue on their own.

The “Test Coach” is a trend similar to “shift-left” and all the other shifts in testing and test management. I see it as a pattern, and what I read from the threads and discussions is that many test managers gradually shift towards test coaches.

The Shift-Coach Testing Trend

Shift-Coach is when testers and test managers trends towards being coaches and facilitators of the testing activities. Shift-Coach is more about leading the testing than leading the testers to paraphrase from @DevToTest Joe DeMeyers blog post.

The ground breaker for this trend, is to me, the talk “How I Lost My Job As a Test Manager” presented at Test Bash 2015 by Stephen Janaway. Stephen explains how reorganization of the test manager role forced them to be more a facilitator than embedded in the teams. Similarly many other great test managers talk more and more about people skills and coaching, especially in agile projects. I want to define shift-coach around the facilitation testing activities, and place testers that doubles as scrum masters in the Shift-Deliver trend.

In traditional (v-model) projects testing has often included people that were not professional testers; – in user acceptance tests this has often been business subject matter experts. The testing was done by someone with the best knowledge of the topic, and that may not have been the professional tester. That more and more projects do this – more and more, is a big challenge for many testing folks. But it is a significant trend in testing world of 2016.

Shift-Coach trend is visible when Alan Page  talks at Test Bash Philly 2016:

You’ve heard the rumors, and you’ve seen it happen. An organization or development team decides they don’t need testers, and you have big questions and massive concerns. Is quality not important anymore? Are they irresponsible or idiotic? Are their hats on too tight? Do testers still have jobs?

Alan Page is a career tester who has not only gone through the “no-tester” transition, he’s taking it head on and embracing it. Alan will share experiences, stories, strategies, and tactics (and failures) on how he’s taken everything he’s learned in over twenty years of software testing, and used those skills to have an impact on software engineering teams at Microsoft. Whether you’re going through this transition yourself, think it may be coming, or just want to tell someone what an absurd idea this is, this is the talk for you.

This trend goes along with Shift-Right, Shift-Left and Shift-Deliver discussed separately. I discussed these trend labels at Nordic Testing Days 2016 during the talk “How to Test in IT operations“ and coined the labels on the EuroStar Test Huddle forum.

Drive the Testing – Coach!

Trending: Shift-Left

TL;DR: Shift-Left is about testing early and automated. Shift technical with this trend or facilitate that testing happens.

Shift-Left is the label we apply when testing moves closer to development and integrated into the development activities. The concept is many IT years old, and there are already some excellent articles out there: What the Shift Left in Testing Means (Smart Bear, no date), “Shift left” has become “drop right” (Test Plant 2014), Shift Left QA. How to do it. Why it matters (Work Soft, 2015).

To me Shift-Left is still an active trend and change how to do testing. This goes along with Shift-Right, Shift-Coach and Shift-Deliver discussed separately. I discussed these trend labels at Nordic Testing Days 2016 during the talk “How to Test in IT operations“.

Here are some contexts where Shift-Left happens:

• Google have “Software Engineer in Test” as job title according to the book “How we Test Software at Google“.
• Microsoft have similar “Software Design Engineer in Test” as discussed by Alan Page in “The SDET Pendulum” and in the e-book “A-word“
• A project I was regarding pharmaceutical  Track and Trace, had no testers. I didn’t even test but did compliance documentation of test activities. The developers tested. First via peer review, then via peer execution of story tests and then validation activities. No testers, just the same team – for various reasons.
• A project I was in regarding a website and API for trading property information had no testers, but had continuous build and deploy with even more user oriented test cases that I could ever grab. (see: Fell in the trap of total coverage)

The general approach to Shift-Left is that “checking” moves earlier in the cycle in form of automation. More BDD, more TDD, more automated tests, continuous builds, frequent feedback and green bars. More based on “Test automation pyramid” (blog discussion, whiteboard testing video). Discussing the pyramid model reveals that testing and checking goes together in the lower levels too. I’m certain that (exploratory) testing happens among technicians and service-level developers; – usually not explicitly, but still.

To have “no QA” is not easy. Not easy on the testers because they need to shift and become more SET/SDET-like or shift something else (Shift-Right and Shift-Coach and Shift-Deliver). Neither is it easy on the team, as the team has to own the quality activities – as discussed in “So we’re going “No QA’s”. How do we get the devs to do enough testing?”

Testers and test managers cannot complain, when testing and checking is performed in new ways. When tool-supported testing take over the boring less-complex checks, we can either own these checks or  move to facilitate that these checks are in place. Similarly when the (exploratory) brain-based testing of the complex and unknown is being handed over to some other person. Come to think of it I always prefer testing done by subject matter experts in the project, be it users, clients, testers or other specialists.

We need to shift to adapt to new contexts and new ways of aiding in delivering working solutions to our clients.

Less Software, more Testing

I rarely test software these days. I mostly lead testing of IT solutions.

Testing in the context of:

• Updating all corporate PC’s from windows 7/8 to Windows 10
• Consolidating network equipment from more devices to one box, on 80 global locations *
• Move 40 live business applications from one data center to another *
• Take over application maintenance for a specialized public organization
• Implement track and trace for pharma products from production to shops
• Migrate HR data for 2500 people from one system platform to another

Yes, it happens that I participate in a project that is about developing a new business application, but my activities are less about testing software and more about testing in IT solutions in general.

Mostly I manage test activities and describe testing in these contexts. My preferred way of working is in setting and implementing test strategies. I prefer complex and non-ordered projects (Complex and Chaotic – I’m looking at you), it fits well with my context-driven approach of finding the “test solution that fits the context”.

Testing is in it self a solution, that must solve a business problem. Great testing is all about providing information to the stakeholders. I don’t care especially if this is done by someone TESTING or a TESTER. It is my responsibility to setup the testing activities (information gathering) that supports the team, faces the business & technology and challenges the product “sufficiently“.

Sometimes “sufficiently” is merely confirming and going through the motions of explicit requirement coverage. This is a special challenge to me, as I know of many effective and Rapid approaches, that could add valuable information. When I face this challenge, I try to look at the full picture of the project, and what the business want’s to achieve.

The business of the business is business. What matters is not software or projects, but the solutions to the challenges the business have. And the context of testing is similarly so much more than the software.

*: As mentioned in “How to test in IT Operations” at Nordic Testing Days 2016.