Staff-Level Testing Roles, Career Paths for High-Level Testing Roles, TBD 2025 on LeanPub: https://leanpub.com/staffleveltestingroles
Leading Testing Activities, A Guidebook for New Leaders of Testing, > TBD 2024 on LeanPub: https://leanpub.com/ltabook
Goal-Aligned Test Strategies, Using Wardley Maps to visualize test strategies and align the test strategies with business goals, 2022. -> Available on LeanPub: https://leanpub.com/goatsbook
Yes, you. What if you’re not around – what will happen with your tasks? Perhaps you feel that you can’t really take time off – because you are in the middle of everything. All teams need some way to handle both planned and unplanned absences. It’s not a matter of if – it’s a matter of when.
The topic came up recently on the Ministry Testing Club: How do you mitigate the risk of someone leaving the company. My answer was shared on LinkedIn, which led to Melissa reminding me to blog about it. Having been absent from the blog for a while, it was an appropriate topic 😉.
First of all, plan your planned absences. Whatever your local PTO and vacation law allow plan as much as you can in advance. Knowing that you usually need X amount for summer and Y amount for the holidays is a great start. Perhaps keep a few in reserve, if you are more of a person who has to take time off more reactively. I know the Nordics have generous laws with both vacation allowance, illness with pay, and child illnesses. While laws are laws, the culture can vary from company to company on how to manage absences. As with many other themes – be in control of your own time.
Where I work, we have a project allocation system that also covers “non-working” requests. As shared in the Club and in the LinkedIn post, a test lead recently submitted vacation 8 months in advance and had it approved. The customer, though, updated the project release date and a test lead was required during the test lead vacation. We struggled to find a replacement, as everyone else was already accounted for. There was no immediate mitigation of that test lead being away in that team.
We do have graduates, student workers, and other entry-level positions in the company. I try to engage with them and train them. So that they can learn and I work on something else. I called it a “peer system” in the Club reply above. It really goes for all roles, especially for those that are prone to bottlenecks. As I’m writing this I see yet another post on soMe, where a tester has 11 work items in status “Ready for test” at the end of the sprint and on a Friday. Testers are not the only ones prone to being bottlenecks. Principal staff easily becomes bottlenecks if they are the only ones to approve PRs, review work, do firefighting, etc. Being a bottleneck you become prone to stress, overworking, and potentially long-time unplanned health events that will make you absent.
How about if we formalized a peer system a little more. Appoint a peer for everybody! Have someone that you keep in the loop and someone that keeps you in the loop. It’s not about Cc’ing your manager on everything. They are usually doing management stuff and should have a similar peer. Perhaps you – if you are an aspiring manager. Some years back a coworker of mine became manager after having been a maternity leave temp for the previous manager. A sign of a good company is that they take business continuity management seriously in the leadership group. It’s actually a thing – if you need it at your place let me know.
Good leaders, even the informal ones without hierarchical responsibilities always try to delegate and work themselves out of a job. They share everything, work in public, and don’t sit on information or documentation. While it seems beneficial to be the go-to person (even for testing), it doesn’t scale in the long run. If you hoard skills and practices you become a bottleneck, a single point of failure. Rather you should move from “me” to “we” and learn to grow what you master in others. This will also help you grow yourself. There will be more about growing into leading testing activities in my book on the topic 😉.
I will always try to work collaboratively and share what I know. Plan your own time both for today, this week, and for the future. Yet life happens and having a peer helps both you and your peer if you are not around.
This is my blog post 300 since the start in early 2012. 🎉I blog to reflect on the world happening around me, at work, at conferences, and in the global testing community. Sometimes blogging helps me articulate patterns and trends I sense, other times I reflect on reality based on things I have read (Go bookclub! 📕). Here are my 20 most used tags:
agile atWork business change collaboration communication community conference context decisions exploratory goatsbook knowledge leadership learning skills test_strategy value wardley wicked problem
Originally, I set aside an hour every other Friday afternoon to spend an hour blogging. The last hour of work was never that productive anyway, so spending a few percent of my work time has surely paid off. It was my personal 20% rule (just 2%). Setting aside continuous timeslots for learning and creation has helped me many times when I have had to spend dedicated time on creating something. If I skipped one slot I always had another slot coming up soon. My experience is very similar to Alan’s in This is Me Trying.
I love every time my posts are shared by curators of testing news, and then share my articles with thousands of testing professionals around the world. My posts have been frequently featured in:
Currently, I don’t have a blogging schedule – I blog when I have time and the urge to write. Both time and urge are required, though. It’s no help in having ideas if you don’t have time – and have time without the urge. I run a “blog backlog” in my personal tasks management system, which is more of a stack (First-in-Last-Out). I have the next 2-3 ideas already – but I’m currently focussing on my second 🗺📕book. Three hundred blog posts in almost 12 years correspond to 25 a year and on average 2 each month. Over the last couple of years, this average has been quite consistent. Full Stats below – this blog though is a side project besides work, family, and life in general.
I do not make a living as such from blogging. Indirectly blog posts have led to successful conference submissions. I have been fortunate to have one yearly conference presence since 2016. In 2022 I realized I could turn many of my blogposts into the foundation of a book. The royalties from the book have helped pay for other books in the community and an upgrade to a commercial-free version of WordPress. Thank you, if you have purchased my 🐐📕 book.
The choice of platform and the name have both been a bit lost over the years. But I stick around here because it’s a platform that is portable. I own the content and the cadence. It’s not paywalled nor does it require a login.
All-time (2012-2023) top 5 blog posts are:
Spending your time writing long-form test documents is not an efficient way to communicate, nor will it get you the situation-specific support that you probably need. Don’t listen to the stochastic parrot – request information that is relevant to the situation at hand.
Recently I read a Slack thread on test plans, and one of the comments was that we should stop making things so wordy and reduce overexplaining everything. That reminded me of the time I wrote a 100-page test strategy that was very far from strategic – it was rather a specification of the how & the what, and less about why. Unfortunately, I had no choice in the content as it was part of a local public tender reply. To win the thing we had to provide a document called “Test Strategy” with items like:
I’ve been around the testing block to recognize the old book this is coming from it’s older than 2008 BTW). While it seems helpful to elaborate on everything and the kitchen sink, it’s mostly busy work of empty calories. We have better ways of working than a big specification (of testing activities) upfront.
Don’t spend time writing a detailed Test Plan which will be out of date in no time and most people don’t bother reading (TL;DR). Instead, focus on communicating. The best plan doesn’t win. The best intelligence wins!
Andy Glover, LinkedIn 2023
My bet for the future is that long-form template documents will soon be generated using large language models. The more you ask for the more will be auto-generated by a stochastic parrot [On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 Authors: Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, Shmargaret Shmitchell]
The better approach for customers of IT services is to ask for less text. Have the documents zoom in on the vendor’s perception of the client’s challenges. Yes, it will take more time and effort – but you can still score a short essay for the tender process.
Make shorter planning documents – and for strategies, make a map.
Every day I hear a focus on delivering reliably. Day-to-day project tasks, milestones, and generally keeping busy. Given all the changes happening in our world and industry, we must also be adapting to change – just to keep up. You need to make room for new things, as John Cutler puts it: You have to make room for the high-value ad-hoc/new activities. You can’t just ADD this on top of everything else. Otherwise, you’ll have no energy stores available to practice and work through the discomfort.
The DevOps Handbook framed the challenge of adapting and delivering reliably like this:
“the chronic conflict of pursuing both: responding to the rapidly changing competitive landscape for features and platforms while maintaining business as usual”
The DevOps Handbook, How to Create World-Class Agility, Reliability, & Security in Technology Organizations, 2016
What I see more and more is that this applies not only to delivering software solutions using DevOps practices. The challenge is inherent in any technology organization – when you are delivering consulting services, software solutions, and cloud solutions.
While reading how Simon Knight applies research from outside product management to product management in the recent article: On Signals & Noise, I wondered do we really need to invent the wheel all over again? Do we need yet another strategy framework to both keep up with change and deliver reliably? Would the techniques of today apply to the challenges of tomorrow?
The thing is, though, for technology organizations to tackle the core conflict above – has been solved, documented, and proven successful. If you want organizational monetary performance, job satisfaction among the staff, and sound organizational culture as a technology organization, look up the research in Accelerate as illustrated below:
Notice how lean product development (work in small batches, visible flow, Customer feedback, and Experimentation) applies not only to software development – but to any delivery for a technology organization. Similarly is Lean Management in itself a key driver to support items like Less Burnout, Software Delivery Performance, Job satisfaction, and a generative culture.
Business organizations can be designed – and they can be designed for flow. This has been documented in Team Topologies: It is a model that treats teams as the fundamental means of delivery, where team structures and communication pathways are able to evolve with technological and organizational maturity.
(Rant: Stop talking about centers of excellence already – design your organisation for flow (Team Topologies) and learning)
There is a model for your trouble already
Which leads me to the map below. To prioritize reacting to a rapidly changing competitive landscape, you need to make an active choice to move your practices along the lines of “Accelerate” and “Team Topologies.”
The more you learn in your role, the more important it becomes for you to share and build skills in others. If you hoard skills and practices you become a liability, a single point of failure. Rather you should move from “me” to “we” and learn to grow what you master in others. This will also help you grow yourself.
In my many years in the same role, I simply took on more tasks. We just dug in and worked, often putting in overtime. People grew and were promoted based on being scarce. We could never recruit as skilled as us. We were the Go-To team. Tasks and knowledge were often hoarded. After a change in the team, I realized that it was the wrong way around. Both for me, for the organization, and for all the new people in the field.
Experienced people generally thrive with novel and emerging problems. They eat wicked problems for breakfast and know their way around the complexities of Cynefin. They will usually be bored by the more simple and rote tasks. Their preferred field is to work with new situations (Illustrated below, left figure).
To new people in the field, though, it’s the other way around. They love a good template, a cookbook, or an example. Their best work is, usually, to take some half-baked good enough and make it follow given (best) practices. (Illustrated below, right figure).
When experienced individuals disregard the red cross and monopolize tasks and knowledge, it may initially appear beneficial. They are praised for their perceived irreplaceability. However, over time, this can become a burden for the organization. New systems and patterns emerge, and the inability to adapt becomes a challenge. Work halts, People crack under stress and everyone stops using COBOL eventually. Over time, skills that are not shared move to the left on the evolution scale, but just lower down on the visibility to the stakeholders. (In the jargon of the Wardley Mappers). A pet project in a not-so-good way.
On the other hand, having new people sharing the load and contributing to the tasks helps build new people who can handle tasks like this. It’s easier to recruit people with an appetite for learning and training them, than hiring skilled staff. Thus making more people grow. You can’t be everywhere.
Another way to elaborate on this change is the difference between Senior (L5) vs. Staff (L6) as described by The Developing Dev in the drawing below.
As elaborated here:
Staff engineers uplift others around them. They should have the ability to help L5 engineers grow. There are a few ways they uplift others:
- Mentorship – Dedicated mentorship, preferably with senior engineers
- Knowledge sharing – Writing wikis, giving presentations, contributing to Q&A groups
- Collaborations – Growing others while working with them (e.g. code reviews, design reviews, discussions)
L6 engineers should also contribute to growing the organization. This means that they help with recruiting and partner with their manager to improve team health.
Developing Dev, 2023
In the future, I will do my best to stop hoarding skills and start building capability in others. Not only actively sharing knowledge but actively sharing tasks. As I said a few years ago: As you are learning new stuff, I am unlearning to help you
In my experience, managing testing and quality activities is becoming more about managing compliance, security, and contractual activities than worrying about feature testing and scalability. Sure, feature development and scalability need testing, it’s just that we have industry practices in place that handle these more effectively than a team of testers and old-school tools.
What do you see as changes to the scope of your testing activities? If you manage testing, especially in a business-to-business context, two primary areas are becoming more important to the business: Compliance, Security, Contracts, and Operational Readiness.
Quality is many things to many different people. In this blog post, I will cover some of the perspectives you can encounter in an enterprise setting. Enterprise settings differ from the usual software development shops in that the enterprise corporations have regulations and audits to comply with, and IT features merely support the delivery of something else being produced. By the way, even software development companies will eventually need to have similar roles as the company grows.
Especially in a Life Science and Healthcare setting “QA” is not your usual software testers. Quality Assurance and Quality Control (QA/QC) is an independent job role, that focuses on if we have followed the process. Companies with actual factories often have a range of standard operation procedures defined in their “Quality Management System” (QMS). A QMS might be required by law or similarly required to operate in a market. The quality control and quality assurance activity is to compare if these processes have been followed. Examples:
Work items are usually documents, including test plans and test cases. Their measure of quality is the better adherence to the QMS the better. Similarly for auditing and regulatory compliance – the more aligned the organization is with a given standard/law/contract – the better. There is a quest for “objective” evidence in the sense that screenshots and the like are captured before something is considered “done“. Their model of Quality is similar to the classic V-model of testing – it’s about what is set forth in a specification, nothing more, nothing less.
Software testing, though, has moved on its own in the last 10 years. Driven primarily by the essence that “quality is a relationship“.
It’s probably best illustrated and articulated by Dan Ashby: “Testing is an investigatory activity (exploration) that has the effect of uncovering more information“. Good testing compared to QA above is to explore what the customer really wanted – and what trade-offs she has.
The world is not binary – every time we look closer at a binary split it becomes more fuzzy. Even the computer bit 0/1 is a simplification of low and high voltage. The historic idea of testing frameworks to have either functional or non-functional requirements in focus is failing us similarly. There is much more to testing activities than both functional and non-functional requirements.
One of the testing activities that fail to be recognized in the functional/non-functional split, is compliance requirements. US SARBOX and EU GDPR were just the start, on the horizon are EU NIS2, US FISMA, WCAG, and others. WCAG deals with accessibility for websites and is a good example of a compliance requirement that is part functional (dictates functionality: alt-text, screen reading) and part non-functional (dictates contrasts and navigational aids). Many emerging governmental regulations are around broader security controls and information security. To be compliant your organization has to do secure application development – the requirement is to your organization rather than the product. Yet the requirement still has to be met – the business “license to operate” might depend on it.
I often work with solutions that are outsourced to where I work. There might even be a consortium (collection) of vendors in the mix for building the thing. These projects have activities called “Transitional Trials,” “Operational Trials,” or “Operational Readiness Tests” [I also wrote about this in 2017]. The terms seem to come from a more physical form of delivery. These trials fall under my hat in my role as test manager/test lead aka “project manager of the testing and trial activities“.
Operational Trail is often a 30 days period with the solution in full operations with live users, where it’s confirmed that the whole ecosystem of the solution is operational and follows the service level agreements and ITIL processes. The “test cases” of these phases could be:
The requirements come from the outsourcing contract, not from a specification of the system’s functional- or non-functional requirements. It’s a specific activity on its own that needs someone to manage it and write the confirming “test reports around it”. And yes a “trial” report similar to a test report is often required.
A last testing activity that might fall under the scope of the project test manager, that is not classically functional or non-functional is the activity of gathering evidence of contractual deliverables. the last couple of years I have been part of projects, where we had not only to deliver the system and functionally test the system – but, had also to collect evidence of the contractual deliverables.
The outsourcing contract converted its textual requirements into a huge spreadsheet of ~1000 lines. Each line had to point to objective evidence of each contractual requirement being fulfilled. Examples:
This could also be a testing/trials activity, as we would create a plan document of the scope, List the items under test, confirm each item by finding evidence, fix bugs, and finally collect a report.
These three testing activities are about the testing – not the testers. The actual performance of the controls might be done by a range of very different subject matter experts. If any of the above testing activities fails to be handled – the business is equally at risk as if it was a functional or non-functional requirement failing.
Test cases have been a large element of testing activities for the last 20+ years. While they are representations of testing activities, having a strict regime around their format backfires into trust issues and bureaucracy. Elaborate test case specifications go against the primary purpose of the testing activity they are to support.
Here are three specific situations where too much thinking has been put into the test cases:
Project S: To enable a new feature in the solution the database needs to be remodeled, adding a new table and moving data around. We prepare a database migration script and a verification script. The purpose of the verification script is to test that the data are in the right places. We also do some regression testing and performance testing. To have a complete overview and transparency of the testing activity I add all three “types” to our test case management platform. But the verification script test case is mostly a placeholder and will not have the same characteristics as classical manual test cases wrt steps, test data, and expected results.
Project T: As part of an outsourcing agreement the customer requires approval of all system documentation, and whether they have been updated. It’s a list of potentially hundreds of documents that are listed, with each their approval status and metadata. While each of them could be a “test case” in a test case management tool, none of the document verifications would match the properties of functional test cases: requirement coverage, expected results conditions, input, user roles, etc. It’s still a test activity, albeit a static one.
Project V: Building an API platform using high-level cloud components – most of the features can be confirmed using test automation using an API testing framework. Plenty of guides exist on how to automate API testing coding positive, negative, and similarly technically exploring the API by automated payloads. We use MS Azure DevOps and can integrate test automation into the CI/CD pipelines. Why do we need high-level test cases to verify the same as the automation?
One of the pitfalls of test cases is that they are countable and provide a form of structure, and any form of structure feels safer. We mistake countable formalized work items as value. While there is some truth to moving elements to a more explicit form, spending the focus on sub-optimizing the content is a misguided form of value creation. Additionally, discussions mostly on form and attributes lead to mistrust between the parties – not a collaborative partnership.
The key purpose of the testing in all three projects above is to confirm expectations in a consistent form and provide objective evidence. The world has moved since formatted test cases, and objective results can be provided in many media forms: video, screenshots, run results, etc. What is needed is a repository of facts and an aggregated overview.
Some would probably argue that everything and every little detail is equally important to them. I’m sure that when we explore this a bit, it turns out that the delivery of software is prioritized over comprehensive documentation. In most cases. Mean Time To Repair is key for the stakeholder’s willingness to take risks.
Business results are better supported if the testing can be repeated consistently – ie. it’s automated. Go read Accelerate wrt supporting business needs with DevOps practices. DevOps practices cover the things that can be automated: You should automate 100% of the tests that should be automated.
Plenty of publications have been made for the value of more exploratory testing to explore the unknown and unarticulated expected results (aka requirements).
To me, the best test cases are representatives of a testing activity that cannot be automated. They go in folders of folders for organization. Each has a unique ID, a title, a state (ToDo>Doing>Done would suffice), a responsible, and a description field. Attachments, tags, and references to other work products are nice – everything else is just overhead. Don’t overthink it.
Subscribe to get access to the rest of this post and other subscriber-only content.
Complexity is a Matter of Perspective 