What If You Are Not Around?

Yes, you. What if you’re not around – what will happen with your tasks? Perhaps you feel that you can’t really take time off – because you are in the middle of everything. All teams need some way to handle both planned and unplanned absences. It’s not a matter of if – it’s a matter of when.

The topic came up recently on the Ministry Testing Club: How do you mitigate the risk of someone leaving the company. My answer was shared on LinkedIn, which led to Melissa reminding me to blog about it. Having been absent from the blog for a while, it was an appropriate topic 😉.

First of all, plan your planned absences. Whatever your local PTO and vacation law allow plan as much as you can in advance. Knowing that you usually need X amount for summer and Y amount for the holidays is a great start. Perhaps keep a few in reserve, if you are more of a person who has to take time off more reactively. I know the Nordics have generous laws with both vacation allowance, illness with pay, and child illnesses. While laws are laws, the culture can vary from company to company on how to manage absences. As with many other themes – be in control of your own time.

Where I work, we have a project allocation system that also covers “non-working” requests. As shared in the Club and in the LinkedIn post, a test lead recently submitted vacation 8 months in advance and had it approved. The customer, though, updated the project release date and a test lead was required during the test lead vacation. We struggled to find a replacement, as everyone else was already accounted for. There was no immediate mitigation of that test lead being away in that team.

A Peer System

We do have graduates, student workers, and other entry-level positions in the company. I try to engage with them and train them. So that they can learn and I work on something else. I called it a “peer system” in the Club reply above. It really goes for all roles, especially for those that are prone to bottlenecks. As I’m writing this I see yet another post on soMe, where a tester has 11 work items in status “Ready for test” at the end of the sprint and on a Friday. Testers are not the only ones prone to being bottlenecks. Principal staff easily becomes bottlenecks if they are the only ones to approve PRs, review work, do firefighting, etc. Being a bottleneck you become prone to stress, overworking, and potentially long-time unplanned health events that will make you absent.

How about if we formalized a peer system a little more. Appoint a peer for everybody! Have someone that you keep in the loop and someone that keeps you in the loop. It’s not about Cc’ing your manager on everything. They are usually doing management stuff and should have a similar peer. Perhaps you – if you are an aspiring manager. Some years back a coworker of mine became manager after having been a maternity leave temp for the previous manager. A sign of a good company is that they take business continuity management seriously in the leadership group. It’s actually a thing – if you need it at your place let me know.

Delegate Your work

Good leaders, even the informal ones without hierarchical responsibilities always try to delegate and work themselves out of a job. They share everything, work in public, and don’t sit on information or documentation. While it seems beneficial to be the go-to person (even for testing), it doesn’t scale in the long run. If you hoard skills and practices you become a bottleneck, a single point of failure. Rather you should move from “me” to “we” and learn to grow what you master in others. This will also help you grow yourself. There will be more about growing into leading testing activities in my book on the topic 😉.

I will always try to work collaboratively and share what I know. Plan your own time both for today, this week, and for the future. Yet life happens and having a peer helps both you and your peer if you are not around.

#300 – Blogging Helps Me Reflect

This is my blog post 300 since the start in early 2012. 🎉I blog to reflect on the world happening around me, at work, at conferences, and in the global testing community. Sometimes blogging helps me articulate patterns and trends I sense, other times I reflect on reality based on things I have read (Go bookclub! 📕). Here are my 20 most used tags:

agile atWork business change collaboration communication community conference context decisions exploratory goatsbook knowledge leadership learning skills test_strategy value wardley wicked problem

Originally, I set aside an hour every other Friday afternoon to spend an hour blogging. The last hour of work was never that productive anyway, so spending a few percent of my work time has surely paid off. It was my personal 20% rule (just 2%). Setting aside continuous timeslots for learning and creation has helped me many times when I have had to spend dedicated time on creating something. If I skipped one slot I always had another slot coming up soon. My experience is very similar to Alan’s in This is Me Trying.

I love every time my posts are shared by curators of testing news, and then share my articles with thousands of testing professionals around the world. My posts have been frequently featured in:

• 5blogs (38 times during 2012-2023)
• Software Testing Weekly (Issues 180,175,186, 149, 80, 35, 67, 90, 165,75)
• Software Testing Notes (Issues: 26, 55, 79, 96)
• Testing Bits / Testing Curator (stopped?)

Currently, I don’t have a blogging schedule – I blog when I have time and the urge to write. Both time and urge are required, though. It’s no help in having ideas if you don’t have time – and have time without the urge. I run a “blog backlog” in my personal tasks management system, which is more of a stack (First-in-Last-Out). I have the next 2-3 ideas already – but I’m currently focussing on my second 🗺📕book. Three hundred blog posts in almost 12 years correspond to 25 a year and on average 2 each month. Over the last couple of years, this average has been quite consistent. Full Stats below – this blog though is a side project besides work, family, and life in general.

I do not make a living as such from blogging. Indirectly blog posts have led to successful conference submissions. I have been fortunate to have one yearly conference presence since 2016. In 2022 I realized I could turn many of my blogposts into the foundation of a book. The royalties from the book have helped pay for other books in the community and an upgrade to a commercial-free version of WordPress. Thank you, if you have purchased my 🐐📕 book.

The choice of platform and the name have both been a bit lost over the years. But I stick around here because it’s a platform that is portable. I own the content and the cadence. It’s not paywalled nor does it require a login.

All-time (2012-2023) top 5 blog posts are:

1. Visualize the test strategy
2. Go Read Accelerate
3. Pink LEGO is not new to the Friends Theme
4. Seriously Joking or joking seriously
5. Test ALL the things

94.000 words apparently!

#299 – Wordy Documents Are Not Better

Spending your time writing long-form test documents is not an efficient way to communicate, nor will it get you the situation-specific support that you probably need. Don’t listen to the stochastic parrot – request information that is relevant to the situation at hand.

Recently I read a Slack thread on test plans, and one of the comments was that we should stop making things so wordy and reduce overexplaining everything. That reminded me of the time I wrote a 100-page test strategy that was very far from strategic – it was rather a specification of the how & the what, and less about why. Unfortunately, I had no choice in the content as it was part of a local public tender reply. To win the thing we had to provide a document called “Test Strategy” with items like:

• Start and Approval Criteria
• Listing of Applied Testing Types
• Milestones
• Risk areas
• Methods for testing coverage
• Test environments and test data
• Test tools
• Consideration of the Need for Test Automation

I’ve been around the testing block to recognize the old book this is coming from it’s older than 2008 BTW). While it seems helpful to elaborate on everything and the kitchen sink, it’s mostly busy work of empty calories. We have better ways of working than a big specification (of testing activities) upfront.

Don’t spend time writing a detailed Test Plan which will be out of date in no time and most people don’t bother reading (TL;DR). Instead, focus on communicating. The best plan doesn’t win. The best intelligence wins!

Andy Glover, LinkedIn 2023

The Stochastic Parrot

My bet for the future is that long-form template documents will soon be generated using large language models. The more you ask for the more will be auto-generated by a stochastic parrot [On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 Authors: Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, Shmargaret Shmitchell]

The better approach for customers of IT services is to ask for less text. Have the documents zoom in on the vendor’s perception of the client’s challenges. Yes, it will take more time and effort – but you can still score a short essay for the tender process.

Make shorter planning documents – and for strategies, make a map.

Business needs are for innovative deliveries – usually not commodity services.

New Fields of Testing Activities

In my experience, managing testing and quality activities is becoming more about managing compliance, security, and contractual activities than worrying about feature testing and scalability. Sure, feature development and scalability need testing, it’s just that we have industry practices in place that handle these more effectively than a team of testers and old-school tools.

What do you see as changes to the scope of your testing activities? If you manage testing, especially in a business-to-business context, two primary areas are becoming more important to the business: Compliance, Security, Contracts, and Operational Readiness.

• In the last few years cybersecurity and compliance activities have only grown in importance to the CEO. Managing solution requirements will require a dedicated focus on security and compliance across the board. Both security and compliance need to be built into the solution and be built into the company processes and procedures.
• Working in the business-to-business domain or delivering solutions to the public sector will often face operational readiness trials. Make sure your test lead addresses the contractual terms and operational readiness. This is both if you are the customer or the vendor of IT solutions.
• Scalability and performance used to be a big deal. Now we have better performance practices and observability tools. The field of Site Reliability Engineering is all about the practices of proactive scaling and performance reducing the for explicit testing up-front for many of the non-functional requirements.
• Similarly, feature testing used to be a big deal to juggle. The industry best practice for software development teams is a holistic approach to a whole team testing effort. Supplemented with test coaches to enable stream-aligned teams to deliver quality features. It’s less about managing a long list of feature test cases.

Is Compliance on Your Strategic Roadmap?

Quality to Whom?

Quality is many things to many different people. In this blog post, I will cover some of the perspectives you can encounter in an enterprise setting. Enterprise settings differ from the usual software development shops in that the enterprise corporations have regulations and audits to comply with, and IT features merely support the delivery of something else being produced. By the way, even software development companies will eventually need to have similar roles as the company grows.

The Other “QA”

Especially in a Life Science and Healthcare setting “QA” is not your usual software testers. Quality Assurance and Quality Control (QA/QC) is an independent job role, that focuses on if we have followed the process. Companies with actual factories often have a range of standard operation procedures defined in their “Quality Management System” (QMS). A QMS might be required by law or similarly required to operate in a market. The quality control and quality assurance activity is to compare if these processes have been followed. Examples:

• Work items are prepared in the right template, with the approved content
• Work items are approved appropriately
• Approval gates are followed

Work items are usually documents, including test plans and test cases. Their measure of quality is the better adherence to the QMS the better. Similarly for auditing and regulatory compliance – the more aligned the organization is with a given standard/law/contract – the better. There is a quest for “objective” evidence in the sense that screenshots and the like are captured before something is considered “done“. Their model of Quality is similar to the classic V-model of testing – it’s about what is set forth in a specification, nothing more, nothing less.

The Testing Quality Model

Software testing, though, has moved on its own in the last 10 years. Driven primarily by the essence that “quality is a relationship“.

• Quality is something that matters, to someone how matters, at some time
• Quality is the absence of unnecessary friction (StooCrock)

It’s probably best illustrated and articulated by Dan Ashby: “Testing is an investigatory activity (exploration) that has the effect of uncovering more information“. Good testing compared to QA above is to explore what the customer really wanted – and what trade-offs she has.

The expert says the 5 bigger pieces should be in the build – though the customer is OK.

More Than a Functional/Non-Functional Split

The world is not binary – every time we look closer at a binary split it becomes more fuzzy. Even the computer bit 0/1 is a simplification of low and high voltage. The historic idea of testing frameworks to have either functional or non-functional requirements in focus is failing us similarly. There is much more to testing activities than both functional and non-functional requirements.

Compliance

One of the testing activities that fail to be recognized in the functional/non-functional split, is compliance requirements. US SARBOX and EU GDPR were just the start, on the horizon are EU NIS2, US FISMA, WCAG, and others. WCAG deals with accessibility for websites and is a good example of a compliance requirement that is part functional (dictates functionality: alt-text, screen reading) and part non-functional (dictates contrasts and navigational aids). Many emerging governmental regulations are around broader security controls and information security. To be compliant your organization has to do secure application development – the requirement is to your organization rather than the product. Yet the requirement still has to be met – the business “license to operate” might depend on it.

Is Compliance on Your Strategic Map? [me, on LinkedIn for a change]

Operational Readiness and Operation Trials

I often work with solutions that are outsourced to where I work. There might even be a consortium (collection) of vendors in the mix for building the thing. These projects have activities called “Transitional Trials,” “Operational Trials,” or “Operational Readiness Tests” [I also wrote about this in 2017]. The terms seem to come from a more physical form of delivery. These trials fall under my hat in my role as test manager/test lead aka “project manager of the testing and trial activities“.

Operational Trail is often a 30 days period with the solution in full operations with live users, where it’s confirmed that the whole ecosystem of the solution is operational and follows the service level agreements and ITIL processes. The “test cases” of these phases could be:

• Perform a penetration test
• Perform backup and recovery procedures
• Collect monthly reporting on the SLA and system response time
• Collect monthly reporting on batch processing

The requirements come from the outsourcing contract, not from a specification of the system’s functional- or non-functional requirements. It’s a specific activity on its own that needs someone to manage it and write the confirming “test reports around it”. And yes a “trial” report similar to a test report is often required.

Contractual Confirmations

A last testing activity that might fall under the scope of the project test manager, that is not classically functional or non-functional is the activity of gathering evidence of contractual deliverables. the last couple of years I have been part of projects, where we had not only to deliver the system and functionally test the system – but, had also to collect evidence of the contractual deliverables.

The outsourcing contract converted its textual requirements into a huge spreadsheet of ~1000 lines. Each line had to point to objective evidence of each contractual requirement being fulfilled. Examples:

• Backup procedures, requirements for RTO/RPO
• Production data access controls
• Network security and integration protocols
• Processes for yearly financial controls

This could also be a testing/trials activity, as we would create a plan document of the scope, List the items under test, confirm each item by finding evidence, fix bugs, and finally collect a report.

To sum up

These three testing activities are about the testing – not the testers. The actual performance of the controls might be done by a range of very different subject matter experts. If any of the above testing activities fails to be handled – the business is equally at risk as if it was a functional or non-functional requirement failing.

Don’t Overthink Test Cases

Test cases have been a large element of testing activities for the last 20+ years. While they are representations of testing activities, having a strict regime around their format backfires into trust issues and bureaucracy. Elaborate test case specifications go against the primary purpose of the testing activity they are to support.

Here are three specific situations where too much thinking has been put into the test cases:

Project S: To enable a new feature in the solution the database needs to be remodeled, adding a new table and moving data around. We prepare a database migration script and a verification script. The purpose of the verification script is to test that the data are in the right places. We also do some regression testing and performance testing. To have a complete overview and transparency of the testing activity I add all three “types” to our test case management platform. But the verification script test case is mostly a placeholder and will not have the same characteristics as classical manual test cases wrt steps, test data, and expected results.

Project T: As part of an outsourcing agreement the customer requires approval of all system documentation, and whether they have been updated. It’s a list of potentially hundreds of documents that are listed, with each their approval status and metadata. While each of them could be a “test case” in a test case management tool, none of the document verifications would match the properties of functional test cases: requirement coverage, expected results conditions, input, user roles, etc. It’s still a test activity, albeit a static one.

Project V: Building an API platform using high-level cloud components – most of the features can be confirmed using test automation using an API testing framework. Plenty of guides exist on how to automate API testing coding positive, negative, and similarly technically exploring the API by automated payloads. We use MS Azure DevOps and can integrate test automation into the CI/CD pipelines. Why do we need high-level test cases to verify the same as the automation?

A Question of Trust

One of the pitfalls of test cases is that they are countable and provide a form of structure, and any form of structure feels safer. We mistake countable formalized work items as value. While there is some truth to moving elements to a more explicit form, spending the focus on sub-optimizing the content is a misguided form of value creation. Additionally, discussions mostly on form and attributes lead to mistrust between the parties – not a collaborative partnership.

Supporting the Key Purpose

The key purpose of the testing in all three projects above is to confirm expectations in a consistent form and provide objective evidence. The world has moved since formatted test cases, and objective results can be provided in many media forms: video, screenshots, run results, etc. What is needed is a repository of facts and an aggregated overview.

Some would probably argue that everything and every little detail is equally important to them. I’m sure that when we explore this a bit, it turns out that the delivery of software is prioritized over comprehensive documentation. In most cases. Mean Time To Repair is key for the stakeholder’s willingness to take risks.

Business results are better supported if the testing can be repeated consistently – ie. it’s automated. Go read Accelerate wrt supporting business needs with DevOps practices. DevOps practices cover the things that can be automated: You should automate 100% of the tests that should be automated.

Plenty of publications have been made for the value of more exploratory testing to explore the unknown and unarticulated expected results (aka requirements).

To me, the best test cases are representatives of a testing activity that cannot be automated. They go in folders of folders for organization. Each has a unique ID, a title, a state (ToDo>Doing>Done would suffice), a responsible, and a description field. Attachments, tags, and references to other work products are nice – everything else is just overhead. Don’t overthink it.

Resources for Test Strategy Mapping

Subscribe to continue reading

Subscribe to get access to the rest of this post and other subscriber-only content.

Already a subscriber?

Teaser – A Guidebook to Leading Testing

While my first book was on the more advanced topic of visual test strategies my next book will be an entry-level guide to people moving into test management and test leadership roles – primarily for all those without a background in testing. But also for testers moving into leadership. as Meg (@castusflamingo) has asked: Folks who moved into test leadership roles – what do you wish you could go back and tell yourself before you started?

I have been playing with the idea of it being a guidebook to these destinations:

• The Sea of Testing Know-How
• The Community Deep
• The Pearly Personalities
• The Leadership Lands
• Management meadows
• Shallow Shores
• Project roadways
• Transition highways
• Agile release trains
• Artifact wetlands
• Document pitfalls
• The Certification Wastelands
• Rivers of Tenacity
• And the various tool towns scattered across the land

Which parts of the testing world would you need a guidebook for? Let’s see how it goes – the voice and style will more likely be similar to the first. Currently, the introduction starts with the following verse

Once upon a time, 
our protagonist set out on a quest to lead a testing activity.
While they searched and searched across the world,
they found no clear map to guide their quest.
And there, my friend, is where this story begins.

But do sign up for alerts here: https://leanpub.com/ltabook/

Applying Accelerate as a Strategy Legend

TL;DR: What I learned from applying the research from the book Accelerate (2018) by Forsgren et.al. as a strategy legend on my Wardley Strategy Map. Spoiler alert – it’s more about people than technology.

I use Wardely Maps to read my projects’ context and identify a test strategy that aligns with stakeholder needs. Currently, I am doing yet another project – this one is primarily based on Infrastructure as Code (IaC) and Functions as Services (FaaS) and creating an API engine/event-driven broker. My usual starting point was my Situational Aware Test Plan template – below with the headings from my DPCG mnemonic.

	Darlings	Pets	Cattle	GUID’s
Situation	X
Delivery / Release frequency			X
– Staffing	X
– – Collaboration	X
– Environments			X
– – Tools				X
– Test scope / Verification			X
– – Test Data		X

This is not a map

This, of course, is not a map, as I have not considered the user need, but more given each element a score based on my initial reading of the project. It’s a new project team that hasn’t worked together before – building a new system, that does not yet exist similarly. That we know of – so it’s Cynefin Complex domain. The tech stack and testing challenges are well-known and can be mostly automated.

There is probably a natural order to the items above: It’s a new situation (for the customer too) and a release is needed. the delivery needs staffing – staffing needs collaboration and tools. Delivery (releases) needs environments and verification and verification needs test data. The table is neither a strategy as I have not considered which items to move – but I had an idea.

Research From Accelerate

ACCELERATE – The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations by NICOLE FORSGREN, JEZ HUMBLE, and GENE KIM (2018) identified a key set of practices to optimize for organizational performance and a great delivery culture:

Saved you a click: https://itrevolution.com/book/accelerate/

If we zoom in, we can see the prerequisites for Continuous Delivery to be successful:

• Test Automation
• Deployment Automation
• Trunk-based Development
• Shift Left on security
• Loosely Coupled Architecture
• Empowered teams
• Continuous Integration
• Version Control
• Test Data management
• Monitoring
• Proactive system notifications

These items are just in the order from the book – but I can similarly score our ability and tools. Because of the platform, most are a commodity or best practices: Loosely Coupled Architecture, version control, CI/CD, trunk-based development, and deployment automation. A few require a bit of a push from possibility to active use (proactive system notifications, monitoring). The sore spots are (as detailed in 24 Key Capabilities to Drive Improvement in Software Delivery):

• for the development team to own test automation: Test automation is a practice where software tests are run automatically (not manually) continuously throughout the development process. Effective test suites are reliable—that is, tests find real failures and only pass releasable code. Note that developers should be primarily responsible for the creation and maintenance of automated test suites. 
• establish test data management: Test data requires careful maintenance, and test data management is becoming an increasingly important part of automated testing. Effective practices include having adequate data to run your test suite, the ability to acquire necessary data on demand, the ability to condition your test data in your pipeline, and the data not limiting the number of tests you can run. We do caution, however, that teams should minimize, whenever possible, the amount of test data needed to run automated tests. 
• Empowered teams: Research shows that teams that can choose which tools to use do better at continuous delivery and, in turn, drive better software development and delivery performance. No one knows better than practitioners what they need to be effective.

The Test Strategy

Hence my focus as a test manager would obviously be the first two. Let me be the first to move for test automation by the developers (the domain for the test is mostly API endpoints) and the first to ask for killing the test environment so that we can learn to bring it up again – and again.

But because I care – and because I have read in the book what is a requirement for the listed team capabilities above – I should be mindful to consider also empowering teams and encouraging transformational leadership: Vision, Inspirational Communication, Supportive Leadership, Intellectual Stimulation, and Personal Recognition.

Using the research from Accelerate as a mapping legend for the team’s Continuous Delivery Practices I can see that even with state-of-the-art tooling and a modern high-level stack – it’s a people problem first of all – and thus the test strategy is threefold:

1. Encourage testing by the developers (evolve test automation)
2. Prioritize the construction of stubs and drivers for test data (evolve test data)
3. Encourage collaboration, learning, and a supporting culture (evolve culture)