Don’t Overthink Test Cases

Test cases have been a large element of testing activities for the last 20+ years. While they are representations of testing activities, having a strict regime around their format backfires into trust issues and bureaucracy. Elaborate test case specifications go against the primary purpose of the testing activity they are to support.

Here are three specific situations where too much thinking has been put into the test cases:

Project S: To enable a new feature in the solution the database needs to be remodeled, adding a new table and moving data around. We prepare a database migration script and a verification script. The purpose of the verification script is to test that the data are in the right places. We also do some regression testing and performance testing. To have a complete overview and transparency of the testing activity I add all three “types” to our test case management platform. But the verification script test case is mostly a placeholder and will not have the same characteristics as classical manual test cases wrt steps, test data, and expected results.

Project T: As part of an outsourcing agreement the customer requires approval of all system documentation, and whether they have been updated. It’s a list of potentially hundreds of documents that are listed, with each their approval status and metadata. While each of them could be a “test case” in a test case management tool, none of the document verifications would match the properties of functional test cases: requirement coverage, expected results conditions, input, user roles, etc. It’s still a test activity, albeit a static one.

Project V: Building an API platform using high-level cloud components – most of the features can be confirmed using test automation using an API testing framework. Plenty of guides exist on how to automate API testing coding positive, negative, and similarly technically exploring the API by automated payloads. We use MS Azure DevOps and can integrate test automation into the CI/CD pipelines. Why do we need high-level test cases to verify the same as the automation?

A Question of Trust

One of the pitfalls of test cases is that they are countable and provide a form of structure, and any form of structure feels safer. We mistake countable formalized work items as value. While there is some truth to moving elements to a more explicit form, spending the focus on sub-optimizing the content is a misguided form of value creation. Additionally, discussions mostly on form and attributes lead to mistrust between the parties – not a collaborative partnership.

Supporting the Key Purpose

The key purpose of the testing in all three projects above is to confirm expectations in a consistent form and provide objective evidence. The world has moved since formatted test cases, and objective results can be provided in many media forms: video, screenshots, run results, etc. What is needed is a repository of facts and an aggregated overview.

Some would probably argue that everything and every little detail is equally important to them. I’m sure that when we explore this a bit, it turns out that the delivery of software is prioritized over comprehensive documentation. In most cases. Mean Time To Repair is key for the stakeholder’s willingness to take risks.

Business results are better supported if the testing can be repeated consistently – ie. it’s automated. Go read Accelerate wrt supporting business needs with DevOps practices. DevOps practices cover the things that can be automated: You should automate 100% of the tests that should be automated.

Plenty of publications have been made for the value of more exploratory testing to explore the unknown and unarticulated expected results (aka requirements).

To me, the best test cases are representatives of a testing activity that cannot be automated. They go in folders of folders for organization. Each has a unique ID, a title, a state (ToDo>Doing>Done would suffice), a responsible, and a description field. Attachments, tags, and references to other work products are nice – everything else is just overhead. Don’t overthink it.