Besides the visualization of the delivery pipeline it can be a good idea to visualize your quality model. There’s more to it than V-model, Agile/Scrum and other old models. The following models are at least as established and practice based as any old book …
Continue readingrapidsoftwaretesting
Who is the tester?
In my current and primary projects the testing is not done by software testing professionals – and it’s probably for the better too! It is in contexts like these:
- A Microsoft Dynamics “D365O” implementation of health registration forms. Tested by public service clerks partly comparing to the previous solution, partly testing the new system platform.
- Moving 700+ servers running 50+ applications from one data center to another while keeping everything from mainframe to SaaS integrations live. Tested by the application staff that have maintained the system since for ever (10+ years).
- Implement at standard commercial off-the-shelf tool for 2000+ IT savvy users. To most users this tool is their primary work tracking system, so they get to test it too.
In contexts like these the act of testing done by subject matter experts of the field – infrastructure specialists, public service clerks, support staff, application developers and the like. These persons qualify as the “customer” in the Modern Testing Principle that “the customer is the only one capable to judge and evaluate the quality of our product“. They might have a testing /role/ during the project, but that is because of their high domain knowledge, but at the end of the project they continue with their “real business job” of using the system to produce stuff for the business.
It’s not their job to know ISTQB from “MT Principles” and “RST methodology“. That is up to me, as the manager of the testing. My role is more and more about the guidelines for the testing and the facilitation of the people doing the testing. My reach goes so far as to ask them to think about how the product fails and succeeds. But I cannot expect them to know checking from testing.
Long gone are the days of managing testers that put all their skill into the niches of the testing craft. There are less software testing professionals doing the testing in projects like the above. Part of it is, that the describing the whole system explicitly is simply to expensive in time and money. This makes the requirements inherently fuzzy and undefined. And part of it is that learning the skills simply takes to long. Some technical tests require skills of a certified VMware specialist, others having an eye for every unwritten tacit business rule.
Another angle is that the skills that the usual software testing specialist brings to the table is handled on a lower level. Testing is done by the organisation (like Microsoft) that builds the standard solutions and commercial off the self systems (SAP, D365O etc). Another is that the test techniques of the software testing field simply no longer applies. I mean how does boundary value analysis add value to enterprise data center transition executions, when the system under test it not even software?
The better tester is neither the software developer nor the software testing specialist. It’s the person who ponders:
- How could this go wrong…
- I wonder if…
- For this to work, we need to do…
Come to think of it, everyone in the project does that! Some do it more explicitly, some do it more experimental. Everyone evaluates how their actions add value to the people that matter (at some time).
Co-creating smarter testers
“Co-creating smarter testers” is the byline of the Ministry of Testing, a small company with a great impact that I have been following and supporting for 7 years* now . I have attended TestBash’es, webinars, challenges, discussions and memes. And now for the first time in Denmark – Anders Dinsen and I are bringing the world known Meetups to Copenhagen (Aarhus 2017 you’re next).
The topics so far are:
- Let’s make a Risk Based Testing Strategy recently had 20 participants, hosted and chaired by Anders.
- Testing roles are shifting, but where to? is planned by me (see shiftx) and hosted by a sponsor for food and location (Miracle!)
- Playful Software Testing… needs both a date and a venue.
At the first meetup we split into three groups, discussed risks and how to TEST THEM RISKS. Dearest to me was the discussion of stakeholders and new places to test. Great to see that even with very little information, we can still do a rapid testing based on business objectives. There is so much more to testing these days.
1: since a EuroStar 2010 t-shirt competition 🙂
Less Software, more Testing
I rarely test software these days. I mostly lead testing of IT solutions.
Testing in the context of:
- Updating all corporate PC’s from windows 7/8 to Windows 10
- Consolidating network equipment from more devices to one box, on 80 global locations *
- Move 40 live business applications from one data center to another *
- Take over application maintenance for a specialized public organization
- Implement track and trace for pharma products from production to shops
- Migrate HR data for 2500 people from one system platform to another
Yes, it happens that I participate in a project that is about developing a new business application, but my activities are less about testing software and more about testing in IT solutions in general.
Mostly I manage test activities and describe testing in these contexts. My preferred way of working is in setting and implementing test strategies. I prefer complex and non-ordered projects (Complex and Chaotic – I’m looking at you), it fits well with my context-driven approach of finding the “test solution that fits the context”.
Testing is in it self a solution, that must solve a business problem. Great testing is all about providing information to the stakeholders. I don’t care especially if this is done by someone TESTING or a TESTER. It is my responsibility to setup the testing activities (information gathering) that supports the team, faces the business & technology and challenges the product “sufficiently“.
Sometimes “sufficiently” is merely confirming and going through the motions of explicit requirement coverage. This is a special challenge to me, as I know of many effective and Rapid approaches, that could add valuable information. When I face this challenge, I try to look at the full picture of the project, and what the business want’s to achieve.
The business of the business is business. What matters is not software or projects, but the solutions to the challenges the business have. And the context of testing is similarly so much more than the software.
*: As mentioned in “How to test in IT Operations” at Nordic Testing Days 2016.
QA Aarhus – Exploratory Testing How and When
QA Network Aarhus is a local non-affiliated network of testers (and good friends) in Aarhus. Where I had the great pleasure of talking about Exploratory Testing. This is the link collection, the slides are attached.
All oracles are failable
[Software Testing Club Blog | Oct 6 2011 | myself ]
All oracles are fail-able to a certain level of confidence.
Recently I had the opportunity to participate in the acclaimed dice game. I also had the chance to be game master on a variation of the dice game session for a small test team. Reflecting on the experiences I had two considerations: (Some spoilers apply)
When are you confident enough?
The dice game is played by a loop of theories/ideas and tries/tests on the idea. The goal is to produce a theory/algorithm that can successfully predict the number that the game master presents. How many tries/tests/checks would give you confidence in the theory you have in mind? Options:
- When you successfully predict one throw. Ie you say 7, game master say 7. Do you yell “LOOSERS, Sea ya”?
- When you have 7 successfully predicted in a row? (why 7)
- All 7776 combinations of 5 pieces of 6-sided one-colored standard dices?
- Do you try every throw just once, or more times?
- Would you know if every trial number divisible by 100, the game master would say “pi” (think leap years)
The dice game seems simple, but the problem domain of even the dice game is infinite. Or at least practically infinite (7776 is practically infinite in the dice game IMHO). The number of tests doesn’t matter, but the character of the test, relative to your mental models, matters a great deal. My purpose is not to find a fixed number of tries, but to make you consider the underlying assumption on confidence levels. That is you have a confidence in your model until it fails. You are confident to the level of x successful predictions, where the x+1 prediction fails. All you know at that time is that your theory is “incomplete” (not wrong, not right) – and this calls for more learning and more ideas…
All oracles are failable
The oracle in software projects is the ressource of answers – the documents, the mindmap, the subject matter expert. In the dice game the game master is the oracle. We are humans hence failable. The physical oracles (docs, …) even more. This made me ponder:
- Would you approach the dice game differently actively knowing that the dice game master is failable?
- If the game master (aka oracle) made a deliberate error ever once in a while – would you know?
- If there is a bug (non deliberate) in the game masters algorithm would you know?
- How would you test for the oracle making mistakes?
- Do you test the dice game different if it was a human or machine oracle
- First think about the dice game being computer based
- Secondly what if it was a human behind a computer based interface
- Consider the implications of the Turing Test
- Oh, did you forget that I could make mistakes? – was that a rule, or an assumption?
The key framing of the dice game is usually a lesson in learning, in theory setting and trials of that theory – still under the underlying assumption that the game master can deal with any test that is offered. What would happen if the game master was blindfolded? What would the case if the algorithm was more complex – less humanly processable in a short time. There will always be a level of capability of the oracle, and it will fail – eventually.
Complexity is a Matter of Perspective 