Many test processes and test tools mention that you have to establish the expected results to every test. It is at best risk ignorance not to take the notion of expected results with a kilo of salt #YMMV.
If you can establish the result of your solution in a deterministic, algorithmic and consistent way in a non-trivial problem, then you can solve the halting problem. But I doubt your requirements are trivial.. or even always right. Even the best business analyst or subject matter expert may be wrong. Your best oracle may fail too. Or you may be focussing on only getting what you measure.
When working with validation in seemingly very controlled environments changes and rework happens a lot, as every new finding needs the document trail back to square one.. Stuff happens. Validation is not testing, but looking to show that the program did work as requested at some time. It is a race towards the lowest common denominator. IT suppliers can do better that just to look for “as designed” .
Still the Cynefin framework illustrates that there are repeatable and known projects, and in those contexts you should probably focus on looking to check as many as the simple binary questions in a tool supported way, and work on the open questions for your testing.
Speaking of open ends – every time I see an explicit expected result I tend to add the following disclaimer (song to the tune of nothing else matters ):
And nothing odd happens … that I know of … on my machine, in this situation 
And odd is something that may harm my user, business or program result
But I’d rather skip this test step and work on the description of the test and guidelines to mention:
- Think! Use your head – use what you know.
- You are the expert.
- If it smells fishy, there is something fishy going on
- Wonder, Wander, Galumph if you may
- Analyse, consider,.. if you get an idea, try it!
- Learn, Explore, and by all means test
See also: The unknown unknown unexpressed expectations, Eating wicked problems for breakfast
1: Anyone can beat us, unless we are the best, todays innovation becomes tomorrows commodity
3: I’ve heard that somewhere…
8 thoughts on “The Expected”
[…] always a threshold on what you want to fix. What sets you apart from others is HOW you handle the unexpected issues you DO […]
[…] Amongst my secret weapons are intuition, square lashings, preparing for the unexpected, […]
[…] of testing, v-model, and only one book of testing the ISEB (later ISTQB) vocabulary. And only one expected output of testing: Testers designed test cases, executed and perhaps wrote both a test plan […]
[…] Bits under the Bridge, Less Software, more Testing, Test Criteria for Outsourced Software, The Expected, Fell in the trap of […]
[…] about how to deliver quality. The other side held towards IEEE definition of delivering the expected. But even when he did, he failed to see that the unmeasured and irrational parts affected the […]
[…] to the specifications and codified knowledge, but to use imagination and implicit knowledge to imagine the unexpected. Testers, to me, are anyone performing the act. Be it developers, technical application […]
[…] is ever changing. Coming up around the corner is a new change – some can be planned for and others not so much. Imagining that things can be different is the first step to dealing with change (Thank you, […]
[…] results in test cases. It seems the topic is still relevant – I wrote about it in “The Expected” back in […]