I wonder if

I wonder if… the Norwegian and Swedish texts are correct on this picture:

2015-10-29 21.38.35

I wonder if is surely among the things that I as a tester say  or think a lot. You will also hear me cheer when we find a critical bug. Every defect / bug / observation  / issue / problem / incident we find is our chance to learn about the product. It’s a natural part of the game to find things and then to handle them. Defer them if so inclined, mitigate the risks, fix the bugs, update the  processes – but always take a decision based on the new knowledge you have.

awesome

Here are some other things I often say:

revert_thatsodd  strange

Originally at the Ministry of Testing Facebook page,  but the twists above are mine.

Advertisements

The Expected

Many test processes and test tools mention that you have to establish the expected results to every test. It is at best risk ignorance not to take the notion of expected results with a kilo of salt #YMMV.

If you can establish the result of your solution in a deterministic, algorithmic and consistent way in a non-trivial problem, then you can solve the halting problem. But I doubt your requirements are trivial.. or even always right. Even the best business analyst or subject matter expert may be wrong. Your best oracle may fail too.  Or you may be focussing on only getting what you measure.

Nailed it
Nailed it

When working with validation in seemingly very controlled environments changes and rework happens a lot, as every new finding needs the document trail back to square one.. Stuff happens. Validation is not testing, but looking to show that the program did work as requested at some time. It is a race towards the lowest common denominator. IT suppliers can do better that just to look for “as designed” [1].

Still the Cynefin framework illustrates that there are repeatable and known projects, and in those contexts you should probably focus on looking to check as many as the simple binary questions in a tool supported way, and work on the open questions for your testing.

Speaking of open ends – every time I see an explicit expected result I tend to add the following disclaimer inspired by Michael Bolton (song to the tune of nothing else matters [2]):

And nothing odd happens … that I know of … on my machine, in this situation [3]

And odd is something that may harm my user, business or program result

Significantly…

But I’d rather skip this test step  and work on the description of the test and guidelines to mention:

And now to something completely different:

See also: The unknown unknown unexpressed expectationsEating wicked problems for breakfast

1: Anyone can beat us, unless we are the besttodays innovation becomes tomorrows commodity

2: https://www.youtube.com/watch?v=tAGnKpE4NCI 

3: I’ve heard that somewhere…

Can you see beyond the visible

As mentioned in “Diversity is important in testing” my view is that the best testers are those that know that testing can be done in many ways. “Testing practices appropriate to the first project will fail in the second. Practices appropriate to the second project would be criminally negligent in the first.” ( http://context-driven-testing.com/) There is no “one way to do testing” (despite what standards tells you). Granted there are – in context –  best practices.

The best testers are those that can see beyond the current project and company framework. Those that realize that there is a fundamental difference between life-science validation and modern enterprise IT projects – and for agile projects even more. If the company frameworks fails to keep current and allow clear tailoring, then “life finds a way“.

There will be contexts where UX is not very interesting, where there is no software as such, where they release directly to production (so what we have TitW). There will even be contexts where structured software testing has very little business value. As well as, there will be contexts, where it’s one-shot only and testing and dress-rehearsals are done, over and over again. (consider though that for space launches superstition and good-luck charms play a very large role).

But don’t confuse your one context and what you have seen in some domains to be directly applicable in others. See beyond the visible, extrapolate your testing knowledge and approaches for different contexts, and you are the better tester.

Iceberg Factory. Torsuut Tunoq Sound and Kulusuk Island. Southeastern Greenland. http://www.mikereyfman.com/

Read for your kids – special interest edition

If you are a parent to (early) school children you should know that it is important to read  to your kids. Reading the words out trains vocabulary, recognition, imagination, wondering etc etc. So I read subtitles from movies… because

The boys currently have Star Wars as their special interest [1], and wanted to see the “people” movies. The have played the scenes via the LEGO Video Games (GC) and have a range of the LEGO sets – so they had the basic plot already. Feature movies like Star Wars are usually subtitled in Denmark – while animation movies are dubbed [2]. So in order both to keep up with “PG” [3] and helping them read the titles – I get to watch the movies and read the subtitles…

Poor daddy, it’s almost as hard as when he has to finish the ice cream they can’t 😉

In the last months the (soon to be) 9yo have cracked the reading code and have gone from LIX11 books to the shorter subtitles. The 11yo have rest covered, but some of the longer texts are tricky (I’m looking at you – opening Scroll).

2015-04-04 16.51.08

I tried reading Harry Potter (in Danish) but even if the story was very elaborate and detailed it didn’t catch their interest. Neither did classics from when I was a kid (Sorry Bjarne Reuter), so I had to rethink the acceptance criteria for “read for your kids“.

See these two boys are not as easily motivated – it has to tie into something they can see a direct interest in. Their autism makes them very picky on the choice of subject. What I try is to meet them where they are, expand their competencies and give them a lot of positive feedback until they master it on their own.

Links: The yardstick of mythical normalityAcceptance is more than what can be measured

  1. special interest, as in overly dedicated into the topic and cannot talk about anything else.
  2. The Danish “dubbers” are usually world class, luckily.
  3. Episode 3 is still to come, though.

Asking Open Questions

It has always been a good interview technique to ask open questions. Then the person being interviewed have to elaborate and talk in full sentences. In contrast to closed questions, that replied to in binary [1]: yes, no, 42 – the red pill [2]. Until now I really didn’t understand how simple yet powerful this questioning technique is in testing. I might have done it all along, for some time :-).

The primary eye opener was the Copenhagen Context 2015 [4] workshop on Exploration Under Pressure by Jon Bach. One of the treats was that he showed us a list of things to find on the ebay.com website. Not specific items, but information about the items. Finding the most expensive item, and by that stumbling over a live production bug in the max value field. Finding the number of blue shoes available etc. What a fun “online scavenger hunt” – we could battle to find the oldest, longest and most odd details etc.

Later the same week eBay Classified hosted a local meetup of “QA Aarhus” with a live demo of how they do testing sessions of their app. They had to host the session twice,  due to popular demand, and what we got was an intro to a setting of exploration, thinking loud and doing pair testing. And I got to try my new-found quest to ask open questions. To search for things – but look out of the corner of the eye for oddities and what-ifs.

But how could I apply this technique in my current testing project of migrating an HR solution for a large IT outsourcing company. I did today. A staff member allocated to the project to test during UAT [3] specifically the processes they use in the old system and to act distribute this knowledge back to the team. For reasons the testing scope in this are had yet not been established, so she didn’t really know where to start – but I did… open questions 

  • What processes do you have?
  • What kind of events do you need to register on an employee
  • Tell me more about vacation calculation
  • Where, if any, are your current processes described (I’m fallible)
  • What has likely changed comparing the old and new solution

I asked her to go as deep until no new learning could be achieved, but not to detail it in scripts or discrete steps. Because from here we have test cases – test ideas – “a question that someone would like to ask (and presumably answer) about a program

Eureka!

 

[1]: Binary replies can be checked, open questions are testing. Testing is “Testing is the process of evaluating a product by learning about it through exploration and experimentation, which includes: questioning, study, modeling, observation and inference, output checking, etc.” http://www.satisfice.com/blog/archives/1509

[2]: I have seen how deep the rabbit hole goes…

[3] Let’s pretend there is such a thing as a “user acceptance test

[4]  Disclaimer: I was part of the program committee, and by chance most speakers hosts their own testing conferences. See more on http://copenhagencontext.com/blog/2015/01/meet-jesper-at-the-copenhagen-context-conference-venue/

Lego Role Models for Girls

Who had the family’s largest LEGO set his Christmas – not the boys (age 8-10), neither the “boys” (age 40 and up) – it wasn’t me* – but the 11-year-old girl and her 8 wheel 42008 Service Truck – 1276 pieces, power functions, pneumatic, gears and 44 cm forcefulness. There was no boy band merchandize, no glitter or similar gender framing. Quite a project – as is the story about the “Research Institute” mini-figure set.

42008-121110 Continue reading

FDA, Exploration and time to information

A key driver in implementing enterprise knowledge management is to reduce time to information (77% are seeing faster access to knowledge). But that goes for LinkedIn and Twitter too. Using twitter professionally helps you meet the famous people and help you see the communication layers at conferences. Case story: Today I was reading about test processes in a regulated environment, and got curious towards exploratory testing in that context. So I reached out to the #twitterbrain and asked the giants, whose shoulders I am standing on*:

  • Griffin Jones ‏@Griff0Jones, help clients struggling with regulatory compliance and context-driven software testing problems.
    • CAST 2011: Cast 2011 What Do Auditors Expect From Testers
    • What is good evidence – Let’s Test 2013
    • WREST – Workshop on REgulated Software Testing
  • Johan Åtting ‏@JohanAtting Chief Quality Officer -atSectra’s Medical Operation
    • turned the testing from a traditional scripted approach into a context driven approach and introduced exploratory testing.
    • ensuring that the company are regulatory compliant with e.g. FDA, MDD, CMDR, ISO13485, ISO14971 etc.
  • James Christie ‏@james_christie
    •  interested in testing’s relationship with audit and governance.
    • dedicated to the audit, control, and security of information systems.
  • Michael Bolton ‏@michaelbolton Program Chair of EuroStar 2013 and key guru pointed me to an article by James Marcus Bach ‏@jamesmarcusbach on the topic.
  • Keith Klain ‏@KeithKlain Head of Barclays Test Service retweeted on the fallacy on the evidence of scripted testing.
  • Claire Moss ‏@aclairefication (my favorite retweeter) and many others retweeted

Within 2 hours I had both relevant references, a debate on the pitfalls and base for further details. Follow the tread of this tweet: https://twitter.com/jlottosen/status/411473074312052736 

*: really, not to brag – I have met both Griffin, Johan and James, and they know me too 🙂