Chicken for Christmas – Tradition is a choice

TL;DR: Testing is always something that we choose to do – and how we test is similarly a matter of choice. As is Christmas traditions. .. it’s just man-made rules, we can choose to change them.

So I was discussing what we should have for Christmas with the stakeholders. One of them wants the traditional rice pudding, the other wants – untraditionally: chicken. And you know what – that’s ok. Traditions are guidelines, not rules – we can make new traditions, just by choosing to.

Testing in production used to be a great no-no. I’m still feeling odd when we do it, but I have come to see it as another tool of the trade. It has a name now “testing in the wild” TTitW as has been presented at EuroStar 2015. Also this is how Netflix have been testing for years, from GitHub, as it is open source too (!).

You might argue that changing testing (in the wild) is not allowed. I will challenge that assumption – being allowed to do something is a choice too. You choose to follow the the process frameworks, requirements, rules… and you can choose not to. The tradition of manual predefined testcases are so four years ago!

Sometimes it’s just a matter of saying up-front, that you are tailoring the process. So choose an approach that actually gives meaning and value to the stakeholders and context. Deconstruct the traditions and commercial bodies of knowledge and make some new!

http://www.eurobricks.com/forum/index.php?showtopic=64241
http://www.eurobricks.com/forum/index.php?showtopic=64241

The Expected

Many test processes and test tools mention that you have to establish the expected results to every test. It is at best risk ignorance not to take the notion of expected results with a kilo of salt #YMMV.

If you can establish the result of your solution in a deterministic, algorithmic and consistent way in a non-trivial problem, then you can solve the halting problem. But I doubt your requirements are trivial.. or even always right. Even the best business analyst or subject matter expert may be wrong. Your best oracle may fail too.  Or you may be focussing on only getting what you measure.

Nailed it
Nailed it

When working with validation in seemingly very controlled environments changes and rework happens a lot, as every new finding needs the document trail back to square one.. Stuff happens. Validation is not testing, but looking to show that the program did work as requested at some time. It is a race towards the lowest common denominator. IT suppliers can do better that just to look for “as designed” [1].

Still the Cynefin framework illustrates that there are repeatable and known projects, and in those contexts you should probably focus on looking to check as many as the simple binary questions in a tool supported way, and work on the open questions for your testing.

Speaking of open ends – every time I see an explicit expected result I tend to add the following disclaimer inspired by Michael Bolton (song to the tune of nothing else matters [2]):

And nothing odd happens … that I know of … on my machine, in this situation [3]

And odd is something that may harm my user, business or program result

Significantly…

But I’d rather skip this test step  and work on the description of the test and guidelines to mention:

And now to something completely different:

See also: The unknown unknown unexpressed expectationsEating wicked problems for breakfast

1: Anyone can beat us, unless we are the besttodays innovation becomes tomorrows commodity

2: https://www.youtube.com/watch?v=tAGnKpE4NCI 

3: I’ve heard that somewhere…

Read for your kids – special interest edition

If you are a parent to (early) school children you should know that it is important to read  to your kids. Reading the words out trains vocabulary, recognition, imagination, wondering etc etc. So I read subtitles from movies… because

The boys currently have Star Wars as their special interest [1], and wanted to see the “people” movies. The have played the scenes via the LEGO Video Games (GC) and have a range of the LEGO sets – so they had the basic plot already. Feature movies like Star Wars are usually subtitled in Denmark – while animation movies are dubbed [2]. So in order both to keep up with “PG” [3] and helping them read the titles – I get to watch the movies and read the subtitles…

Poor daddy, it’s almost as hard as when he has to finish the ice cream they can’t 😉

In the last months the (soon to be) 9yo have cracked the reading code and have gone from LIX11 books to the shorter subtitles. The 11yo have rest covered, but some of the longer texts are tricky (I’m looking at you – opening Scroll).

2015-04-04 16.51.08

I tried reading Harry Potter (in Danish) but even if the story was very elaborate and detailed it didn’t catch their interest. Neither did classics from when I was a kid (Sorry Bjarne Reuter), so I had to rethink the acceptance criteria for “read for your kids“.

See these two boys are not as easily motivated – it has to tie into something they can see a direct interest in. Their autism makes them very picky on the choice of subject. What I try is to meet them where they are, expand their competencies and give them a lot of positive feedback until they master it on their own.

Links: The yardstick of mythical normalityAcceptance is more than what can be measured

  1. special interest, as in overly dedicated into the topic and cannot talk about anything else.
  2. The Danish “dubbers” are usually world class, luckily.
  3. Episode 3 is still to come, though.

Asking Open Questions

It has always been a good interview technique to ask open questions. Then the person being interviewed have to elaborate and talk in full sentences. In contrast to closed questions, that replied to in binary [1]: yes, no, 42 – the red pill [2]. Until now I really didn’t understand how simple yet powerful this questioning technique is in testing. I might have done it all along, for some time :-).

The primary eye opener was the Copenhagen Context 2015 [4] workshop on Exploration Under Pressure by Jon Bach. One of the treats was that he showed us a list of things to find on the ebay.com website. Not specific items, but information about the items. Finding the most expensive item, and by that stumbling over a live production bug in the max value field. Finding the number of blue shoes available etc. What a fun “online scavenger hunt” – we could battle to find the oldest, longest and most odd details etc.

Later the same week eBay Classified hosted a local meetup of “QA Aarhus” with a live demo of how they do testing sessions of their app. They had to host the session twice,  due to popular demand, and what we got was an intro to a setting of exploration, thinking loud and doing pair testing. And I got to try my new-found quest to ask open questions. To search for things – but look out of the corner of the eye for oddities and what-ifs.

But how could I apply this technique in my current testing project of migrating an HR solution for a large IT outsourcing company. I did today. A staff member allocated to the project to test during UAT [3] specifically the processes they use in the old system and to act distribute this knowledge back to the team. For reasons the testing scope in this are had yet not been established, so she didn’t really know where to start – but I did… open questions 

  • What processes do you have?
  • What kind of events do you need to register on an employee
  • Tell me more about vacation calculation
  • Where, if any, are your current processes described (I’m fallible)
  • What has likely changed comparing the old and new solution

I asked her to go as deep until no new learning could be achieved, but not to detail it in scripts or discrete steps. Because from here we have test cases – test ideas – “a question that someone would like to ask (and presumably answer) about a program

Eureka!

 

[1]: Binary replies can be checked, open questions are testing. Testing is “Testing is the process of evaluating a product by learning about it through exploration and experimentation, which includes: questioning, study, modeling, observation and inference, output checking, etc.” http://www.satisfice.com/blog/archives/1509

[2]: I have seen how deep the rabbit hole goes…

[3] Let’s pretend there is such a thing as a “user acceptance test

[4]  Disclaimer: I was part of the program committee, and by chance most speakers hosts their own testing conferences. See more on http://copenhagencontext.com/blog/2015/01/meet-jesper-at-the-copenhagen-context-conference-venue/

Quality comes in all shapes and sizes

Quality comes in all shapes and sizes .. like Christmas trees. This Christmas I was out selling trees at the local “shopping center ” with my oldest.Most left with a tree that satisfied the acceptance criteria – explicit as well as implicit – yet still no one came with a requirement spec…

2013-12-14 10.33.55

Heuristics from the merry christmas tree salesmen:

  • The  tree looked at first – is usually returned to and bought
  • Do A/B split testing between one or two trees
  • Too many options makes selecting even more confusing
  • One family’s reject – is another family’s perfect fit
  • Context is important – like how much room inside, how many people, how many kids
  • The closer to deadline – the less options
  • No one notices the wicked branches, when the music plays and the tree is lit
  • After christmas it doesn’t matter how picky you were with the details

A young woman came to us looking a bit puzzled – she had never bought a tree herself, and the tree been bought was not for her. All she knew was that she had volunteered to do charity help to a down-and-out family. They wanted a tree for christmas – but could not themselves. I can only guess that this specific christmas tree was the family’s perfect tree. The cost didn’t matter to the young woman at all – but the implicit value even more.

Many decisions are never about the monetary (sunk) costs. Hence your customer makes seemingly odd decisions – and that’s OK. 

See also: Acceptance criteria are more than what can be measuredLook for Minimum Viable TestingWithout Timing – Quality, Schedule and Cost is nothingValue of Information for Decisions , 16 points that may rock the boatWhen do testing happen? Are you looking too hard

Testing has to be fit for purpose too

Reading some quality process documents I found the following definition:
  • To detect and remove errors before the computer system is put into production
  • To demonstrate that the computer system is fit for intended use

But when we look at the ITIL definition of the value composition of a service – it looks fit for use, as above: follows the requirements, sufficient – what the customer gets. But also – at an equal base, it looks at fit for purpose: it has a positive effect on the business, it solves a business problem, solving the right problem. The product is a solution. If the problem isn’t solved, the product doesn’t work.

see also: Uncovering better ways  Softwaretesting is only dead, if it stands still I didn’t open it

Uncovering better ways

I am uncovering better ways of developing solutions – by doing it and helping others do it. Through this work I have come to value:

  • Apply the costs to add business value – over cutting costs
  • Being flexible and open  – over adding predictability 
  • Providing information for decisions – over ensuring the reliability

Continue reading

On applying a single method

Simon Wardley, Leading Edge ForumOh not again – should we be an agile or six sigma shop? ]

First, every system of any reasonable scale consists of multiple components. Those components are all evolving (due to the effects of competition) and you can (and should) map out the components of any system before embarking on trying to build it. Below in figure 1 is a basic map from a heavy engineering project with a large IT component

Now, all the components are evolving from left to right and as they do so their characteristics change – they move from an uncharted space (the novel, the chaotic, unpredictable, uncertain, potential differential) to the more industrialised (the common, appearance of linear order, the predictable, the certain, the cost of doing business). 

The same thing with your testing – that is: If you dare to take a holistic approach and not only focus on the mechanics. See also:  Mapping testing Competencies , Learn to think like a businessWhen do testing happen? 3D model for testing contexts Black or white – it is the same box

Come join the Context Banquet

We are setting up Dansk Workshop on Exploratory Testing (again) inviting testing people to join the banquet. We all test the software, the hardware, the context, the project, the environment the performance, – to give the decision makers information. We need testing people to have all kinds of backgrounds

| This is what the C-D-T is about |  Also know as the exploratory testing and the muh muh |  further more know as a jam |

Let me tell out about the invitation to the Context Driven banquet in another way: There once was this man who had a great fortune and many talents – and wanted to celebrate. He walked to his friends to invite them to come for the fiesta. One friend just landed a new job – but she was not yet as equal as the others. Another just had a new minivan, and had to test drive it (pun intended). But there where room for more at the banquet – so the invitation is sent to the insecure, to the un-educated, to the start-ups and the emerging thinkers, to the bloggers and the twitters, the black swans, the unicorns and the dancing monkeys….

Look for Minimum Viable Testing

How much critical mass will the product/project/service need to allow for (software) testing?

Recently I participated in a local “coffee shop meet-up” along with photographers, coaches, entrepreneurs and start-ups. We could agree that coaches as well as testers give indirect value to the business – but while staff coaching could be individually sold to carpenters and hairdressers – (software) testing could not. Afterwards I challenged my self to think otherwise!

Good testing is an information and exploration activity – to find risks and present information to the stakeholders. Usually it’s easy to find the known risks to entrepreneurs – but good testing can test for unknown unknowns – even if there is no product or “just” a minimum viable product:

The minimum viable product is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort.” The definition’s use of the words maximum and minimum means it is decidedly not formulaic. It requires judgment to figure out, for any given context, what MVP makes sense.

Similarly minimum viable testing is an effort, that allows the team to collect tacit and explicit learning about the solution, given the context. Go look for it in your context – Testing can add business value to any project state

tractor

Related: Acceptance is more than what can be measured , You call that testing – how can that add value,