The Expected

Many test processes and test tools mention that you have to establish the expected results to every test. It is at best risk ignorance not to take the notion of expected results with a kilo of salt #YMMV.

If you can establish the result of your solution in a deterministic, algorithmic and consistent way in a non-trivial problem, then you can solve the halting problem. But I doubt your requirements are trivial.. or even always right. Even the best business analyst or subject matter expert may be wrong. Your best oracle may fail too.  Or you may be focussing on only getting what you measure.

Nailed it
Nailed it

When working with validation in seemingly very controlled environments changes and rework happens a lot, as every new finding needs the document trail back to square one.. Stuff happens. Validation is not testing, but looking to show that the program did work as requested at some time. It is a race towards the lowest common denominator. IT suppliers can do better that just to look for “as designed” [1].

Still the Cynefin framework illustrates that there are repeatable and known projects, and in those contexts you should probably focus on looking to check as many as the simple binary questions in a tool supported way, and work on the open questions for your testing.

Speaking of open ends – every time I see an explicit expected result I tend to add the following disclaimer inspired by Michael Bolton (song to the tune of nothing else matters [2]):

And nothing odd happens … that I know of … on my machine, in this situation [3]

And odd is something that may harm my user, business or program result

Significantly…

But I’d rather skip this test step  and work on the description of the test and guidelines to mention:

And now to something completely different:

See also: The unknown unknown unexpressed expectationsEating wicked problems for breakfast

1: Anyone can beat us, unless we are the besttodays innovation becomes tomorrows commodity

2: https://www.youtube.com/watch?v=tAGnKpE4NCI 

3: I’ve heard that somewhere…

Read for your kids – special interest edition

If you are a parent to (early) school children you should know that it is important to read  to your kids. Reading the words out trains vocabulary, recognition, imagination, wondering etc etc. So I read subtitles from movies… because

The boys currently have Star Wars as their special interest [1], and wanted to see the “people” movies. The have played the scenes via the LEGO Video Games (GC) and have a range of the LEGO sets – so they had the basic plot already. Feature movies like Star Wars are usually subtitled in Denmark – while animation movies are dubbed [2]. So in order both to keep up with “PG” [3] and helping them read the titles – I get to watch the movies and read the subtitles…

Poor daddy, it’s almost as hard as when he has to finish the ice cream they can’t 😉

In the last months the (soon to be) 9yo have cracked the reading code and have gone from LIX11 books to the shorter subtitles. The 11yo have rest covered, but some of the longer texts are tricky (I’m looking at you – opening Scroll).

2015-04-04 16.51.08

I tried reading Harry Potter (in Danish) but even if the story was very elaborate and detailed it didn’t catch their interest. Neither did classics from when I was a kid (Sorry Bjarne Reuter), so I had to rethink the acceptance criteria for “read for your kids“.

See these two boys are not as easily motivated – it has to tie into something they can see a direct interest in. Their autism makes them very picky on the choice of subject. What I try is to meet them where they are, expand their competencies and give them a lot of positive feedback until they master it on their own.

Links: The yardstick of mythical normalityAcceptance is more than what can be measured

  1. special interest, as in overly dedicated into the topic and cannot talk about anything else.
  2. The Danish “dubbers” are usually world class, luckily.
  3. Episode 3 is still to come, though.

Lego Role Models for Girls

Who had the family’s largest LEGO set his Christmas – not the boys (age 8-10), neither the “boys” (age 40 and up) – it wasn’t me* – but the 11-year-old girl and her 8 wheel 42008 Service Truck – 1276 pieces, power functions, pneumatic, gears and 44 cm forcefulness. There was no boy band merchandize, no glitter or similar gender framing. Quite a project – as is the story about the “Research Institute” mini-figure set.

42008-121110 Continue reading

Use case 115: It was a dark and stormy night

Discussing relevance of testcases, user stories and requirements is an age-old challenge in IT development. Sometimes we think we know the usage of our software so much better, than the users – that we laugh and say: That would never be the case. But it may very well be.

The reason for undertaking the largest national construction project is so that the capital region can get fresh milk.” That’s what the minister of transportation said [1] – and boy we laughed. Why would we invest billions, 7 years and 18 km bridge so that one part of the country could supply fresh milk for the other (that had it’s own dairies).

A commercial[2] for a dairy snack (oh the irony) later alleged that this decision was made on an empty stomach[3]. But it wasn’t – with regular ferry service since 1883, the people wanted to cut the time from 90 minutes to 15 minutes, with all the added benefits of increased trade, travel and traffic.

The link opened in 1998 and a stormy night in 2006 the bridge closed for traffic. No big deal – it happens. It so happened that it closed for 22 hours. And hence the ecological milk dairy on the ”countryside” couldn’t deliver milk for the ”capital” side [4]. And the scenario from the minister of transportation had become no laughing matter.

Your user is not you http://www.developsense.com/blog/2013/12/your-user-is-not-you/

The baristas wept as there was no ecological skimmed milk

1: DK video: http://larslars.net/blog/2009/04/derfor-fik-vi-storeb%C3%A6ltsbroen/
2: Similar to this one http://www.youtube.com/watch?v=b2UrushQ86I
3: Why being hungry is bad for decision-making http://blog.bufferapp.com/8-things-you-dont-know-are-affecting-your-decisions-every-day
4: DK text: http://www.landbrugsavisen.dk/Landbrugsavisen/2006/5/26/Ingen+frisk+maelk+over+Storebaelt.htm

Quality comes in all shapes and sizes

Quality comes in all shapes and sizes .. like Christmas trees. This Christmas I was out selling trees at the local “shopping center ” with my oldest.Most left with a tree that satisfied the acceptance criteria – explicit as well as implicit – yet still no one came with a requirement spec…

2013-12-14 10.33.55

Heuristics from the merry christmas tree salesmen:

  • The  tree looked at first – is usually returned to and bought
  • Do A/B split testing between one or two trees
  • Too many options makes selecting even more confusing
  • One family’s reject – is another family’s perfect fit
  • Context is important – like how much room inside, how many people, how many kids
  • The closer to deadline – the less options
  • No one notices the wicked branches, when the music plays and the tree is lit
  • After christmas it doesn’t matter how picky you were with the details

A young woman came to us looking a bit puzzled – she had never bought a tree herself, and the tree been bought was not for her. All she knew was that she had volunteered to do charity help to a down-and-out family. They wanted a tree for christmas – but could not themselves. I can only guess that this specific christmas tree was the family’s perfect tree. The cost didn’t matter to the young woman at all – but the implicit value even more.

Many decisions are never about the monetary (sunk) costs. Hence your customer makes seemingly odd decisions – and that’s OK. 

See also: Acceptance criteria are more than what can be measuredLook for Minimum Viable TestingWithout Timing – Quality, Schedule and Cost is nothingValue of Information for Decisions , 16 points that may rock the boatWhen do testing happen? Are you looking too hard

Stakeholders

Motivated by LEGO, Pasta with ketchup, DR Ramasjang Rally – as other boys

Yet with autism (both, as in official  ICD-10 and DSM-IV). They could have been placed on a side track. They could be educated and trained to know that structure and predictability is the known world. But they are too curious, communicative and smart…. #methinks 🙂

We (ABA) train them to be able to deal with change, unpredictability and the benefits of both direct and intrinsic motivationBecause they benefit from it and it helps them being accepted and included.

Related: DK om at udsætte sine behov,  Weekend formulaThat’s what friends are forThe 860 kcal bug, will work for LEGOThe yardstick of mythical normality acceptance is more than can be measured 

I killed an IT department

It wasn’t just a glitch or a bug, or a wicked hack. It is gone – there is no IT department anymore … Staffing and services will be transferred to the communications & knowledge department, but the hardcore business of developing IT solutions is closing. From now on we primarily use customization and configuration of standard tools: Facebook, LinkedIn, WordPress, Podio and email (sigh – still).

Yet IT is everything and everything is IT

Continue reading

Pragmatic choices of what is important and possible

[ Master thesis about fathers to a child with autism ]

To have a child with autism is a continuing and constantly changing process of experience. The fathers collect experiences along the way with this different family-life, which did not turn out as they expected it  to. They live a pressured and unpredictable daily life, of which they use metaphors such as “war zone” and “marathon”. They often have to take one day at a time, and in periods of their life one can argue, that it is more survival than actually collecting experience.
It is very important for the fathers to be a good father. They all place a high value on children and family. Although they fulfill the role as a father in the Danish societal sense, it shows that the choices they make through the process, and what they value in proportion to being a good father, is very different. The analysis shows, that their road is shaped by pragmatic choices of what is important and possible.

dad blackbelt

Value of Information for Decisions

If you ask “what is the ROI of context-driven testing” it is the same as asking:

  • What is the value proposition of providing information to the stakeholders?
  • Will management and customers pay for information?

Let me tell you a story: Just today I finally got around to changing tires on my car. Three months ago I bought a campaign voucher for a cheap switch of winter tires to summer tires – so it was about time. I booked a time and went to the shop with the summer tires in the trunk. BUT then … the front tires where out of shape due to wrong “tracing”, brake cables and other stuff worn and empty for lubrication. sigh!

So … the shop had to repair those critical defects (yeah, the vouchers a good business generator, #I’mOKwithThat). They gave me the keys to a replacement car for the day for free. And we discussed fixing some other stuff – the tricky ignition was Deffered/FixedUpStream but the defective brake lights added to the work order (New bug raised due to a hunch). I got an estimate and went for the day. The quote was pretty close, the repair on time and the requirements verified on the release bill. 

And then they provides me with a list of a few things they noticed along the way. 

  • I probably paid for an automated test and configuration of a “trace” balancing – I assumed it there.
  • We did discuss scope, price, schedule and timing – along with bug triage
  • The shop did provide me with enough information and estimation up front to base my decisions on
  • As the product owner I did not pay directly for the list of test ideas not covered – but I appreciated it!

The shop could have just swapped the tires for the voucher cost – and noticed nothing else. They could have chosen not to tell me about the additional bugs. They could not have offered me a replacement car for the day. They probably where more expensive than a moonlighting garage dude – I known now what the difference can be.

I value that they provide information to aid my business decision-making – besides just swapping the stupid tires. They will probably get repeat business from me – directly or indirectly. 🙂 And yes, Scott, they did have free coffee