testcases

Chess and Testing

jo2sn Family and fun, Softwaretesting , , , ,

[Originally on the Ministry of Testing, Jan 2014, now only on the Web Archives]: The Day Testing Died But Didn’t

In May 11 1997 a computer beat the world champion Kasparov in chess [1] – not convincingly, but still. From then on chess could be reduced to a set of scripts and the scripts automated so fast that it was comparable to the human mind [2]. But the human chess players continued to succeed – not by more rote memorisation, but by more intuition and feelings.

Imagine that – to play world champion chess and base your moves on feelings. This is what Magnus Carlsen does [3]. As of January 2014 they are the reigning World Chess Champion [4] and the no. 1 ranked player in the world [5] with the highest rating in history [6]. I must admit that I read about them in the paper [7][8] [9], but the story relates to how even one of the most complex brain games can be automated, and yet there are still moves to explore.

To play according to textbooks is fine, up to a certain level. Perhaps up to master level, but not to grandmasters. [10]

Originally chess was a game played on a board, but even more so in the brain of the players. Grand masters of the cold war super powers played each other with full focus on both the board moves and the body moves.

Encyclopaedias of chess moves have been written; 1700+ chess moves have been given mnemonics like “the Sicilian Defence”, “King Gambit” (SFDEPOT anyone?). And the chess masters have played and played and memorized and played (against) the computer again and again.

There are books, terminology, strategies and schools of chess [11]. To quote Wikipedia:

A school (of chess)means a (chess) player or group of players that share common ideas about the strategy (of the game). There have been several schools in the history (of modern chess). Today there is less dependence on schools – players draw on many sources and play according to their personal style.

After Kasparov there were other world chess champions [12] – and lately 23 year old Magnus Carlsen, as mentioned. Carlsen started playing chess in 1998; they played Kasparov [13] as a 13 year old for a draw and later had Kasparov and the Danish grand master Peter Heine Nielsen as a trainer. Heine Nielsen explains about Carlsen:

“While the existing World Champion Anand [14]’s strength was being able to prepare thoroughly and calculate moves very fast while playing, Carlsen is different – they thrives in the contexts that are not distilled by the computer or text books. When it’s man to man – then their the opposite of a computer; the one that often does the unexpected yet effectual play. They plays a variety of openings – making it really hard to prepare for.”

Carlsen can’t describe, what goes on in their brain, while they play chess. Some moves just feels good; and when the opponent play is somewhat based on computer calculations – that is maybe the best response. [15]

Chess didn’t die with the automation, chess didn’t die by being distilled in text books and templates and mnemonics – but chess evolved. The current unfair advantage for Carlsen is their irrationality and intuition – it’s what sets them apart from the scripts.

The day testing died – but didn’t, is another story Or is it?

References

  1. http://en.wikipedia.org/wiki/Deep_Blue_versus_Garry_Kasparov
  2. http://en.wikipedia.org/wiki/Turing_test
  3. http://en.wikipedia.org/wiki/Magnus_Carlsen
  4. http://en.wikipedia.org/wiki/World_Chess_Championship
  5. http://en.wikipedia.org/wiki/FIDE_World_Rankings
  6. http://en.wikipedia.org/wiki/Comparison_of_top_chess_players_throughout_history
  7. The article is my inspiration – and will be paraphrased
  8. Danish and pay-walled http://jyllands-posten.dk/eceRedirect?articleId=6190682
  9. By the way, I don’t know much about chess
  10. http://en.chessbase.com/post/vladimir-kramnik-che-is-so-deep-i-simply-feel-lost-
  11. http://en.wikipedia.org/wiki/School_of_chess
  12. http://en.wikipedia.org/wiki/World_Chess_Champion
  13. http://en.wikipedia.org/wiki/Garry_Kasparov
  14. http://en.wikipedia.org/wiki/Viswanathan_Anand
  15. Quote from the Danish article, my translation

Rant on Login Screen examples

jo2sn Softwaretesting , , ,

If you are demonstrating testing technologies or testing examples around RPA, ML, Selenium and so on – Please: DO NOT USE A LOGIN FORM!

The test scenarios I usually deal with are not this… mundane. While a few testers probably still have to build login forms from scratch, a login feature is a commodity by now. Use OAuth for public facing sites and Active Directory Federation internally in the organisation. Really – there’s no need to reinvent the wheel. To the end user and even the Product Owner logging in is just a stepping stone. 

I just want to log in, and then I’m done for the day

Said no user ever*

Showing that you can train an AI network or other framework to login might solve a tedius testing task, but is usually not the thing I’m after. When a user is logged in, they are there to solve something, to process something, to do something – to engage with something. And this is where the best tests are heading too – this is the tests that adds value to the business and tells something about the product. 

For instance: In one project I did, we disabled the login entirely to make the CI/CD run feature testing. The plain login screen was temporary anyways, as the solution authentication would be based on certificates. We never spend much time on it, neither on total coverage.

Less combinations – More Real Life Scenarios

So what can you use as an example instead of login boxes and combinatoric bar stories? How about anonymizing the latest test you had to do on your latest live-action testing project? This will tell me about your challenges, your business domain and when the last time was – among other things.

Let me start, as of writing the latest test case I touched (same day as writing this) was for a new public data registration project. The tester and end user “subject matter expert” was testing the data registration form from both a GUI and web-services perspective. Export the entered data form as XML and import it again via web-services to see the consistency.

Could that be tool supported – maybe. The knowledge about the system is not very explicit. It’s a bit complicated, actually. Could it be trained by an ML/AI system – I doubt it. There is no global training set for this class of systems – we have the old version but are adding new things. 

If you are (a tool vendor) demonstrating something – do try to understand the context and problems your customers are trying to solve first. Ask them what their latest tests were and where the challenges are. If it’s a login box you’re good to go, but I doubt it. 

*: Unless you are somehow measured on “logging in”. For instance to claim unemployment benefit, login to job site daily…Snap! OTOH shallow measure.

http://rebrick.lego.com

Many Bits under the Bridge

jo2sn Softwaretesting , , , , , , , ,

I’ve been in the testing business for 14 years – when I started in late 2002 it was all about using HP Test Director 7.6 – in a browser… There was only one model of testing, v-model, and only one book of testing the ISEB (later ISTQB) vocabulary. And only one expected output of testing: Testers designed test cases, executed and perhaps wrote both a test plan document and test report document. Test process improvement was a thing, but even so testing was often a pointy cog…  

Many Bits under the Bridge Later

It is not about the test cases any more, it’s about being part of a team – that delivers an IT solution to the business. First of all, if it’s just about the test cases then it is a race to the lowest paid off-shore location, a run to the bottom in repetitiveness and mechanic activity. Checking! with more focus on crossing the t’s and dotting the i’s. We have tools for that now – the plates are shifting.

When testing professionals puts “writing test cases” on their LinkedIn description. It seems to me that they are stuck in the testing world of 10 years ago. Standing still and not seeing that Testers are Knowledge Workers – not workers of producing artifacts. It is much more important to see beyond the visible, Uncovering better ways and seeing testing as an activity to provide information to the stakeholders, based on experiments and observations.

Skill up and be smarter! And don’t listen to old tapes – it’s not worth it :).

See also this from QA Symphony & Ministry of Testing:

The Software Tester: Modern vs. Traditional [Infographic]
The Software Tester: Modern vs. Traditional [Infographic]