Chess and Testing

[Originally on the Ministry of Testing, Jan 2014, now only on the Web Archives]: The Day Testing Died But Didn’t

In May 11 1997 a computer beat the world champion Kasparov in chess [1] – not convincingly, but still. From then on chess could be reduced to a set of scripts and the scripts automated so fast that it was comparable to the human mind [2]. But the human chess players continued to succeed – not by more rote memorisation, but by more intuition and feelings.

Imagine that – to play world champion chess and base your moves on feelings. This is what Magnus Carlsen does [3]. As of January 2014 he is the reigning World Chess Champion [4] and the no. 1 ranked player in the world [5] with the highest rating in history [6]. I must admit that I read about him in the paper [7][8] [9], but the story relates to how even one of the most complex brain games can be automated, and yet there are still moves to explore.

To play according to textbooks is fine, up to a certain level. Perhaps up to master level, but not to grandmasters. [10]

Originally chess was a game played on a board, but even more so in the brain of the players. Grand masters of the cold war super powers played each other with full focus on both the board moves and the body moves.

Encyclopaedias of chess moves have been written; 1700+ chess moves have been given mnemonics like “the Sicilian Defence”, “King Gambit” (SFDEPOT anyone?). And the chess masters have played and played and memorized and played (against) the computer again and again.

There are books, terminology, strategies and schools of chess [11]. To quote Wikipedia:

A school (of chess)means a (chess) player or group of players that share common ideas about the strategy (of the game). There have been several schools in the history (of modern chess). Today there is less dependence on schools – players draw on many sources and play according to their personal style.

After Kasparov there were other world chess champions [12] – and lately 23 year old Magnus Carlsen, as mentioned. Carlsen started playing chess in 1998; he played Kasparov [13] as a 13 year old for a draw and later had Kasparov and the Danish grand master Peter Heine Nielsen as a trainer. Heine Nielsen explains about Carlsen:

“While the existing World Champion Anand [14]’s strength was being able to prepare thoroughly and calculate moves very fast while playing, Carlsen is different – he thrives in the contexts that are not distilled by the computer or text books. When it’s man to man – then he’s the opposite of a computer; the one that often does the unexpected yet effectual play. He plays a variety of openings – making it really hard to prepare for.”

Carlsen can’t describe, what goes on in his brain, while he plays chess. Some moves just feels good; and when the opponent play is somewhat based on computer calculations – that is maybe the best response. [15]

Chess didn’t die with the automation, chess didn’t die by being distilled in text books and templates and mnemonics – but chess evolved. The current unfair advantage for Carlsen is his irrationality and intuition – it’s what sets him apart from the scripts.

The day testing died – but didn’t, is another story Or is it?

References

  1. http://en.wikipedia.org/wiki/Deep_Blue_versus_Garry_Kasparov
  2. http://en.wikipedia.org/wiki/Turing_test
  3. http://en.wikipedia.org/wiki/Magnus_Carlsen
  4. http://en.wikipedia.org/wiki/World_Chess_Championship
  5. http://en.wikipedia.org/wiki/FIDE_World_Rankings
  6. http://en.wikipedia.org/wiki/Comparison_of_top_chess_players_throughout_history
  7. The article is my inspiration – and will be paraphrased
  8. Danish and pay-walled http://jyllands-posten.dk/eceRedirect?articleId=6190682
  9. By the way, I don’t know much about chess
  10. http://en.chessbase.com/post/vladimir-kramnik-che-is-so-deep-i-simply-feel-lost-
  11. http://en.wikipedia.org/wiki/School_of_chess
  12. http://en.wikipedia.org/wiki/World_Chess_Champion
  13. http://en.wikipedia.org/wiki/Garry_Kasparov
  14. http://en.wikipedia.org/wiki/Viswanathan_Anand
  15. Quote from the Danish article, my translation

Advertisements

A Track down History

Currently I am doing a large V-model waterfall project with a three month testing period and 500+ requirements. To track the progress I want to dust off my old “test progress tracking” method that I matured and described in 2011 and 2014.

The approach was documented in two articles for “The Testing Planet” a paper magazine by the Software Testing Club. The “STC” later became the now world famous Ministry of Testing. Unfortunately the original articles are no longer available on the MoT site – they are on the WayBackMachine. So not to track down that path, here’s a recap of:

A Little Track History that goes a Long Way

For large (enterprise, waterfall) projects tracking test progress is important, as it helps us understand if we will finish a given test scope on time. Tracking many of projects have given me one key performance indicator: daily Percent Passed tests as compared to an s-curve. The data in the S-curve is based on the following data points, based on numerous projects:

Time progressExpected Passed Progress
10%0%
20%5%
30%10%
40%30%
50%45%
60%60%
70%75%
80%90%
90%95%
100%100%
Adding a 3rd order polynomial trend-curve gives the s-curve.

If the “Actual” curve is flat over more days or is below the blue trend line, then investigate what is blocking the test progress… defects perhaps:

The Defect Count and Camel Image

During a large project like this the active bug count goes up and down. No one can tell what we find tomorrow or how many we will find. In my experience tracking the daily count of active defects (i.e. not resolved) is key, and will oscillate like the humps on a camel:

Camel background is optional

If the curve doesn’t bend down after a few days there are bottlenecks in the timely resolution of defects found. When the count goes up – testing a new (buggy) area is usually happening. Over time the tops of the humps should be lower and lower and by the end of the project, steep down to 0.

And thus a little track history has come a long way.

The Testing Planet, July 2011