Go read Accelerate!

ACCELERATE – The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations

by NICOLE FORSGREN, JEZ HUMBLE, GENE KIM

The authors have “multiple examples of applying these practices within mainframe environments, traditional packaged software application delivery teams, and product teams“. It’s not just for business-to-consumer web-based organizations.

The book is a tour the force into software delivery performance – the research and statistics shows a clear correlation from DevOps and Lean principles to high achieving organisations. Every arrow on the below model is backed with research. Read the arrow clearly as “drives”, “improves” and “lead to”. E.g. Continuous Delivery leads to Less Burn out.

Saved you a click: https://itrevolution.com/book/accelerate/

A last thing to highlight: High performing organisations have lower manual work percentages in areas like: configuration management, testing, deployments and in the (ITIL) change approval process.

So – if you want to increase the boxes on the right, go do the stuff on the left.

Read the book and act on it.

Further reading for Contest NYC 2019

Materials used for the talk and workshop at Contest NYC 2019:

One page test plan

Wardley Maps

Research:

Test management / Test Coach

Subject Matter Experts

Practical tips:

The subject matter expert in LEGO knows the bigger pieces left goes into the model.
The subject matter expert in LEGO knows the bigger pieces left goes into the model.

Closing the Gaps

[Previously on the Ministry of Testing, Nov 2014 – now only on the Web Archives]: About Closure

When I’m in a testing activity I want my test cases [Passed], my user stories [done] and my coffee [black].  Stuff may have a start point, some states in between and an end state. Lets look at ways to represent states and articulate the meaning of states.
One way to illustrate status about the product being tested is to model the activities we have with states. An agile user story may be [ready], [in progress] or [done]. A document may be [final], [approved] and a mind map may be iconified etc. States are so common that we sometimes forget the theory behind the model, and what benefits we have from the theories.

The representation of closure

For instance we can look to computer science graph theory[1]  to help us understand and control the states diagrams. It is the same graph theory that brings us state machines and state-transition diagrams, but that is another story[2] . In a graph theory state model we want one unique start state (Like [To do]), and one unique end state (like [passed]), everything in between is intermediate.

A (single) end state helps prevent the state machine from going on forever[3] , and us from going on forever too. [Deferred] and [Rejected] are temporary states to me. Setting [Rejected] back to “detected by” will aid that the tester reflects on the reasons. The reasons are then tested (in the brain). Sometimes it’s a “my bad” but quite often also the tester finds that issue is simply not “rejected” with more data and examples.

The understanding of closure

Similarly the agile “Definition of Done”[5]  and “Definition of Ready”[6]  helps the agile team phrase when the task is to change state, sometimes it’s explicit, sometimes it’s implied. The understanding of terms (the semantics) are usually more imperative than the syntax (the rules and representation). Sometimes it’s necessary to “connect the lines”.

There are a two related psychology theories on closure. One is the Gestalt law of closure[7]  – that is that we tend to self-organize items into an orderly structure. As the image above isn’t really about triangles – it’s about our human tendency to connect the dots. The other psychology part is the desire to close stuff to gain controllability[8]:

The need for closure is the motivation to find an answer to an ambiguous situation. This motivation is enhanced by the perceived benefits of obtaining closure, such as the increased ability to predict the world and a stronger basis for action.

Management and stakeholders often want a “firm and unambiguous” answer from our testing investigations. And this is often the business justification for setting states to our work products; that we have states to illustrate work progress in. Sometimes loose representations and a strong shared understanding goes well, sometimes a more strict representation and elaboration is required.

The syntax of states may be easily explained and codified (and checked), while the semantics and perception is less direct – and needs analysis (and testing). All work products (even for mind maps and test charters) have states and we must articulate both the syntax and the semantics to the team and stakeholders.

References

  1. Graph theory http://en.wikipedia.org/wiki/Graph_theory
  2. Fell in the trap of total coverage https://jlottosen.wordpress.com/2012/11/05/fell-in-the-trap-of-total-coverage/
  3. Finite state machines http://en.wikipedia.org/wiki/Finite-state_machine
  4. A Track down History
  5. definition of done https://www.scrum.org/Resources/Scrum-Glossary/Definition-of-Done
  6. definition of ready http://guide.agilealliance.org/guide/definition-of-ready.html
  7. Law of closure http://jeremybolton.com/2009/09/gestalt-design-principles-the-law-of-closure/
  8. Closure (Psychology) http://en.wikipedia.org/wiki/Closure_(psychology)

Chess and Testing

[Originally on the Ministry of Testing, Jan 2014, now only on the Web Archives]: The Day Testing Died But Didn’t

In May 11 1997 a computer beat the world champion Kasparov in chess [1] – not convincingly, but still. From then on chess could be reduced to a set of scripts and the scripts automated so fast that it was comparable to the human mind [2]. But the human chess players continued to succeed – not by more rote memorisation, but by more intuition and feelings.

Imagine that – to play world champion chess and base your moves on feelings. This is what Magnus Carlsen does [3]. As of January 2014 he is the reigning World Chess Champion [4] and the no. 1 ranked player in the world [5] with the highest rating in history [6]. I must admit that I read about him in the paper [7][8] [9], but the story relates to how even one of the most complex brain games can be automated, and yet there are still moves to explore.

To play according to textbooks is fine, up to a certain level. Perhaps up to master level, but not to grandmasters. [10]

Originally chess was a game played on a board, but even more so in the brain of the players. Grand masters of the cold war super powers played each other with full focus on both the board moves and the body moves.

Encyclopaedias of chess moves have been written; 1700+ chess moves have been given mnemonics like “the Sicilian Defence”, “King Gambit” (SFDEPOT anyone?). And the chess masters have played and played and memorized and played (against) the computer again and again.

There are books, terminology, strategies and schools of chess [11]. To quote Wikipedia:

A school (of chess)means a (chess) player or group of players that share common ideas about the strategy (of the game). There have been several schools in the history (of modern chess). Today there is less dependence on schools – players draw on many sources and play according to their personal style.

After Kasparov there were other world chess champions [12] – and lately 23 year old Magnus Carlsen, as mentioned. Carlsen started playing chess in 1998; he played Kasparov [13] as a 13 year old for a draw and later had Kasparov and the Danish grand master Peter Heine Nielsen as a trainer. Heine Nielsen explains about Carlsen:

“While the existing World Champion Anand [14]’s strength was being able to prepare thoroughly and calculate moves very fast while playing, Carlsen is different – he thrives in the contexts that are not distilled by the computer or text books. When it’s man to man – then he’s the opposite of a computer; the one that often does the unexpected yet effectual play. He plays a variety of openings – making it really hard to prepare for.”

Carlsen can’t describe, what goes on in his brain, while he plays chess. Some moves just feels good; and when the opponent play is somewhat based on computer calculations – that is maybe the best response. [15]

Chess didn’t die with the automation, chess didn’t die by being distilled in text books and templates and mnemonics – but chess evolved. The current unfair advantage for Carlsen is his irrationality and intuition – it’s what sets him apart from the scripts.

The day testing died – but didn’t, is another story Or is it?

References

  1. http://en.wikipedia.org/wiki/Deep_Blue_versus_Garry_Kasparov
  2. http://en.wikipedia.org/wiki/Turing_test
  3. http://en.wikipedia.org/wiki/Magnus_Carlsen
  4. http://en.wikipedia.org/wiki/World_Chess_Championship
  5. http://en.wikipedia.org/wiki/FIDE_World_Rankings
  6. http://en.wikipedia.org/wiki/Comparison_of_top_chess_players_throughout_history
  7. The article is my inspiration – and will be paraphrased
  8. Danish and pay-walled http://jyllands-posten.dk/eceRedirect?articleId=6190682
  9. By the way, I don’t know much about chess
  10. http://en.chessbase.com/post/vladimir-kramnik-che-is-so-deep-i-simply-feel-lost-
  11. http://en.wikipedia.org/wiki/School_of_chess
  12. http://en.wikipedia.org/wiki/World_Chess_Champion
  13. http://en.wikipedia.org/wiki/Garry_Kasparov
  14. http://en.wikipedia.org/wiki/Viswanathan_Anand
  15. Quote from the Danish article, my translation

When Subject Matter Experts test

Teaser for [Test Bash Brighton 2020] : How to Coach Subject Matter Experts to Do Testing

In the recent years I have been working on projects with no dedicated testers but plenty of testing. The testing has primarily been performed by subject matter experts. This is where it gets interesting, as my role on these projects has been to lead the testing being performed by people that have limited experience in testing. They also have no desire to be testing specialists, after all they are already specialists in their own subjects, however, everyone agrees and insist that the testing needs doing. So how do we ensure that the testing being done is done well? 
 After having worked on several very different projects, yet still with subject matter experts doing the testing, I have been able to get both the public process clerks and the technology specialists to perform excellent testing. This talk is about the approaches that I have found work well: 

  • One of the approaches is for me to prepare the test cases and prepare them only as headlines. Sometimes preparing the tests as open questions helps too. 
  • Another approach is to lead them as if they are doing the project participation voluntarily. They probably are, but still it helps to respect where they are coming from.

 The lessons though (good and bad) is relevant to many testers in other situations, especially being the only “tester” on the team. The story applies equally to developers and business end users doing most of the testing and you will have them contributing with great testing in no time!

What you will know after the talk:

  • An understanding of how testing looks when done by subject matter experts
  • How to lead a testing activity with an appreciative and motivating style
  • Examples of how teams can do great testing without dedicated testers
Test Bash Brighton, March 2020

A Track down History

Currently I am doing a large V-model waterfall project with a three month testing period and 500+ requirements. To track the progress I want to dust off my old “test progress tracking” method that I matured and described in 2011 and 2014.

The approach was documented in two articles for “The Testing Planet” a paper magazine by the Software Testing Club. The “STC” later became the now world famous Ministry of Testing. Unfortunately the original articles are no longer available on the MoT site – they are on the WayBackMachine. So not to track down that path, here’s a recap of:

A Little Track History that goes a Long Way

For large (enterprise, waterfall) projects tracking test progress is important, as it helps us understand if we will finish a given test scope on time. Tracking many of projects have given me one key performance indicator: daily Percent Passed tests as compared to an s-curve. The data in the S-curve is based on the following data points, based on numerous projects:

Time progressExpected Passed Progress
10%0%
20%5%
30%10%
40%30%
50%45%
60%60%
70%75%
80%90%
90%95%
100%100%
Adding a 3rd order polynomial trend-curve gives the s-curve.

If the “Actual” curve is flat over more days or is below the blue trend line, then investigate what is blocking the test progress… defects perhaps:

The Defect Count and Camel Image

During a large project like this the active bug count goes up and down. No one can tell what we find tomorrow or how many we will find. In my experience tracking the daily count of active defects (i.e. not resolved) is key, and will oscillate like the humps on a camel:

Camel background is optional

If the curve doesn’t bend down after a few days there are bottlenecks in the timely resolution of defects found. When the count goes up – testing a new (buggy) area is usually happening. Over time the tops of the humps should be lower and lower and by the end of the project, steep down to 0.

And thus a little track history has come a long way.

The Testing Planet, July 2011

To Transform Testing

There is no doubt that our long lived testing narrative is under pressure. To continue to bring business value, both the IT field and testing is transforming to be about proactive business enabling.

The IT domain is currently buzzing with the word “IT transformation” – the idea that IT services should be more about “inspire, challenge and transform the digital businesses“. That it should be less about delivering IT products and artifacts and more about enabling digital business outcomes. Even for testing – it should be less about a product/service, and more about business necessity. As Anders writes:

Stop focusing on the things that bug people, and dare to do both IT and testing smarter and more business focused from the start. Build Quality in – smarter from start. That goes for IT services as a whole, and definitely also for the testing activities.

What can you do to transform your testing? I have three areas:

Discuss Business Strategy

Learn Wardley mapping – and use it like Chris McDermott to create context specific maturity models with Wardley Maps informed by Cynefin. Use the mapping to Broaden the scope of the system under test.

Align with the Business Strategy

Leading Quality [Cummings-John, Peer 2019] has a whole chapter on “Align your team to your company growth metric“. Consider if the company you work for is driven by Attention, Transaction or Productivity metrics, and arrange your test activities accordingly.

Dare to Deliver in New Ways

We are usually talk so much about optimizing the (IT and testing) delivery, that we forget other ways to be innovative and provide business enabling. One way could be to dare use new technology like RPA or a HoloLens to support tedious tasks in testing – to use an existing product to something new. Another approach to actually test “all the things” that matter or to apply testing to IT outside the realm of application delivery.

To Transform Testing I will discuss, align and dare so that test solutions can be proactive business enablers – (not only achieve shippable quality).

Mapping Mondays – Pioneers, Settlers, Town Planners