A Track down History

Currently I am doing a large V-model waterfall project with a three month testing period and 500+ requirements. To track the progress I want to dust off my old “test progress tracking” method that I matured and described in 2011 and 2014.

The approach was documented in two articles for “The Testing Planet” a paper magazine by the Software Testing Club. The “STC” later became the now world famous Ministry of Testing. Unfortunately the original articles are no longer available on the MoT site – they are on the WayBackMachine. So not to track down that path, here’s a recap of:

A Little Track History that goes a Long Way

For large (enterprise, waterfall) projects tracking test progress is important, as it helps us understand if we will finish a given test scope on time. Tracking many of projects have given me one key performance indicator: daily Percent Passed tests as compared to an s-curve. The data in the S-curve is based on the following data points, based on numerous projects:

Time progressExpected Passed Progress
Adding a 3rd order polynomial trend-curve gives the s-curve.

If the “Actual” curve is flat over more days or is below the blue trend line, then investigate what is blocking the test progress… defects perhaps:

The Defect Count and Camel Image

During a large project like this the active bug count goes up and down. No one can tell what we find tomorrow or how many we will find. In my experience tracking the daily count of active defects (i.e. not resolved) is key, and will oscillate like the humps on a camel:

Camel background is optional

If the curve doesn’t bend down after a few days there are bottlenecks in the timely resolution of defects found. When the count goes up – testing a new (buggy) area is usually happening. Over time the tops of the humps should be lower and lower and by the end of the project, steep down to 0.

And thus a little track history has come a long way.

The Testing Planet, July 2011

The 860 kcal bug

Simon 7yo made the sticks with the space alians. The egg candy is flying saucers
Simon 7yo made the sticks with space aliens.        The egg candy is flying saucers

One day I was typing in my personal measurements into my health tracker. I had to go do the weight and height and then I came back and entered them.  And I started running, burning 800+ kcal‘s every time. First thought, whatever – it’s probably right. But apparently it was awesome for running a few clicks in 20 minutes (said my wife). After some time I returned to the settings for a different reason and found:

Height 70 cm / 2.3 feet
Weight 180 kg / 374.8 lbs

No wonder I had burned some carb’s if I’d been running along with those numbers…. Bug / No bug ?

  • Apparently 70 cm is a valid height – What would be an invalid interval? – What would be invalid input?
  • Apparently 180 is a valid weight – What would be an invalid interval? – What would be invalid input?
  • How many angry users will you get for hinting that their body details are invalid?
  • Does it matter? – It didn’t matter to me (or I would have spotted it sooner, I guess)

Testers need critical thinking

[ Atlassian blog series | Lanette Creamer | Feb 6, 2012]

First, to be effective as a software testers, you develop a different kind of critical thinking in order to find the exceptions to a given rule. It can be said that if you hear hoofbeats, think horses, not zebras. Testers know you’ve considered horses, but can you handle a herd of zebras? This is one reason why our arguments may sound strange to you, but when your code is on Safari, will you be less sorry?

Another reason why testers often do not agree, even on things that seem simple to other people, is that testers need to convince other people in order to see changes happen. 

For many of us, the number of bugs we FIND isn’t important. It is only the number of bugs important enough to be fixed or that we prevented that adds value to the end user. 

We don’t intend to argue for bugs being fixed or prevented unless they ARE important, but when that bug is important, a good tester will be as convincing as a pitbull lawyer.

There will be research, evidence, and convincing testimony from bystanders. By the time you’ve reached many years experience as a professional software tester, you are as adept at explaining risk and sharing information with others as a good insurance agent, and that is a good thing, unless you happen to be debating against one.

See also A little track history goes a long way