A Track down History

Currently I am doing a large V-model waterfall project with a three month testing period and 500+ requirements. To track the progress I want to dust off my old “test progress tracking” method that I matured and described in 2011 and 2014.

The approach was documented in two articles for “The Testing Planet” a paper magazine by the Software Testing Club. The “STC” later became the now world famous Ministry of Testing. Unfortunately the original articles are no longer available on the MoT site – they are on the WayBackMachine. So not to track down that path, here’s a recap of:

A Little Track History that goes a Long Way

For large (enterprise, waterfall) projects tracking test progress is important, as it helps us understand if we will finish a given test scope on time. Tracking many of projects have given me one key performance indicator: daily Percent Passed tests as compared to an s-curve. The data in the S-curve is based on the following data points, based on numerous projects:

Time progressExpected Passed Progress
10%0%
20%5%
30%10%
40%30%
50%45%
60%60%
70%75%
80%90%
90%95%
100%100%
Adding a 3rd order polynomial trend-curve gives the s-curve.

If the “Actual” curve is flat over more days or is below the blue trend line, then investigate what is blocking the test progress… defects perhaps:

The Defect Count and Camel Image

During a large project like this the active bug count goes up and down. No one can tell what we find tomorrow or how many we will find. In my experience tracking the daily count of active defects (i.e. not resolved) is key, and will oscillate like the humps on a camel:

Camel background is optional

If the curve doesn’t bend down after a few days there are bottlenecks in the timely resolution of defects found. When the count goes up – testing a new (buggy) area is usually happening. Over time the tops of the humps should be lower and lower and by the end of the project, steep down to 0.

And thus a little track history has come a long way.

The Testing Planet, July 2011
Advertisements

Without Timing – Quality, Schedule and Cost is nothing

A project usually has to be on target on Quality, or at least on agreed quality. 

A project usually has to be on Schedule, or at least on agreed schedule.

A project usually has to be on target regarding Costs, or at least agreed costs.

It is an ancient IT project manager mantra, to control these parameters using change requests and change management – typically due to events during the project. It takes to long, it’s too buggy, it costs more etc. etc. To change one parameter – you have to change one of the others. In example (and strictly in some context) – even adding more testcases will increase cost or influence the schedule (discuss).

But there are plenty of examples (of Government IT projects) that are on schedule, on cost, on quality – yet they fail the timing and after a number of years delivers a system on an obsolete platform [true story]. More recent examples – bank D launches their banking app first, get all the press – and a month later bank N comes around – on agreed schedule, agreed costs, agreed quality… and get very little of the “hype”.

When contemplating the business decisions – what seems right now right now might be wrong later – considering the Timing.

4triangles

Acceptance criteria are more than what can be measuredAn Expected Gathering

I’ll fix this later 🙂

Softwaretesting is only dead, if it stands still

[ Software Testing Club Blog | January 23, 2012 ]

Here are some quotes from actual test mission statements from the last 10 years. Test always is changing. Come to think of it, it’s only dead if it stands still.

2003

We discover and measure the quality of the system:
– to test everything is utopia,
– we should test something, something new, something old (and something blue?)

2006

The primary objective for testing is to reveal the quality of the delivery. The revealed quality is documented in test reports, that serves as suggestions for decisions on approval or rejection of the delivery. Key principles: early involvement, structured testing, based on V-model, documented, COTS tools (TestDirector)

2008

Testing is the structured activity of identifying the quality and improving the quality of the an IT-supported product or project
– Identification of quality = Preparing and executing a test activity
– Improving the quality = Evaluation, fixing and retesting of raised incident

Test provides information about the quality of IT-supported product or project, with that information:
– Quality of products can be improved
– Project decisions can be based on facts
– Processes can be investigated for improvement

2011 

Testing has the mission of providing information to the project about the quality of the business features of the solution.

Testing is about learning about the quality solution, not about confirming, verifying or assuring.
Test is about adjusting, not about following a strict script and knowing everything up front.

Originally at my SoftwareTestingClub blog – more from my STC Blog here: All oracles are failableTesting decommissioning of ICBMs and software and Testing a new version.