Shift-Deliver, TestOps and Management of Changes

5 Comments

Shift-Deliver is a label I choose to put on the changes the roles and activities of the TEST MANAGER, when the test manager moves towards (also) being involved in the ITIL change requests, delivery management, configuration management and branch management that happens when the solution goes from the test phase to production. Another label could be “TestOps” as presented here, as the intersection of Testing and Operations. TestOps have been identified for along time. ….Interesting.  🙂

In my IT outsourcing context, this is less about software, and more about solutions. In at least two of my long term enterprise scale projects, half the job was test management (of operations) projects, half the job was regarding ITIL change management. My change management activities was mostly making sure that

  • the process was followed
  • that information was provided to the stakeholders
  • that testing happened
  • risk mitigation happened

I was hired as “the quality guy”, but expanded the role over the time I’ve been on the team to take ownership of all of our build and release infrastructure as well. Basically, I’m responsible for everything from the moment code is checked in, until it hits our production servers 

To use a quote by Alan Page. Again Alan is a representative of what happens with regards to trends in testing. He might be wrong, as well as I. I try to label the trends to understand them. These four trends that I have spotted are not mutually exclusive, neither do they all four need directions. Change is happening to the classic test manager rolle of going through the motions of test cases and documents. This is clear when looking into these posts:

Initially I discussed Shift-Deliver, Shift-RightShift-Left and Shift-Coach  at Nordic Testing Days 2016 during the talk “How to Test in IT operations“ and coined the labels on the EuroStar Test Huddle forum.

DSC03562

A commercial body of Knowledge

3 Comments

What I know of the ISO29119 is that specifies specific numerated techniques, documents and document content. I know this from their website, where I can read that it will cost me $1000 to buy “the book” (club discounts available) – the body of knowledge.

It’s a collective work written by a number of people in the industry, and have been years in the making. Some of the people work in consulting and provide training in the framework, some of the companies sponsoring the work provide consulting in implementation of the framework. Companies can have an audit for a certificate too. That will require a large investment as the organisation have to (norminative) conform to plenty of “shall”.

But besides that it’s a closed book (and it’s not even on Amazon). To me the 29119 is misguided from the beginning, it should be a book – a commercial body of knowledge, like TMAP or like ITIL. Something that you could buy into or not. Not something in any respect labelled as a international standard.

  • It seems it requires either a range of documents and lot of tailoring
  • It seems to be some what “dated” in the addressing ways of testing being added in recent years
  • It seems to claim that it has consensus in the industry
  • It seems that some people have tried to participate , but failed
  • It seems that some people did not want to participate on principle, even if invited
  • It seems to claim that it is a silver bullet, a one size fits all

I cannot evaluate the implications for my customers asking about compliance without elaboration – on the details of 29119, and on the customers objectives. What is the business driver for complying with said framework? Which is actually what I was looking for – what helps the (customer) business making a business?

I doubt that someone else’s delivery framework can provide you with the DNA, the unique value proposition, of the specific context that is needed – for you! #ImLookingAtYou. If we blindly comply with the framework what is the driver besides cost and commodity. If the driver is something else, then start right there. Start with how testing and artefacts implements the strategy, values and decisions that you have. Start with “innovative“, “quality of life“, “coherent” – how does that relate to your testing.

See also

Testing has to be fit for purpose too

Leave a comment

Reading some quality process documents I found the following definition:
  • To detect and remove errors before the computer system is put into production
  • To demonstrate that the computer system is fit for intended use

But when we look at the ITIL definition of the value composition of a service – it looks fit for use, as above: follows the requirements, sufficient – what the customer gets. But also – at an equal base, it looks at fit for purpose: it has a positive effect on the business, it solves a business problem, solving the right problem. The product is a solution. If the problem isn’t solved, the product doesn’t work.

see also: Uncovering better ways  Softwaretesting is only dead, if it stands still I didn’t open it

Acceptance is more than what can be measured

16 Comments

A typical acceptance criteria is that a specific percentage of test cases passed and/or a specific number of critical defects unresolved. Yet the expressions have to be recognized as a way of the stakeholder to express the level of expected confidence. In other words “what are you OK with?” How can we help you gain confidence in the solution? What trends should we look for?

I have experienced situations where the solution was delivered, even though the criteria where not met. The so-called Go-NoGo meeting, was usually a Go-Go, and jokingly called so. I have also experienced that the business deferred a fix for a critical defect in production. That even if a planned release fulfilled the elaborate acceptance criteria – the delivery was cancelled. It was a major enterprise release that was technically to spec – yet because of other business risks, it was postponed.

 As a tester you may experience this where all the automatic Factory Acceptance tests (FAT) pass, but the system still fails to react. I had to text “FAT failed” to someone recently, but it auto-corrected to “FART failed“. Indeed if all the requirements pass and but the system fails to deliver – it is a fart.

The challenge is that not everything that can be measured counts, and not everything that counts can measured. 

And that’s not even considering that 100% testcases passed is a metric that makes no sense. Quality is not only the amount of specific attributes, as ISO/IEEE standards may lead you to measure, but a relationship “something that matters – to someone who matters – at sometime“. If you only look at the measurable – you miss half of the story.

The business needs

  • fit for purpose (requirements) and fit for use (business context) [ITIL v3]
  • to solve a business problem – ff the problem isn’t solved, the product doesn’t work. (Even of all the tests are green). [Ben Simo]
  • information from testing to aid in making business decisions

2012-12-09 12.14.32