To Transform Testing

There is no doubt that our long lived testing narrative is under pressure. To continue to bring business value, both the IT field and testing is transforming to be about proactive business enabling.

The IT domain is currently buzzing with the word “IT transformation” – the idea that IT services should be more about “inspire, challenge and transform the digital businesses“. That it should be less about delivering IT products and artifacts and more about enabling digital business outcomes. Even for testing – it should be less about a product/service, and more about business necessity. As Anders writes:

Stop focusing on the things that bug people, and dare to do both IT and testing smarter and more business focused from the start. Build Quality in – smarter from start. That goes for IT services as a whole, and definitely also for the testing activities.

What can you do to transform your testing? I have three areas:

Discuss Business Strategy

Learn Wardley mapping – and use it like Chris McDermott to create context specific maturity models with Wardley Maps informed by Cynefin. Use the mapping to Broaden the scope of the system under test.

Align with the Business Strategy

Leading Quality [Cummings-John, Peer 2019] has a whole chapter on “Align your team to your company growth metric“. Consider if the company you work for is driven by Attention, Transaction or Productivity metrics, and arrange your test activities accordingly.

Dare to Deliver in New Ways

We are usually talk so much about optimizing the (IT and testing) delivery, that we forget other ways to be innovative and provide business enabling. One way could be to dare use new technology like RPA or a HoloLens to support tedious tasks in testing – to use an existing product to something new. Another approach to actually test “all the things” that matter or to apply testing to IT outside the realm of application delivery.

To Transform Testing I will discuss, align and dare so that test solutions can be proactive business enablers – (not only achieve shippable quality).

Mapping Mondays – Pioneers, Settlers, Town Planners

Innovation in Testing

Let’s look at testing and test management as something you can build expertise in, thus it can be placed in various places on a wardley map. Similarly innovation activities in the field of testing can be modelled by “Pioneers, Settlers, Town Planners” [also originally swardley, article by Itamar Goldminz].

The model has three types of talent: those that experiment, those that build products and those that optimize the products/commodity. Shortly put:

Simple illustration of the Pioneers, Settlers & Town Planners Model.

Each group innovates, but there is also an built-in drive from experiment to product, to optimal commodity and back again as components to experiment on. As stated in the original article (from 2015) all three kinds are brilliant people. We can relate the model both to what value the customer looks for and what kind of activity the organization strives for. We can apply it for the broader testing field as not all testing is pure play experiments and not all testing is a commodity.

PST by @swardley

Examples of Pioneer experiments could be all the fuzz around RPA, AI and ML.. and square lashings on the System Under Test – on the technology side. On the practise side, it could be emerging practices of how to test in the space of infrastructure or IT service transition. It’s the “Pippi Longstocking” of – “I have never done that before, so I probably can“.

The settler activities are all about taking the emerging practises mature them and make them repeatable. Shortcutting the time to learn something or repeat some novel practise in a new setting. Some examples could be: A Practical Guide to Testing in DevOps, the shifts of testing (at their time of writing) as ways to codify emerging practices.

Example: In 2018 I did management of testing of a large enterprise IT transition of 700 servers, running 100+ applications – it was a novel first time, so we put together some testing practices that seemed to work (for that context). In 2019 I’m doing a similar transition of similar size, where we try to repeat the practices and approaches.

The brilliant quest of the settlers is to take ideas and built innovative and established solutions for the broader audience. Most settlers are probably framework (and content) creators .. not framework maintainers.

As soon as a practise has been established it’s up to the Town Planners to maintain and optimize the practices. To me, examples in this space includes:

  • Using Selenium to test web applications with
  • Using BDD/Gherkins for collaboration
  • Using agile practices and embed testing in the agile teams
  • … following the ISTQB cook book

You mind find it harsh that I group all of those practices together. To me, they are so established by now that they can be purchased. It’s a commodity market and it’s frowned upon if you don’t use it. But still – innovation happens and town planners do a brilliant job. It’s about faster, better, smarter – and especially about building more effective teams.

Also the Town Planners build the components that the Pioneers stand upon for their next novel idea. One example could be that to test web applications with code-less test an RDA tool utilizes the Selenium framework.

Broaden the scope of the SUT

When testers talk about SUT (System Under Test) there seem to be an implied context of it being software, developed, bespoke software to be specific. Let me broaden the notion of a SUT using Wardley Maps and with that illustrate how testing can add value across the board.

Bespoke software (aka Custom built) is where the solution (SUT) is built and maintained tailormade to a specific company by a specific team that answers to order giver. When you build and maintain an app or web site for a company and is embedded in the team delivering the code base – it’s usually in a bespoke context.

Experiment/emerging example: Built an internal web site to do some simple public service case management. Write it based on MicroSoft .Net and IIS technology. The solution is new and novel, so interaction with the user is important.

All the commotion on buiding MVP experiments and interaction with the Product Owner are usual symptoms of a genesis situation. As the processes mature and products emerge, the solution development becomes more an customization activity.

Customization example: Implement Dynamics 365, SiteCore, SalesForce etc – but taylor and code them to your specific purpose. I have worked in a project taking Dynamics 365O and creating custom forms to handle public sector health events.

The last class of software “development” is the pureplay configurations of standard solutions. This is the context of SaaS – pay the license and get started. Think SAP or Office Applications, anything that is so accepted that it’s almost free (OpenOffice) and kills the IT department.

Let me draw this on the axis of Wardley Maps Evolutions:

Similarly we can add the underlying infrastructure to the drawing. As solutions move to the cloud and infrastructure becomes code, the system under test could very well be code around infrastructure. Initially bespoke infrastructure experiments (in perl?), and as time moves – even infrastructure becomes a commodity in the form of Amazon S3.

So where is your SUT? – what is the path down the stack? Because there is a huge difference between testing custom code for cloud services, as compared to product customization on actual physical owned hardware.

Let’s think testing outside the bespoke areas on the map too. Some current examples I am working on are:

  • Infrastructure transition of 700 serves from being owned to being hosted
  • Application transition of 50+ applications from being owned to being outsourced
  • Transformation of a standard form management solution
  • Implementing a standard system for ITIL case management

These projects have no code, the SUT is either a server, an environment (collection of servers), a form, a process or something else. While we do know a lot about testing in bespoke software contexts, the practices for testing in transition and transformation are emerging practices! This gives us and extra layer. And this is where it gets interesting.

There are plenty of standard practices (SAFe, agile..) but the practices for testing in the context of transition is yet to materialize.

The same model can be applied to IT as a whole. IT support and end user computing (devices, desktop operation) are to the very far right as commodity services. While on the far left is the constant experimentation and tinkering (of AI, ML and RPA) to become actual products.

If we only see testing as part of building bespoke software we fail – we fail to see the horizontal and vertical contexts, where the testing disciplines can add similar value and impact.