Broaden the scope of the SUT

When testers talk about SUT (System Under Test) there seem to be an implied context of it being software, developed, bespoke software to be specific. Let me broaden the notion of a SUT using Wardley Maps and with that illustrate how testing can add value across the board.

Bespoke software (aka Custom built) is where the solution (SUT) is built and maintained tailormade to a specific company by a specific team that answers to order giver. When you build and maintain an app or web site for a company and is embedded in the team delivering the code base – it’s usually in a bespoke context.

Experiment/emerging example: Built an internal web site to do some simple public service case management. Write it based on MicroSoft .Net and IIS technology. The solution is new and novel, so interaction with the user is important.

All the commotion on buiding MVP experiments and interaction with the Product Owner are usual symptoms of a genesis situation. As the processes mature and products emerge, the solution development becomes more an customization activity.

Customization example: Implement Dynamics 365, SiteCore, SalesForce etc – but taylor and code them to your specific purpose. I have worked in a project taking Dynamics 365O and creating custom forms to handle public sector health events.

The last class of software “development” is the pureplay configurations of standard solutions. This is the context of SaaS – pay the license and get started. Think SAP or Office Applications, anything that is so accepted that it’s almost free (OpenOffice) and kills the IT department.

Let me draw this on the axis of Wardley Maps Evolutions:

Similarly we can add the underlying infrastructure to the drawing. As solutions move to the cloud and infrastructure becomes code, the system under test could very well be code around infrastructure. Initially bespoke infrastructure experiments (in perl?), and as time moves – even infrastructure becomes a commodity in the form of Amazon S3.

So where is your SUT? – what is the path down the stack? Because there is a huge difference between testing custom code for cloud services, as compared to product customization on actual physical owned hardware.

Let’s think testing outside the bespoke areas on the map too. Some current examples I am working on are:

  • Infrastructure transition of 700 serves from being owned to being hosted
  • Application transition of 50+ applications from being owned to being outsourced
  • Transformation of a standard form management solution
  • Implementing a standard system for ITIL case management

These projects have no code, the SUT is either a server, an environment (collection of servers), a form, a process or something else. While we do know a lot about testing in bespoke software contexts, the practices for testing in transition and transformation are emerging practices! This gives us and extra layer. And this is where it gets interesting.

There are plenty of standard practices (SAFe, agile..) but the practices for testing in the context of transition is yet to materialize.

The same model can be applied to IT as a whole. IT support and end user computing (devices, desktop operation) are to the very far right as commodity services. While on the far left is the constant experimentation and tinkering (of AI, ML and RPA) to become actual products.

If we only see testing as part of building bespoke software we fail – we fail to see the horizontal and vertical contexts, where the testing disciplines can add similar value and impact.

Advertisements

Shift-Deliver, TestOps and ITIL Changes

Shift-Deliver is a label I choose to put on the changes the roles and activities of the TEST MANAGER, when the test manager moves towards (also) being involved in the ITIL change requests, delivery management, configuration management and branch management that happens when the solution goes from the test phase to production. Another label could be “TestOps” as presented here, as the intersection of Testing and Operations. TestOps have been identified for along time. ….Interesting.  ūüôā

In my IT outsourcing context, this is less about software, and more about solutions. In at least two of my long term enterprise scale projects, half the job was test management (of operations) projects, half the job was regarding ITIL change management. My change management activities was mostly making sure that

  • the process was followed
  • that information was provided to the stakeholders
  • that testing happened
  • risk mitigation happened

I was hired as ‚Äúthe quality guy‚ÄĚ, but expanded the role over the time I‚Äôve been on the team to take ownership of all of our build and release infrastructure as well. Basically, I‚Äôm responsible for everything from the moment code is checked in, until it hits our production servers 

To use a quote by Alan Page. Again Alan is a representative of what happens with regards to trends in testing. He might be wrong, as well as I. I try to label the trends to understand them. These four trends that I have spotted are not mutually exclusive, neither do they all four need directions. Change is happening to the classic test manager rolle of going through the motions of test cases and documents. This is clear when looking into these posts:

Initially I discussed Shift-Deliver, Shift-RightShift-Left and Shift-Coach  at Nordic Testing Days 2016 during the talk ‚ÄúHow to Test in IT operations‚Äú and coined the labels on the EuroStar Test Huddle forum.

DSC03562

Pizza as a Service

Snitched off the intarwebs* an illustration of Pizza as a Service. Testing services can engage with the recipient in the same way, we usually refer to this as the engagement model. I have personally experienced all the variations:

  • As a company I want deliveries, – don’t bother me with the rest
  • As a company I want deliveries, a test plan and a test report
  • As a company I want deliveries, test documents and I want to determine the tests
  • As a company I want deliveries, documents and I want to test some myself
  • As a company I want to do everything myself

pizza services

*: Credit to the maker, where ever she is.

On applying a single method

[ Simon Wardley, Leading Edge Forum: Oh not again Рshould we be an agile or six sigma shop? ]

First, every system of any reasonable scale consists of multiple components. Those components are all evolving (due to the effects of competition) and you can (and should) map out the components of any system before embarking on trying to build it. Below in figure 1 is a basic map from a heavy engineering project with a large IT component

Now, all the components are evolving from left to right and as they do so their characteristics change Рthey move from an uncharted space (the novel, the chaotic, unpredictable, uncertain, potential differential) to the more industrialised (the common, appearance of linear order, the predictable, the certain, the cost of doing business). 

The same thing with your testing – that is: If you dare to take a holistic approach and not only focus on the mechanics.¬†See also: ¬†Mapping testing¬†Competencies¬†,¬†Learn to think like a¬†business,¬†When do testing¬†happen?¬†3D model for testing¬†contexts¬†Black or white ‚Äď it is the same¬†box

I killed an IT department

It wasn’t just a glitch or a bug, or a wicked hack. It is gone¬†– there is no IT department anymore … Staffing and services will be transferred to the communications & knowledge department, but the hardcore business of developing IT solutions is closing. From now on we primarily use customization and configuration of standard tools: Facebook, LinkedIn, WordPress, Podio and email (sigh – still).

Yet IT is everything and everything is IT

Continue reading

Wither the test manager?

[ Paul Gerrard | Will The Test Leaders Stand Up | The Testing Planet issue 10, Page 4, March 2013 , Paywall] paraphrased:

There are five broad choices for you to take if you are a test lead or test manager:

  1. Provide advice to the business leaders, as an independent agent cajole project leadership and review performance
  2. Take control of the knowledge required to define and build systems. Demand clarity and precision in requirements
  3. Help agile projects to recognise and react to risks, coach and mentor and manage testing
  4. Manage the information flow between the key groups and continuous integration system, control change and delivery
  5. Manage outsourced and offshore teams, manage relationships

It’s probably 6) All of the above

legodinosaur

for heaven‚Äôs sake, don‚Äôt make any changes to our mother-ship

[ The undisputed facts about outsourcing, Part 6: Europeans love money, but hate change |  JULY 18TH, 2011 | http://www.horsesforsources.com ]

Ah‚Ķ mes amis! ¬†Let‚Äôs rip out ze costs, but for ‚Äėeaven‚Äôs sake, don‚Äôt make any changes to our mother-ship. ¬†By all means, sack all the expensive foreign staff in the vorldvide offices and sheeft ze vork to India or Les Philippines, but ‚Äď we repeat ‚Äď don‚Äôt CHANGE anything!

See also: Are you an outsourcing pro, or a global business pro?