The domain expert is the tester

Leave a comment

Sometimes the best tester is the domain expert, the person that knows all the in’s and out’s and corners of the system. I have worked with testers that have had hands-on on a system since the late 1970’es, but I also know testers of mobile app’s that marvel in being the subject matter expert of the domain. Sometimes the professional tester doubles as agile Product Owner(1) too given her vast knowledge. The tester becomes the the SME …

The subject matter expert, though, is usually a business analyst, or perhaps a User Experience expert. Those persons might have a better stand to be testing the system, than testers with no prior knowledge. Often the SME is the best tester available. I see this happening in a shift-left setting – but also in settings with a heavy user and business involvement. Like SAP releases to enterprise systems – where the business users and SAP architects still spend a month off their “actual” work (user acceptance) testing corporate configurations and customization.

The UAT is not dead, but the classic role of the tester testing on behalf of the business is declining. The business would rather test their own, with in-house subject matter experts. The field is active, as there is tool support for this activity. Panaya(2) is a tool that specializes in managing the UAT of a corporate system like SAP, and one of the key elements is that test cases can be broken up in steps and handed over between persons. Not even classic HP ALM’s handle hand-over between testers well. While ALM’s support that the tester does the testing, Panaya supports that tests are distributed across many people. People that have other (“real business”) tasks during the work day.

Testing can also be pushed even further out to the users with crowd-based testing, beta releases etc. In both crowd-based and UAT-based testing, the role of the pro’ tester is missing but the testing is still happening. IT’s being done by the most skilled – most valuable for the task.

So what can we as testers do when our tasks are gone – skill up, go with the change and become the expert – or move out to other skills: Coaches, delivery leads etc.

2015-12-18 13.49.28

The expert says 5 pieces should be in the build – though the customer is OK.

  1. If she doubles as scrum master, she’s probably more a Test Coach
  2. this post is not sponsored. I’m just making observations – not recommendations.

 

Testing across the IT landscape

Leave a comment

Good testing is much more than confirmation of explicit requirements. It’s figuring out the implicit requirements, what blocks the business and what drives the business. Businesses are not driven by SDLC’s but by decisions and strategies. IT road maps are just a representation of the business strategy, an engine to build business solutions on. The is much more to the business than the software solutions and related risk mitigation.

Very often the biggest business risks are outside the project scope. When we look at it this way we see that testers and testing activities has an opportunity outside the classic project life cycle. Testing is about experimenting with a IT solution, to evaluate if it fits the business requests. IT solutions that supports the business exists in many forms, I am certain that explicit testing (*) can add value in other parts of the IT landscape.

Here is a model of an enterprise IT landscape consisting of business ideas, solution development, operations and end user devices + support. Solution delivery is boring in the sense of well-known software creation and maintenance. What if the item under test and the requirements are around network, servers, end-user devices and IT support tickets. I am certain that it’s valuable to test business cases before the project is even formally assembled.

 

*: implicit testing in the form of critical evaluation always happen. .. similarly does checking.

Testing for Create, Maintain, Move and Close of Applications

1 Comment

Usually when we talk testing it’s about the road to production. It’s about getting requests from the customer/Product owner and shipping it. We tend to forget that there is more to the life cycle of application than adding to the pile. Inspired by the old CRUD I identify the following stages in the application life. Create, Maintain, Move and Close. Testing can add value all of the four modes, with twists for each one.

Create: “oh shiny” – Creating an application is usually novel, but the more times you have build similar applications it becomes routine. Some applications have to be build from scratch, others merely configured. It matters a lot if you are building a unique app – or if it yet another roll-out of a COTS application. The testing in “create” usually focuses on bugs to be fixed before go-live, and very little on what happens afterwards. Building a new  application is usually a strategic decision to the business that solves a problem or builds on a potential. Requests are numerous for new things.

Maintain: “ship, but don’t destroy production” – At some point the customer sends you more requests to the stuff already build than new features. Application maintenance is all about balancing new features and updates to existing features. Existing features are being used by the end users, and they will eventually request updates and bug fixes. Fragmentation, merging and branching becomes and issue – especially if you maintain the application as a solution for a range of customers. Customers might want to differentiate between their requests – as they won’t want to pay for bugs in previous releases, but rather want to pay for new additions.

Move:  “It has to work as before, just with a new team“. To many businesses maintaining an application is not their key area; They might be a public organisation with no need to have their own staff of developers. So “Application Outsourcing” becomes a thing for many applications, and with deals being won and lost – it will happen that  the development tasks moves from one supplier to another. Testing can make sure that processes are in place in the new location and that the state of the application is known in the new location. The testing during “move” doesn’t involve the functionality of the application, but rather the ability of the new team. Sometimes the hosting of the application stays they same,  in other cases it is the hosting of the application that changes and nothing else.

Close: test that it’s gone” Sometimes IT strategies and businesses decide to close down an application. Perhaps it’s being replaced, perhaps it’s redundant after a long time. Examples could be hospitals moving from one patient journal to another or whole systems being sunsetted. It could also be the closing of end-of-life applications (Windows servers, HPQC etc). The value to the business is knowing the application is gone, and the information in the old systems not trusted anymore.

It is very much possible to have testing in all modes of the application life cycle. Similarly it is very much possible for testing to add value in all stages of the software development life cycle. It’s a matter of perspective.

Getting Testing in Early

4 Comments

Even before there is an “system development life cycle” – testing in the form of thought experiments and  evaluation can take place and add valuable information to the context.

My test management tasks are often about the next thing coming up. Bids for outsourcing agreements and application development often comes with a large document of test activities to be answered and elaborated. In this role I am the the subject matter expert (in test), and while have to write the tender reply for my domain. Sometime down the line the bid materials becomes an actual project, but by then I’m onto the next thing.

Sometimes I draw an analogy to the Secret Service advance team arriving two weeks before the president, setting up protection and identifying gaps – while then moving on to the next location before the president even gets there.

Another example of advance work for test people, is where the organisation uses frequent releases of systems. While the majority of the test effort is put into the release currently being tested, some effort must go into looking into the frame of the coming release. In the coming release the test manager can look for headlines to test, review initial high level design and find flaws and conflicts in the release content.

Sometimes I draw the analogy to the blue and gold teams of US nuclear submarines. While one full crew is out sailing/delivering, the shore team prepares, trains for the next big push.

Testing early can also be in the form of running simulations on various business case scenarios. Business simulations is all about experimenting and evaluating. For novel solutions prototyping, wire-framing and user experience activities helps develop minimum solutions to be tested for viability by the customer.

In the article “Continuous Testing in Dev Ops…” we see testing happening during Plan and Branch. In the article “A Context-Driven Approach to Delivering Business Value” testing can help establish viable market, match to vision and in identifying business risks.

testing related to revenue generation may focus on functionality or regulatory compliance; testing related to revenue protection may focus on maintainability or legal defense; testing related to supporting revenue may focus on business process improvement or cost reduction.

Testing is a lot of things – also outside the SDLC.

 

Testing roles are shifting

1 Comment

New ways of delivering software and solutions challenge existing perceptions of what roles and activities testers and test managers have. Some are getting more into development others into people skills. Sometimes forcefully, other times as part of self-organizing teams.

This post is a link collection for the talk: Testing roles are shifting, but where to? for the Ministry of Testing Meetup Copenhagen chapter, march 2017.

Introduction

Shift Left:

Shift Right:

Shift Coach

Shift Deliver:

 

Additional links

Continue the discussion Where are testing roles shifting to?

Co-creating Smarter Tester

Co-creating Smarter Tester

3 Sessions of Security Testing

1 Comment

One way to collaborate in a team is to achieve shared knowledge together. An example of this is the online activity of “30 days of testing” that The Ministry Of Testing has been putting out to the online community to participate it. My test team has a “Work Group / Special Interest Group” with regards to security testing, so when a 30 day challenge for security testing came up, we scheduled sessions to learn from the topics provided (see below).

As we are testing consultants doing work for our customers, we scheduled 3 sessions – initially for an hour. At the start of the hour we picked 4-5 topics from the list, and worked our way through them in a prioritized order – within the time box of the hour. Come to think of it we might as well have used the Lean Coffee format. As we have team members two places in DK and one place in PH, it was a skype call using screen sharing. After the call I  summarized sending out a “link mail” to all in the testing group (DK and PH). Evaluating the sessions we extend our ordinary scheduled WG meetings to make room for collaboratively investigate additional security testing topics.

12 From the list: ZAP, Google Gruyere, threat models, HTTP proxies, posture assessments, tiger boxes, recent hacks (elaborated by Troy Hunt), OWASP top 10, OWASP SQL injections, adding data integrity testing into a test plan, share ideas for security testing internally and externally, discuss security testing with regards to EU GDPR compliance.

7 Not on the listNaughty Strings form GitHub, Bug Magnet plugin, How real persons names trick IT systems, how to be careful with custom license plates, DDoS attacks, IoT privacy failures, Chaos monkeys/Siamese army and little Bobby Tables:

exploits_of_a_mom

XKCD: Exploits of a mom

To sum up, we have learned about: what tools that can make testing easier, where to read about vulnerabilities and and simple exploits, understand how personal data and logins are used and stored, how to pitch security testing based on fear of breaches and safety concerns, testing the requirements for “by design” security.

30 Days of Security Testing

30 Days of Security Testing

Many Bits under the Bridge

1 Comment

I’ve been in the testing business for 14 years – when I started in late 2002 it was all about using HP Test Director 7.6 – in a browser… There was only one model of testing, v-model, and only one book of testing the ISEB (later ISTQB) vocabulary. And only one expected output of testing: Testers designed test cases, executed and perhaps wrote both a test plan document and test report document. Test process improvement was a thing, but even so testing was often a pointy cog…  

Many Bits under the Bridge Later

It is not about the test cases any more, it’s about being part of a team – that delivers an IT solution to the business. First of all, if it’s just about the test cases then it is a race to the lowest paid off-shore location, a run to the bottom in repetitiveness and mechanic activity. Checking! with more focus on crossing the t’s and dotting the i’s. We have tools for that now – the plates are shifting.

When testing professionals puts “writing test cases” on their LinkedIn description. It seems to me that they are stuck in the testing world of 10 years ago. Standing still and not seeing that Testers are Knowledge Workers – not workers of producing artifacts. It is much more important to see beyond the visibleUncovering better ways and seeing testing as an activity to provide information to the stakeholders, based on experiments and observations.

Skill up and be smarter! And don’t listen to old tapes – it’s not worth it :).

See also this from QA Symphony & Ministry of Testing:

The Software Tester: Modern vs. Traditional [Infographic]

The Software Tester: Modern vs. Traditional [Infographic]

Older Entries