In Charge of Testing

As a Test Manager I oversee the testing in a project or program – I am usually the only testing specialist in the project, so, I need the right leadership skills and the right tools to succeed. I have to own the data about the testing and quality activities.

As the test manager I need to facilitate a quite a range of testing activities:

I need to balance that I need to know what’s going on (with regards to testing) but without micromanaging the people being involved in testing and quality activities. My role is to facilitate that testing things happen – like the project manager making project things happen. I cannot own the activities without owning the data about it. I need to cover the full spectrum of tests – from engineered (RDA and CI/CD) to people-based (scripts and exploration).

The most practical tool for a test manager with this scope is PractiTest, as there is more to testing than just the test cases [2]. The old term “ALM” [3] comes to mind – it is still relevant when I look for a full test management tool. I need to cover both the “inputs” to testing (requirements, tickets and user stories) and the “outputs” (bugs) in one location. I need the requirements and user stories in my tool, as I need to base my test analysis and planning on the delivery model (that may not always be agile). I need the bugs in the testing tool too, as bugs can happen in any work product of the project: documents, code base and even the tests. PractiTest acknowledge that there is more to IT projects than code.

I appreciate the key driver of PractiTest – that all activities happen in-flow. You don’t have to change window, stack pop-ups or go to another tool in order to run the tests or create bugs. Creating bugs happens in context of the test case and seamlessly moves all data about the run to the bug. Everything you need to do is context-based, and available to you on screen. And it has some cool features of read-only links to graphs for management reporting, and a smart built-in “rapid reporter” for exploratory testing notes.

It can be a challenge to switch to PractiTest if you are in a compliance setting, if you need on-Premise or if your team generally uses Azure DevOps (the tool formerly known as TFS). To get the full potential of Azure DevOps, though, you need the full Microsoft Test Pro licenses, so it’s not a free tool either – nor is DevOps intuitive for testing things doesn’t have the code available. As with Azure DevOps PractiTest is also SaaS only, with multiple data centers for regional data compliance. As there is always inertia towards a commodity it won’t be long before there is no good arguments to have test management tools on-Premise and for the tool vendors to provide the compliance certificates (ISO/SOC really should be sufficient, IMO).

Out of the box PractiTest supports the categories of testing above (engineered, scripted, exploratory) and has the necessary integrations too: Surefire for unit testing, Maven for CI/CD, Jira, ServiceNow or any other ITSM for requirement input. There is even a two way integration to Azue DevOps. As the web design is “responsive” it could probably run off a tablet. That would enable easier test documentation for field tests. It would be even better to have a small version of it on a phone and be able to use the camera for “screen shots”.

At work I am currently running a large project regarding customizing and implementing a standard commercial software system, PractiTest would fit right in, as we have the following test activities:

  • Unit test by the developers
  • Automation by test engineers
  • Exploratory test by Subject matter experts
  • Formal scripted testing with end users

And I need to own the data around all of this, if I want to in in charge of the testing (and not only the testers). We are very few software testing specialists on the project team, but as the manager of testing I need to cover many other people performing the testing. This transforms my role from test management to one about leadership, coaching, and facilitation of testing being performed by the SMEs – and anyone else really.

I will be talking about Leading When the Subject Matter Experts Test at ConTEST NYC 2019 until then read more about leadership:

  1. Anthropologists and similar humanities educations can be great BA’s
  2. looking at you Test Rail ūüėČ
  3. ALM = Application Life Cycle, like Micro Focus Quality Center etc.

Disclaimer: This is an influencer review sponsored by PractiTest.

The Expected

Many test processes and test tools mention that you have to establish the expected results to every test. It is at best risk ignorance not to take the notion of expected results with a kilo of salt #YMMV.

If you can establish the result of your solution in a deterministic, algorithmic and consistent way in a non-trivial problem, then you can solve the halting problem. But I doubt your requirements are trivial.. or even always right. Even the best business analyst or subject matter expert may be wrong. Your best oracle may fail too.  Or you may be focussing on only getting what you measure.

Nailed it
Nailed it

When working with¬†validation¬†in seemingly very controlled environments changes and rework happens a lot, as every new finding needs the document trail back to square one.. Stuff happens. Validation is not testing, but looking to show that the program did work as requested at some time. It is a race towards the lowest common denominator. IT suppliers can do better that just to look for “as designed” [1].

Still the Cynefin framework illustrates that there are repeatable and known projects, and in those contexts you should probably focus on looking to check as many as the simple binary questions in a tool supported way, and work on the open questions for your testing.

Speaking of open ends Рevery time I see an explicit expected result I tend to add the following disclaimer inspired by Michael Bolton (song to the tune of nothing else matters [2]):

And nothing odd happens … that I know of … on my machine, in this situation [3]

And odd is something that may harm my user, business or program result

Significantly…

But I’d rather skip this test step¬† and work on the description of the test and¬†guidelines to mention:

And now to something completely different:

See also: The unknown unknown unexpressed expectations, Eating wicked problems for breakfast

1: Anyone can beat us, unless we are the best, todays innovation becomes tomorrows commodity

2: https://www.youtube.com/watch?v=tAGnKpE4NCI 

3: I’ve heard that somewhere…

Test Like Sherlock

Sometimes testing is like being Sherlock Holmes – You find your clues hidden in plain sight: Where the users scratch their nails; how the application user interface is cobbled together; odd patterns in the error logs….

But seldom without experimenting, seldom without pushing the subject under test or consulting the weather report, time tables – and getting out in the rain, doing some footwork.

He always seems to know better, always asking questions. He is so passionate about his problem solving skills that his standard by default seems arrogant [1]    (but that is usually not on purpose).

This is very clear in the recent BBC TV Series “Sherlock” – that illustrates and mentions his Asperger clearly. Almost on par with the¬†The Curious Incident of the Dog in the Night-Time. Still when he is out solving mysteries he is a hero – if there ever was one. [2]

1:¬†If your standard is to never be called arrogant, you’ve probably walked away from your calling.¬†http://sethgodin.typepad.com/seths_blog/2014/12/in-search-of-arrogance.html

2:¬†Don’t make people into heroes, John. Heroes don’t exist, and if they did, I wouldn’t be one of them.¬†http://en.wikiquote.org/wiki/Sherlock_(TV_series)

deerstalker

Related:¬†The yardstick of mythical¬†normality,¬†People are people ‚Äď despite their¬†labels,

QA Aarhus – Exploratory Testing How and When

QA Network Aarhus is a local non-affiliated network of testers (and good friends) in Aarhus. Where I had the great pleasure of talking about Exploratory Testing. This is the link collection, the slides are attached.

nnit

Continue reading

When good enough is the perfect fit

Similar to scope creep¬†we may also experience “test creep“. Test Creep is when the tester adds more tests than what is in scope. Just as well as more business functionality is added during scope creep, more testing is added in test creep. Both aren’t necessarily bad, but in time-boxed or similar (budgeted) constraint projects creeping isn’t necessarily value adding. This is probably easier to understand in an agile project focusing on minimal viable product¬†, but may happen in other contexts too.

It is test creep, when the tester feels an obligation to run an extra drill down into browser and OS configurations, when scope is less broad. It is creeping the scope of testing, if the testers feels a “/need/ to write testcases for this first” when exploratory sessions fits the mission. Consider test creep like gold plating, in that way that it tries to refine and perfect the product – when good enough is the perfect fit.

Test creep can happen intentionally, happen by management or by product owner request. It may happen unintentionally, and usually it is with the best intentions – as more testing always is better testing – right? (But it Depends) Sometimes yes, we as testers are to blame that we add more scenarios, rigor and details, because a testing mindset drives us to investigate the product.

In discussing this with Mohinder and Darren, we found that – it’s not only a matter of removing wait time for testers. This may add more time, to test but the scope creep in testing may happen none the less. A Lean mindset with focus on what adds value to the business and a discussion on the minimum viable testing¬†will assist the project in avoiding test creep.

 

minecraft_creeper_wallpaper_by_lynchmob10-1440x900

Mindmaps for 400

Finally non-profit self-organizing software testing is happening in Denmark. On may 21 2014 we actually had two events:

At the first I was glad to share my experiences using mind maps in software testing, note taking and information structuring. (Get the PDF Xmind mind map here: https://jlottosen.files.wordpress.com/2014/05/mindmaps-400.pdf)

You stop going deeper down the tree, when there is nothing more knowledge to gain, just like good (exploratory) testing.

Cultural context of the “for 100″ comes from the Jeopardy TV Quiz, where the questions come in 4 levels: 100, 200,300, 400 for the ¬†increasingly harder questions. The prize is similarly $100 for level 100 etc. ¬†

FDA, Exploration and time to information

A key driver in implementing enterprise knowledge management is to reduce time to information (77% are seeing faster access to knowledge). But that goes for LinkedIn and Twitter too. Using twitter professionally helps you meet the famous people and help you see the communication layers at conferences. Case story: Today I was reading about test processes in a regulated environment, and got curious towards exploratory testing in that context. So I reached out to the #twitterbrain and asked the giants, whose shoulders I am standing on*:

Within 2 hours I had both relevant references, a debate on the pitfalls and base for further details. Follow the tread of this tweet: https://twitter.com/jlottosen/status/411473074312052736 

*: really, not to brag – I have met both Griffin, Johan and James, and they know me too ūüôā