Rant on Login Screen examples

If you are demonstrating testing technologies or testing examples around RPA, ML, Selenium and so on – Please: DO NOT USE A LOGIN FORM!

The test scenarios I usually deal with are not this… mundane. While a few testers probably still have to build login forms from scratch, a login feature is a commodity by now. Use OAuth for public facing sites and Active Directory Federation internally in the organisation. Really – there’s no need to reinvent the wheel. To the end user and even the Product Owner logging in is just a stepping stone. 

I just want to log in, and then I’m done for the day

Said no user ever*

Showing that you can train an AI network or other framework to login might solve a tedius testing task, but is usually not the thing I’m after. When a user is logged in, they are there to solve something, to process something, to do something – to engage with something. And this is where the best tests are heading too – this is the tests that adds value to the business and tells something about the product. 

For instance: In one project I did, we disabled the login entirely to make the CI/CD run feature testing. The plain login screen was temporary anyways, as the solution authentication would be based on certificates. We never spend much time on it, neither on total coverage.

Less combinations – More Real Life Scenarios

So what can you use as an example instead of login boxes and combinatoric bar stories? How about anonymizing the latest test you had to do on your latest live-action testing project? This will tell me about your challenges, your business domain and when the last time was – among other things.

Let me start, as of writing the latest test case I touched (same day as writing this) was for a new public data registration project. The tester and end user “subject matter expert” was testing the data registration form from both a GUI and web-services perspective. Export the entered data form as XML and import it again via web-services to see the consistency.

Could that be tool supported – maybe. The knowledge about the system is not very explicit. It’s a bit complicated, actually. Could it be trained by an ML/AI system – I doubt it. There is no global training set for this class of systems – we have the old version but are adding new things. 

If you are (a tool vendor) demonstrating something – do try to understand the context and problems your customers are trying to solve first. Ask them what their latest tests were and where the challenges are. If it’s a login box you’re good to go, but I doubt it. 

*: Unless you are somehow measured on “logging in”. For instance to claim unemployment benefit, login to job site daily…Snap! OTOH shallow measure.

http://rebrick.lego.com
Advertisements

The Guilty Tester Episode 7 – – With A Little Help From New Friends

Do you ever feel guilty for not meeting the standards set by others in the Software Testing community? You’re in the right place then.

In this episode I talk to Dave (@theguiltytester). We discuss traditions, open questions and how to work within contracts which are specifically requesting traditional test practices based on large numbers of test cases. 

Listen and read all about it here: http://theguiltytester.libsyn.com/the-guilty-tester-episode-4-jesper-ottosen-with-a-little-help-from-new-friends 

Some of the blog posts mentioned are: 

New friends - the subject matter experts of all trades
New friends – the subject matter experts of all trades

You don’t have to be a boss to be a leader

It’s really that simple, yet awesomely profound. And a typical Gerald Weinberg quote, like my other favorites of his points:

  • No matter how it looks at first, it’s always a people problem (The second law of consulting)
  • You’ll never accomplish anything if you care who gets the credit
  • Things are the way they are because they got that way
  • Quality is value to some person

Regarding the last quote; which was later extended with “who matters, at some time” by Bach, Bolton. Once I had an argument about how to deliver quality. The other side held towards IEEE definition of delivering the expected. But even when he did, he failed to see that the unmeasured and irrational parts affected the value to the customer. I agree completely with The Cowboy Tester that knowing works of Weinberg is a measure of seriousness.

Weinberg worked not only with testing, but among other things also consulting and organisational change management. I did not know that when reading “Making Sense of Change Management” (Cameron & Green 2012). I literally jumped up and started laughing while reading the quite serious elaborations to the Satir Change model – the authors found that Quality Software Management: Anticipating Change (1997) is a “masterly book on change, but with a title that might not appeal to everyone“. It might not appeal to change scholars, but definitely appealed indirectly to a lot of people in testing.

Recently (August 2018) Jerry died aged 84. Not a boss – yet a leader.

A 30 Days Agile Experience

In September 2017 the Ministry of Testing had a crowd-based knowledge sharing event called “30 Days of Agile Testing” with a small learning activity for each day of the month. As with the similar security event I set up a weekly schedule at work to meet for an time-boxed hour and discuss 3-5 selected topics each time.

Our score was 17 topics discussed – some more discussed than actually tried out. Hence the half marks on the poster on the window below. Me and my coworkers work on many different teams – so to dig into specific team tools and processes was out of scope.

Here is a few of our findings:

IMG_0007

Links to “the Club” on some of the topics we selected:

 

 

Less Test Managers, More Test Coaches

One of the trends/shifts I experience in testing & test management in particular is the Test Coach as discussed initially here: The Shift-Coach Testing Trend (Oct, 2016). Recently (Aug 2017) it came up again in a Twitter thread, where Stephen Janaway stated the inspiration to the title of this blog post.

Less Test Managers and more coaches. That’s how I see it going.

Fittingly as he inspired the first post with his talk “How I Lost My Job As a Test Manager” presented at Test Bash 2015. This post is a further elaboration of the Shift-Coach test management trend. Here are some of my experiences:

  • I have been assigned to an agile development team to introduce them to 3 Amigos, Test data driven test automation and such things. The purpose of my involvement was to enable the team to continue the practices without me, and without testers besides the business analyst / product owner (See The domain expert is the tester) as they are doing Shift-left. Similar to an agile or scrum coach, my approach was to look at it as a change in the way of working.
  • Another project is an infrastructure project, there are no testers only technicians configuring Cisco routers that by software can replace firewalls, iron ports, VM servers and other network equipment. The project has to implement 80+ of these, so I setup both a test process and an ITIL change request process acting as a test and release manager – another quite frequent trend. I could continue in the project for the duration, but instead I setup guidance and leave when it’s sufficiently in place.

This might be similar to a test architect, a (internal) test consultant activity. It has nothing to do with diminishing testing. Rather I see it as more testing happening, something that would not have been done without the coaching from a test manager. It’s all about finding a test approach that is fit for the context.

Here are some things others have written:

The competence of the test coach is to have enough change management expertise (people skills) and test management expertise (domain skills) to know how to coach and facilitate the change. Should test coaches test too, perhaps when required, but not necessarily. The activity is primarily to up-skill the team to continue on their own.

The “Test Coach” is a trend similar to “shift-left” and all the other shifts in testing and test management. I see it as a pattern, and what I read from the threads and discussions is that many test managers gradually shift towards test coaches.

2017-07-03 13.57.42

Don’t request the kitchen sink

More and more often I see outsourcing contracts that requests 10-15 test phases. It looks like someone has simply thrown the book at it, and not considered if it is an infrastructure project, a software development project or COTS implementation or – what on earth, they actually want to learn from the testing.

These are the phases of a recent project:

  • Environment Acceptance testing
  • Hardware and integration testing
  • Component testing
  • Component- integration testing
  • Installation test
  • System testing
  • Functional testing
  • Regression testing
  • Security testing
  • Performance testing
  • Operational acceptance testing
  • Service Level testing

It’s a challenge in the vendor reply. The vendor will want to reply to all test phases in order to be compliant with the tender, and not loose points. There is no room for elaboration or discussion if you want in on the bid process.

Quite often the requester comes back and say “we didn’t want all those weird testing things, we just want something that works for us”. But when the contract is signed and the work set in motion the project team have challenge to make the testing practical within the framework of the contract. This goes from both sides. Many good hours can be wasted with unwinding cumbersome contractual terms.

What I usually do in such a situation is to bundle the contract’s testing scope into fewer activities, and setup a mapping so that everything is covered. That is if the client allows me to make the activities practical and context-driven. If not – my hands are tied, and we deliver according to spec – even if the chapters of the test plans are set in stone.

Let’s work towards better deals for testing activities. If you are looking to prepare a BID include a test manager – and have a discussion of the value-add and learning of testing up front. There is no one book of how to do testing. Instead spend the time and money figuring out your context. Figure out what phases are on the client side, and what is on the vendor side. Have a test management consultant on retainer for before and after the bid process. Do something to discuss your test strategy and put the guidelines in the contracts, so that the vendors can propose a solution.

Don’t request everything and the kitchen sink too

Everything and the kitchen sink
Everything and the kitchen too

 

 

Test ALL the things

TL;DR: We can add testing to all requirements and all business risks. Testing to document requirements and to debunk risks provides valuable information for the business. Let us not limit testing to things that can be coded. The intellectual activity of trial and learning is happening anyways, we might as well pitch in with ways to find important evidence for the decision makers.

Test all the requirements

Traditionally testing was all about testing the functional requirements that could be coded. Non-functional requirements was left for the specialists, or plainly disregarded. I know I have done my share of test planning, with a range of requirements left “N/A” with regards to testing. Especially performance scope, batch jobs, hardware specs, data base table expansions and virus scanning have been left out of my functional test plans…

When I look at a list of requirements now – I see that we can indeed test all the things, or we can at least work on how to document that the requirement is fulfilled. Some of the requirements are actually quite easy to document. If it’s on a screen somewhere, take a screen shot and attach it to a simple test case. Done deal really. Additionally with a testing mind-set I can think of ways to challenge the details. But do we really, really need to fill up a disk to establish if it’s exactly a 1 Gb allocation – probably not. Do we really really need to document all requirements – yes in some contracts/contexts it’s important for the customer to know that everything has indeed been established. Sometimes the customer doesn’t trust you otherwise, sometimes the tests are more about your ability to deliver and provide evidence that matters.

Test all the business risks

Look into the business case of your project and find the business risks. Sometimes they are explicitly stated and prioritized. A a recent Ministry of Testing Meetup we looked into a case for a large national health system. We looked at the tangible benefits, intangible benefits and on the scored business risks.  What worried the business and management most was budget, time and whether the new system would be used in a standardized way. There is an opportunity for testing here to help address, document and challenge the most important business  risks. Traditional testing would usually look at functional requirements that can be coded or configured, and miss totally to address what matters most to the business.

OK, how do we test the project costs? How about frequent checkpoints of expected spending – would that be similar to tracking test progress. Perhaps – let’s find out. Testing is all about asking questions for the stakeholders and solving the most important problems first. Can we help to analyse risks and setup mitigation activities – sure we can. We just have to step out of our traditional “software only bubble”.

MEME - Test ALL the things
Meme ALL the things

 

 

 

 

 

 

 

Read also: Many Bits under the Bridge, Less Software, more TestingTest Criteria for Outsourced SoftwareThe Expected, Fell in the trap of total coverage.

Links: “A Context-Driven Approach to Delivering Business Value”, Cynefin In Software TestingTesting during Application Transition Trials