Testing across the IT landscape

Good testing is much more than confirmation of explicit requirements. It’s figuring out the implicit requirements, what blocks the business and what drives the business. Businesses are not driven by SDLC’s but by decisions and strategies. IT road maps are just a representation of the business strategy, an engine to build business solutions on. The is much more to the business than the software solutions and related risk mitigation.

Very often the biggest business risks are outside the project scope. When we look at it this way we see that testers and testing activities has an opportunity outside the classic project life cycle. Testing is about experimenting with a IT solution, to evaluate if it fits the business requests. IT solutions that supports the business exists in many forms, I am certain that explicit testing (*) can add value in other parts of the IT landscape.

Here is a model of an enterprise IT landscape consisting of business ideas, solution development, operations and end user devices + support. Solution delivery is boring in the sense of well-known software creation and maintenance. What if the item under test and the requirements are around network, servers, end-user devices and IT support tickets. I am certain that it’s valuable to test business cases before the project is even formally assembled.

• Getting in early: https://jlottosen.wordpress.com/2017/05/09/testing_early/ 
• Testing end user device roll-out: https://club.ministryoftesting.com/t/testing-end-user-device-roll-out/799
• Who tests for the IT support https://huddle.eurostarsoftwaretesting.com/forums/topic/who-tests-for-the-it-support/ 

 

*: implicit testing in the form of critical evaluation always happen. .. similarly does checking.

Co-creating smarter testers

“Co-creating smarter testers” is the byline of the Ministry of Testing, a small company with a great impact that I have been following and supporting for 7 years* now . I have attended TestBash’es, webinars, challenges, discussions and memes. And now for the first time in Denmark – Anders Dinsen and I are bringing the world known Meetups to Copenhagen (Aarhus 2017 you’re next).

Ministry of Testing – Copenhagen

Copenhagen, DK
101 Members

The Ministry of Testing exists to advance the software testing industry in a fun, safe, professional and forward thinking way.Our meetups exist as a way to bring people toget…

Next Meetup

Testing roles are shifting, but where to?

Wednesday, Mar 8, 2017, 4:30 PM
22 Attending

Check out this Meetup Group →

The topics so far are:

• Let’s make a Risk Based Testing Strategy recently had 20 participants, hosted and chaired by Anders.
• Testing roles are shifting, but where to? is planned by me (see shiftx) and hosted by a sponsor for food and location (Miracle!)
• Playful Software Testing… needs both a date and a venue.

At the first meetup we split into three groups, discussed risks and how to TEST THEM RISKS. Dearest to me was the discussion of stakeholders and new places to test. Great to see that even with very little information, we can still do a rapid testing based on business objectives. There is so much more to testing these days.

1: since a EuroStar 2010 t-shirt competition 🙂

QA Aarhus – Exploratory Testing How and When

QA Network Aarhus is a local non-affiliated network of testers (and good friends) in Aarhus. Where I had the great pleasure of talking about Exploratory Testing. This is the link collection, the slides are attached.

Testing for coffee, testing for ǝnןɐʌ

We focus on both testing and the office coffee a lot and may be looking at its real value too little. If testing does not provide direct business value – as the office coffee does – how can we elaborate and talk about it as a value to the business?

Time: October 1st 2013  Cafe Stiften Banegårdspladsen 11, 8000 Aarhus, Denmark

If you attend #GOTO aarhus 2013 this will be a good time to continue discussing “ǝnןɐʌ: Why we have it backwards”http://gotocon.com/aarhus-2013/presentation/ǝnןɐʌ:%20Why%20we%20have%20it%20backwards

The first 25 participants gets

• a cool FREE Ninja sticker from http://www.ministryoftesting.com
• a FREE GOTO; Aarhus sticker http://www.gotocon.com/aarhus-2013/

A tester is

via http://blog.softwaretestingclub.com/2012/02/a-tester-is/
See also: Build-a-tester Workshop

All oracles are failable

[Software Testing Club Blog | Oct 6 2011 | myself ]

All oracles are fail-able to a certain level of confidence.

Recently I had the opportunity to participate in Rapid Software Testing (game master Michael Bolton) and the acclaimed dicegame. I also had the chance to be game master on a variation of the dicegame session for a small test team. Reflecting on the experiences I had two considerations: (Some spoilers apply)

When are you confident enough?

The dicegame is played by a loop of theories/ideas and tries/tests on the idea. The goal is to produce a theory/algorithm that can succesfully predict the number that the game master presents. How many tries/tests/checks would give you confidence in the theory you have in mind? Options:

• When you succesfully predict one throw. Ie you say 7, game master say 7. Do you yell “LOOSERS, Sea ya”?
• When you have 7 succesfully predicted in a row? (why 7)
• All 7776 combitations of 5 pieces of 6-sided one-colored standard dices?
• Do you try every throw just once, or more times?
• Would you know if every trial number divisible by 100, the game master would say “pi” (think leap years)

 

The dice game seems simple, but the problem domain of even the dice game is infine. Or at least practically infinite (7776 is practically infinite in the dice game IMHO). James Bach replied to my tweet that “The number of tests doesn’t matter, but the character of the test, relative to your mental models, matters a great deal.“. My purpose is not to find a fixed number of tries, but to make you consider the underlying assumption on confidence levels. That is you have a confidence in your model until it fails. You are confident to the level of x succesfull predictions, where the x+1 prediction fails. All you know at that time is that your theory is “incomplete” (not wrong, not right) – and this calls for more learning and more ideas…

 

All oracles are failable

The oracle in software projects is the ressource of answers – the documents, the mindmap, the subject matter expert. In the dice game the game master is the oracle (ie Michael Bolton, James Bach or yours humbly). We are humans hence failable. The physical oracles (docs, …) even more. This made me ponder:

• Would you approach the dice game differently actively knowning that the dice game master is failable?
• If the game master (aka oracle) made a deliberate error ever once in a while – would you know?
• If there is a bug (non deliberate) in the game masters algorithm would you know?
• How would you test for the oracle making mistakes?
• Do you test the dice game different if it was a human or machine oracle
  • First think about the dice game being computer based
  • Secondly what if it was a human behind a computer based interface
  • Consider the implications of the Turing Test
• Oh, did you forget that I could make mistakes? – was that a rule, or an assumption?

 

Michael Bolton replied to my tweet that “In the dice game, we need to be prepared to deal with any test that a tester offers. Is it a bug? That depends on the framing.” The key framing of the dice game is usually a lesson in learning, in theory setting and trials of that theory – still under the underlying assumption that the game master can deal with any test that is offered. What would happen if the game master was blindfolded? What would the case if the algorithm was more complex – less humanly processable in a short time. There will always be a level of capability of the oracle, and it will fail – eventually.

 

20 Sided Fuzzy Dice Danglers

Testing a new version

[The Software Testing Club blog | November 29, 2011 | Jesper Lindholt Ottosen ]

Enterprise applications often come in packages and are purchased as (Commercial Off the Shelf – COTS). Every now and then a new version of SharePoint,SAP, Jive, OeBS, Microsoft Windows…. is made available and the business and product owner decides to implement the upgrade.

Usually the setting is that there is a “Factory Acceptance Test” by the vendor of the COTS package and a “Site Acceptance Test” by the implementing IT service organization. Here are some ideas that come to my mind, the couple of times I have had look into a testing strategy for a Enterprise COTS upgrade project. It’s not a best practice – at best it’s a heuristic:-)

 

Regression testing first – it might be considered to examine that quality didn’t get worse. Select some of the key existing features that is most important to the product owner, and examine them. Also involving the super users or application advocates in an exploratory testing activity will provide benefits for both the testers, the super users and the other project participants.

Interfaces – in an enterprise environment there is always interfaces to legacy systems and new “bud shots” to the IT tree. SOA services makes it even more important to look for known and unknown interfaces to the application. Similarly context specific customization’s (additions and removals) and applied “production hot fixes” having been applied or constructed based on v2.5. Analyzing the intermediate versions (above example 2.3, 2.4 and finally 2.5) including known fixes and known new features, can be another approach to identify the required levels of testing. Discuss with the product manager and business representative – the key is to find a level of test that they are OK with.