(aka not every testing problem can be solved by a webdriver)
Web features doesn’t matter as much in the contexts I usually work in. While some may be delivered over the web, the focus for testing is on the whole system’s fit for the business. Adding automation in testing to the mix gives additional challenges as there is no source code in the solution to interact with, and we have to find other solutions to solve the tedious tasks in testing with.
One area where this is the case, is when implementing standard commercial software packages (COTS) for the enterprise or public sector. Solutions like SAP for retail CRM and ERP, Microsoft Dynamics 365 for Finance and Operations, EPIC for hospitals, Service Now for IT Service management etc. These are standard solutions that can be configured and customized, but the general source code is not available.
Thus the “test automation pyramid” falls short to help us automate things as only the GUI is available for interacting with the solution. Test engineers might want to setup CI/CD but the success of that depends on the system architecture and the provided as-is methods of releasing updates. Some of the solutions above are provided as SaaS but quite often they still run “on premise” and the business still wants releases tested before launching things on a corporate scale.
Another example is the many bespoke software solutions that are still in operation. My local electronics store has two interfaces for the sales persons: web to look things up (specs and availability) and a mainframe system to set up the actual purchase (Point of sales, POC). Many public organisations and enterprises are only now transitioning from the desktop applications of the 1990’es to more up to date solutions. Unfortunately systems that are 10+ years old have very little of live and relevant specifications and neither CI/CD suites.
While some COTS and POC solutions are being delivered over the web, testing web particularities the very least of our focus areas. The web particularities seems to matter more if the solutions are business-to-consumer but not so much when it’s business-to-employer or business-to-business.
In a business-to-employer and business-to-business context, usually only one browser is in scope. And that’s it. There is little interest for HTTP status codes, broken links, browser compatibility or login forms.
Focusing only on testing the web in contexts like these we fail in
- covering the whole system landscape across applications of different technologies
- addressing the real questions of the business subject matter experts and the business
It’s in this context that RPA probably has some benefits in providing automation of tedious testing tasks to the tester with a business background. That is, they are business people first – and then they do the testing that matters to the context.
7 thoughts on “When testing web features doesn’t matter”
[…] When testing web features doesn’t matter – Jesper Ottosen – https://jlottosen.wordpress.com/2019/02/24/when-testing-web-features-doesnt-matter/ […]
[…] depends on your context if modern testing is a thing for you, it might work in an enterprise setting of commercial standard systems. Using Robot Desktop automation in testing might work better in that setting, but then again it […]
[…] I have performed an analysis that shows that using RDA for test automation has similar costs and speed as with using Selenium for test automation … but then not all projects are web projects. […]
[…] RDA is interesting in the context of test automation , as they can automate GUI interactions – also on top of enterprise package applications (SaaS, COTS, OOTB etc. ). The test automation challenge for most of these enterprise applications (SAP, MS Dynamics  etc.) is that they come with no access to the code-base, even if these are pure-play web based – the GUI is all there is. […]
[…] The authors have “multiple examples of applying these practices within mainframe environments, traditional packaged software application delivery teams, and product teams“. It’s not just for business-to-consumer web-based organizations. […]
[…] at a small municipality too, they have very little formal IT staff, yet heavy rely on IT systems. Public facing websites are just a part of the IT landscape for most […]
[…] all software projects are about consumer facing native apps and websites. While they are numerous there’s still plenty of systems out there for internal and […]