Lessons Learned in Cloud Testing

For reasons of regulations and data restrictions, we were recently doing a cloud project for a large customer of ours. Cloud resources had to be spun up, granted access to, utilized, and taken down (CRUD-like). But we had to do it without granting any of the customer’s employers access to the cloud controls. They couldn’t even have a cloud admin. So for various organizational reasons, we designed the solution below.

The user would order cloud activities in a “shop” and the requests sent first to the customer ticketing system, then to our ticketing system. And finally to the cloud pipelines and scripts. And there my friend is where the testing started.

Testing Input Sanity

The first script was for initializing the cloud services: Ressource Name. First of I found out the (cloud) developers had already tested the happy paths. I loaded Bugmagnet and gave some funny names, as the requirements for that field hadn’t really been that specific.

https://bugmagnet.org/

I quickly learned that the cloud provider was already sanitizing the input, so no Little Bobby Tables were allowed. It made the customer think, though, and she added a rule for the shop with regard to the Ressource names. This was coded as a regular expression (Start with a capital letter, then some letters, perhaps some numbers) – not that we or the cloud cared for that strict naming scheme, it was more of a business usage rule. Was that a bug? No mostly something that testing finds – not a bug.

I Found A Bug- Can I Go Home Now?

The next request could assign privileges to the cloud resources. Based on the resource name and Access Level, the cloud resources would be enabled. Among the access level options was “admin” rights. Which was directly opposite to the whole purpose of the project, and the customer risked breaching the regulatory inertia/controls.

And that was really the only formal bug we logged. And I don’t go home – I quickly reached out to the team, had the Accces level parameter removed and all was good. The intention was all along that only one contributor access level should be granted. That level was then hardcoded in the code repository.

And then we tested further around creating things already created, revoking resources that were not there, and race conditions for revoking the services. The only thing that really gave us a challenge was when a request failed (say you request something already there). Having the two ticking systems pick up the response and send the error message back to the users was more trick than expected as they were more REQUEST focussed – but we managed. Without creating a bug the team worked together and resolved the issues.

Key Learnings

As always this is a story retold from actual experience, there are details I can’t share and elements I preferred to have done differently – but due to the organizational culture had its own way.

• A mindset toward input sanity and access control is key to supporting business goals
• Bugs, issues, and observations will be found they are learnings for the team, not problems
• Hold end-2-end rehearsals of the solution and play out the scenarios
• Don’t restrict the testing activities to software development, look at the whole solution

How Do you Eat a Mastodon?

… one piece of the time. Really. Don’t worry about it – it’s similar to eating an elephant.

While Discord and Slack sites can be places to go for communities with the faill of Twitter – they are closed systems. You need an invite and communication closed within the instance. This is good for closed discussions you want to gatekeep. Mastodon seems to be the place for serendipity and cross-collaboration across topics and communities. It’s not new, the framework started in 2016 gaining 1 million users in 2017. There are a number of great and extensive guides – one of them is fedi.tips. Another great guide is Mastodon Lessons Learned by Lisi Hocke (@lisihocke@mastodon.social).

Hi I’m Jesper🧷🌻🐐📕🗺🇩🇰 (@Jo2sn@mastodon.social)

Signup and Servers

For Mastodon, you create accounts on a server (or instance) in the Mastodon-based network. Through a shared protocol the people on the servers can communicate with posts, threads, likes, and resharing as known from other platforms. Mastodon is really the software framework (on GitHub) and the network called “Fediverse” – but people mostly call it “on Mastodon”.

Someone wrote that picking a Mastodon Server feels like selecting a character in a new video game. You don’t really know what you’re looking into and how you will play the game. Luckily you are not fixed to one account on the Mastodon network. You can have multiple and easily redirect from the old to the newer. I have tried jumping profiles myself, as the first one I made was on a server with too few general topics. One of the elements of Mastodon is that each server has a timeline – besides the ability to communicate across.  

Some servers are topic-based on Open-source software, others on art and LGBT, you can pick a specific server and expect the local content mostly on that topic. Or you can pick a general-purpose server – if one topic doesn’t define you. There are some tools, like debirdify that can help you in the process – but not yet a 1-click tool to move over. 

Two ways to select a server:

• Browse the directory at https://joinmastodon.org/servers 
• Use https://pruvisto.org/debirdify/ to see where your Twitter people have moved

People can and will move to many places – there is yet to be a primary test community instance. 

Profile and Settings

• Your username is text-based, but you can go crazy with emojis in the profile name field
• Create your profile with a picture, banner, and profile text
• Profile meta-data and hashtags will help people find you and recognize you
• Add 2-factor authentication and save the password and keys
• If English is not your native language, there are other languages available (German is rather prominent). Selecting a language seems to give you more content in that language.
• Enabling the “slow mode” setting is recommended. It forces you manually to reload the feeds and not have the dreaded firehose. 
• Make an #introduction post and write your favorite hashtags, that will help people find you.
• Write your Mastodon username @user@server.name in your Twitter bio so that others can follow you

General Use

As there is no algorithm to drive engagement, so use hashtags and reshare content from others. Likes are only posted to the author – so reshare/retoot great posts. Similarly to following people you can follow hashtags like #MinistryOfTesting and #TwitterTestingPeople. The culture of Mastodon is to be inclusive. Use Content-Warnings and hide images as a common rule – and ALT-text on images as a general rule. 

Mastodon has free apps (Android, iOS) for casual use and some pay-to-use apps. Currently, it seems you can’t follow hashtags in the iOS app, so I tend to use the browser more. There is a range of choices – see what bests fit you. You can get a long way with just the primary site.

Servers are run by small teams of admins, so in peak periods there might be suboptimal performance. Most servers seem to be funded by patreons or similar fundraising. 

In theory, an instance could close down and take everything away. If you have content that is more permanent and you want to own, put it somewhere you control. The same challenge is true for ad-based social media (Facebook) and subscription-based platforms like Medium. 

If you feel like organizing the content coming your way, enabling the advanced web GUI setting will give you columns like “TweetDeck” – but the setting is global for your login. Sometimes I just want a quick read over the timelines, other times I want to monitor specific topics. To monitor topics I have set up columns in Sengi https://sengi.nicolas-constant.com/. It’s a per-browser setting, so it’s more of a topic radar that can be different across my devices. Personal lists are a feature too, but it seems based on people not so much on topics. 

Do notice that direct messages aren’t private messages. All the people you tag can read the messages. So don’t tag the people you want to rant about. 

The visibility of a toot/post follows the flowchart below. 

Via @geekyMary@Mastodon.social: https://mastodon.social/@Geekymary/109290888512800541

Writing a Book in 30 Evenings

As mentioned I have written a book. Looking through my notes I have around 30 versions, one for each significant session of working on it. I have been asked to share tips on writing a book, so here you go, Simon. The key lessons are:

1. Set a Recurring Time and Space
2. Unique Content – Reference the Rest
3. Storyline and Flow
4. The opposite of passive voice is not an aggressive voice
5. LeanPub’bing

Set a Recurring Time and Space

One of my book’s themes was moving something from a hunch to a hard truth. And the same really applies here. When I finished my master’s degree while working (back in 2002), I had two slots a week at a “study office”. From that, I learned that not all study days are equally productive “on paper” – but that’s ok. Each session had its purpose.

Similarly for my book project. I set a weekly evening booking in my Calendar – family chores were arranged around it. After dinner, I work start working on the book and work for 2-2½ hours. I used my personal computer in my work-from-home setup.

Unique Content – Reference the Rest

There is so much great content out there already, my focus was on writing about my experience and my vision of better test strategies. But to do that I needed to stand on the shoulders of others to set the scene and describe the techniques I applied.

While writing I did want to bring in loads of existing content to elaborate and provide a foundation for my thinking. While editing I removed most of it, partly because I didn’t want to sound like a high school book report (thank you for that one Tristan). What I did leave in were quotes, recommendations, and listings of the work of others. The book is full of footnotes directly on the page (as compared to end notes) to highlight everyone in context.

Storyline and Flow

Initially, I outlined the chapters and subchapters and it was important to get the right “flow” and storyline into the content. My base model was inspired by “Situation, Complication, Question, Answer” from the guide “How to present to executives” (StaffEng Book) and similar lessons on taking the first steps.

• This is the situation and challenges
• These are the techniques, we can build on
• These are the first steps, where the techniques are used

One thing I have worked a lot on is the flow of the text: Sections would be 4-6 lines long, with empty lines in between. I also worked to reduce “dangling lines”, so no two lines would linger into the next page. No pages should be text only, so quotes and illustrations are important for readability – as well as making the book content visible. Lastly, I worked a lot on having one section end with words that tap into the next section.

The below recommendation is cool, I could have done that too.

15 years ago my PhD advisor taught me One Weird Trick for editing your own writing. Edit **back to front**, paragraph by paragraph. I still use it and it still surprises me how well it works. When I get my students to do it, it often blows their minds. Try it!

— Arvind Narayanan (@random_walker) October 16, 2022

The opposite of passive voice is not an aggressive voice

Often I started my “book evening” by reading the book from the start all over again. As I’m not a native English writer, I installed Grammarly and paid for the premium version for a period to rewrite phrases that were in a passive form. Fun Fact: the opposite of passive voice, is not an aggressive voice but an active voice.

Have proofread quite a few books for my friends in space. Couple of Stage Mom top tips:
– Avoid colloquial language for global readers
– Know your audience. If for beginners, use footnotes or end notes
– Don’t overuse passive voice
– Tell stories from your experience! pic.twitter.com/sehtnM5bRx

— Tristan Lombard 🇺🇦 (@TristanLombard2) October 2, 2022

The tool you use to write your book isn’t that important. I used Google Docs with embedded Google Drawings, others might prefer Word or other editing platforms. The biggest challenge in publishing was getting the embedded pictures in a resolution that could be printed by the print shop. The book exists in a printed format – but it’s primarily an electronic book.

LeanPub’bing

Self-publishing on LeanPub is intuitive and easy, it also has tools for incremental versions, previews, and updates. Let people sign up for it in advance with price suggestions. That allows you to set a price on the book based on audience-based quotes.

They also have a great system for coupons. I have coupons for those that helped in preparing the book, those I have referenced, and those that participated in the virtual launch. Let me know with the phrase “LeanPubbing” if you want a coupon too.

#263: There is a Model for your Trouble

Often directors, managers and other decision makers talks about an advanced challenge they have: What can we automate, who should automate and what tools to pick. There more and more I listen, the more I hear – they have not applied any models of the problem at hand. And there usually is a model of the problem space already. Any old model is preferable over no model at all. But it can be hard to see in the middle of chaos.

Calculating Time To Information

The key metric for any knowledge work – IT deliveries and testing in particular – is more than Mean Time to Repair (MTTR). While fixing fast matters – timing is everything. Timing in getting information to the people who needs it to make decisions. It’s no use if you can turn the ship around on a plate now, if you needed it yesterday. Key elements in calculating time to information is how far away the information is and how evolved the information is.

Something About Leadership

17yo: Dad, do you not know how old I am - and what I can do myself? 
Me: Oh, I know buddy. As you are learning new stuff, I am unlearning to help you

While this quote from my kitchen about a week ago, as all to do with the young man learning the ropes of life and me unlearning to always to help them and their 15yo sibling out – there is an key parallel to leadership and building self reliance in teams. My role these days are less about direction and (micro)managing a team of testers on a project, more about enabling others to succeed with their testing both in the delivery teams and in the board room.

What’s the Problem Here?

At the the recent “Online Test Conference – Fall 2020” a workshop reminded me to look at the problem. And it’s always a people problem. Which got me thinking later on in the program – what’s this talk trying to solve?

Stuck On a Deserted Island

.. with only 5 songs and a book. Which ones and how they tie into the topics of leadership, biases and figuring out your learning pathway. Listen here:

Someone else will do it

The testing activity has been under change for long. And it’s clear that the testing activity has shifted. Even the test managers have to re-calibrate – as other roles will be doing the test management activity. Be prepared, as someone else will do your testing job. Work on building self-reliance in others and be prepared to hand-over what you can do.

There is more to testing than testing specialists punching test cases. The testing activity as such, has shifted (both left and right), and testing is being done by more roles than “testing people”. Depending on the context, the explicit testing activity is done by a mix of developers, testing specialists, end users and others.

I often find myself as the only testing person on the project. The testing activity is done by automation specialists and end users in one project, and by technical operations staff and end users in another. In these projects either the technology or the business knowledge is paramount, and not so much exploration, flaws and edge cases for specialized testers to explore.

me, 2020. YMMV

Similarly for the test managers – there’s a trend/shift, that sometimes the test management activity is shifting away from the test managers. Even to me – even if I’m sometimes more an a “project manager of the testing activity“, a “Test coach” or similar. The trend is already there – coined sometimes as “whole team approach to quality“. Yes, most of the test management activity can be done by scrum masters, Release Train Engineers and even project managers ….

Recently I was asked to assist a large transition project for a holding company with many brands. Each brand had their own applications and technology stack, but the holding company had decided to move the hosting. So the holding company’s Project Management Office (PMO) was put in charge of facilitating the brand’s testing activities – an activity they had never considered nor done before. My role would only be to provide guidance, not do the actual facilitation.

Which got me thinking….

What if the testers, are not testers
– but other "titles" doing testing.
And what if the facilitation of this activity is done by
– others than test managers

then – then what do YOU do. pic.twitter.com/471AcRiFru

— Jesper (@jlottosen) May 15, 2020

And after some deep thinking. – I do have the privilege to be able to adapt. I don’t need to hoard knowledge or make power moves (anymore) or worry about health-coverage or any of the lower Maslow pyramid terms (anymore).

It’s very natural for me to hand over project approaches to my co-workers. I’m often on the “blue team” to outline the strategy, My best field of work is to bring clarity and consistency, not scalability or repeatability to the practise.

I naturally hand over learning anyways, so why not re-calibrate when the thing I do has reached a stage, where it’s repeatable. And then focus on building the skills in others, work myself out of the test management role as we know it.

And don’t worry that someone else will eventually do my (testing and test management) job. The first step is to acknowledge the trend/pattern, second to redefine and bring clarity! Let’s explore and see what we find!

Someone else will do the building, not Emmet. Their task is repeatable.

Your Learning is on You

You, yourself, is responsible for getting the training, learning and knowledge you need. Don’t wait for your boss – be proactive, it drives your success. Here are some places to start:

Meetup’s are happening online now, which removes one primary barrier to attending great talks. Similarly conferences go online, some with a fee, some for free – some even in multiple time zones. Lastly online training sites are abundant with relevant information for the challenges you have. Yes – also for you!

Stop moaning about getting test automation and accessibility to be a part of Definition of Done, or how to build a whole team approach to quality. It’s already out there – reach out.

Just this week, April 2020, I’m attending:

• Smart Bear Connect 2020, online / live
• Lean Agile Scotland 2019 recording
• Test Bash @ home 24 hour online / live

With plenty of talks about risk based testing, test management in the light of automated deliveries, BDD etc. With live slack groups the experience is almost as the physical conferences :). Next up in may is the Online Test Conf, Spring 2020 with topics for everyone in convenient global time slots.

When your boss says there’s no budget for attending conferences in person this year (again!), there are other ways to attend – physically. You could try to submit a talk and get accepted, but the barrier is quite high. A great way is simply to reach out and volunteer to help the program committee. If you can time it, with regards to the budget year, ask you boss based on the conference program aligned with your company strategy. At least what the boss should do is to allow it to be company time – else take the time off. …

If you are hungry to learn

What I see in the global testing community is that Scandinavians are complacently waiting for the company to pay time, money and effort to their learning, while people in emergent economies (Hi Sfax and Argentina) are eager to learn and on the forefront of the trends of the trade. They are driving the change of a positive inclusive community.

Time to information is the key factor – not only in digital transformation, not only in IT deliveries and but for the organisation as a whole.

And for you!

if you still work in silos, your success – will be less

Mike Lyles, Smart Bear connect 2020