To go along with my course on Exploratory Testing, I had the opportunity to do a live webinar demonstrating how I do exploratory testing. The community management team did some great marketing and over 300 people signed up! Unfortunately, we found out at the start of the webinar that the uTest GoToMeeting license only supported 100 participants so a lot of folks were not able to attend. It was quite a humbling experience to have so many people interested in watching how I test.
As you can see from the video, the app I chose was quite buggy. That coupled with my poor internet connection presented some interesting challenges. It was difficult to maintain my train of thought and to investigate a single issue when other issues kept popping up, but I think it gave a good representation of what exploratory testing can be like sometimes.
Looking back at the video I need to improve in the following areas:
- Speak slower and clearer – I was trying to articulate my every thought and this lead me to speak in quick, incomplete sentences. It might have been beneficial in showing how choppy my thought process can be, but I think it might also made it difficult to follow what I was thinking.
- Repeat the question – During the Q&A portion, I didn’t do a good job of stating the question I was answering. I read the question and kinda mumbled it before jumping into the answer. I did OK on some questions, but I need to remember to do a better job of this next time.
In December of 2012, Ryan Lamontagne and I got into a good discussion on the uTest forms about whether testers should report every bug they find. We decided to kick it up a notch and debate it live in a webinar!
It’s been a busy past few weeks. In addition to picking up two new enterprise customer accounts (uTest TTL work) I was a panelist for three more uTest webinars.
Maximizing Your Benefit From The uTest Forums
Maximizing Exploratory Testing Methods
How to be a Quiet Tester That Customers Shout About
I was able to join Kayla Cox and Todd Smith for a uTest webinar to talk about testing mobile devices and how to find high-value bugs.
Since my microphone was terrible (and I might have been mumbling a little ) here is a summary of the points I made in our discussion.
Understand that not all crashes are valuable.
Out of memory crash may be due to other apps using up 90% of your memory and the app you are testing just pushed you over the limit. The best way to know for sure, is to have a clean test bed. Restart your phone after you install a new app, and make sure no other apps are running in the background.
When you do get a memory related crash, use a memory management app to help you see where your memory usage spikes. Being able to identify a reproducible memory crash is usually a high-value bug
- Kill your connection while data is being transferred
- Unplug your wi-fi router/modem
- Turn on airplane mode
- Turn of wi-fi on your device
- Turn off cellular data on your device
- Find places near you that have low or no signal and test there
Interaction with native and popular apps
- Share something via email with no email set up
- Log in using Facebook account with/without the Facebook app installed
- Interrupt testing with phone calls, text messages, FaceTime calls etc
- If the app changes the phone settings, make sure it does it correctly. Change it back manually in settings and see how the app responds
Investigation and Documentation
There are many topics on how to write good bug reports but there are a few points worth reiterating
- Provide exact reproduction steps
- Do root cause analysis – don’t report symptoms. I once saw 3 testers reported 3 different symptoms of the same bug. On the surface they all looked like different bugs, but a little analysis showed they were all caused by the same step they all overlooked.