Two contributions to the uTest University

Back in December of 2013, uTest officially launched the uTest University (blog post) which is intended to be a single source for testers of all experience levels to access free training resources. This is a neat opportunity for testers to contribute to the growth and development of the testing community by creating courses and writing articles. The university also offers  Author Page

My first course was derived from a uTest forum’s post I I wrote back in June of 2013. I was on a cycle where the customer required logs from Charles Web Debugging Proxy be attached to every bug report, but none of the testers (myself included) or the knew what that was or how to use it. I spent some time learning how to use the tool and then put together a tutorial to share with the rest of the team. Fast forward 8 months later several other customers started required the same thing. To make the information a bit easier to find the tutorial was turned into a uTu (uTest university) course:
How to Set Up Charles Web Debugging Proxy for iOS Devices and Windows 8

My second course came at the request of the uTest Community Management team. They needed a tutorial for new testers to show them how to create videos (screencasts) of their bugs. They specifically wanted it based around the free tool Screencast-O-Matic. I had actually never used that tool before, so I spent some time getting familiar with the tool. I also compiled a list of suggestions and tips based on things I see frequently in the videos of other testers. The result is:
How to Set Up and Use Screencast-O-Matic

 

A Tester’s Haiku

From time to time uTest puts on fun contests for the community. This month the objective was to come up with the best testing-related haiku. Over 90 testers participated, coming up with some great poems ranging from serious to silly to clever.

There was one haiku that got more attention than most so I felt it warranted some further discussion. It was one of the first submissions and it was well-written so the uTest CM team used it to promote the contest. It was mentioned it on the forums and posted it on Twitter. It was nominated as one of the 10 finalists and went on to win the contest, earning the vote of 21 other testers. Here is the poem in question:

Gifted are testers
Fixing a world of errors
Coded by others

It follows haiku syllable structure and has a nice flow, but I feel it’s misleading and potentially damaging. I’ll quickly address each line.

Gifted are testers
Can’t argue with that :)

Fixing a world of errors
This shouldn’t be a  great revelation, but in general, testers do not fix errors; We provide information and dispel misconceptions. One of the way’s we do this is by identifying errors. Some might try to make an argument that part of resolving a problem is finding it, but testers who understand what we do to provide value, realize the importance of making the distinction between finding an fixing a bug. These are two different actions that required two different skillsets.

Coded by others
This line really bothers me because it promotes an us vs. them mentality. Even though someone else (the developer) did write the code, it doesn’t help to point the finger at them. It’s a testers job to identify mistakes in other people’s work and that puts us in a delicate situation. It needs to be handled without judgment or condemnation. Bugs are an inevitable part of software development and since quality is the responsibility of the entire team, we need to work together, take responsibility together and avoid blaming each other.

Now, I’m not trying to say the author of this haiku meant to say that testers literally write the code to fix bugs or that developers and testers are enemies and I don’t mean to pick on him/her, but the fact that so many other testers thought this haiku was a good representation of testing troubled me. I’m sure this can be partially attributed to uTest’s endorsement and also because of language issues, but these are two themes that I see often and wanted to briefly address them.

 

Decision Review: Pursue my MBA

My first Key Decision was to Pursue my MBA. Let’s see how that decision worked out shall we?

After only one semester it became quite clear that pursing my MBA was a poor decision. Hmm… so far I’m 0 for 1. You might think I quit before I even gave it a chance, but I struggled just to finish out the semester. After a lot of reflection on what exactly went wrong, I was able to narrow it down to three things. The impact on my family, the educational aspect was disappointing, and the opportunity costs were too high.

Time with family

Going in, I knew that class time would reduce the amount of time with my family, but that didn’t fully sink in until it became real. A 3 hour class, sandwiched between a 45 minute commute meant that two days a week, I wouldn’t get to see my wife or kids. How could I possibly be so cruel as to deny my family the pleasure of my company? My absence did put a lot of additional burden on my wife and the kids missed wrestling with me, but mostly I’m just selfish; I missed them too much.

Disappointing

After my first class I was super excited. The instructor was fun and engaging. He asked open ended questions and joined myself and others in debate and discussion. Unfortunately that didn’t last long. As we approached our first exam it became clear that grad school is merely a continuation of undergrad. You’re still graded and evaluated on what you know, not on how you think, reason, or your ability to learn and execute. Multiple-choice Scantron tests…SERIOUSLY!? The entire dynamic of the class changed. Boring lectures attempting to “teach” the “right” answers to unimportant questions.

I was really looking forward to learning from my fellow students, from their experiences in industries and business areas new to me. Sadly, most only cared about their grade. “Is that going to be on the test” was the most common question asked. The lack of interest in true self-improvement and overall “quality” of the students admitted to the program was disheartening.

To be fair, I only took one class at one university, but as UNCG is a highly ranked university, I have to suspect that my experience isn’t all that unique.

Opportunity costs

Similar to how my time focusing on school reduced my family time, it also took the place of other career-related opportunities, specifically uTest. The additional 20 hours a week meant that I had to completely abandon uTest. My good friend Rex helped me realize what an expensive trade-off that really was. I had spent two years building my reputation as a tester and done so quite successfully. I was invited to best projects, I was able to participate in various company initiatives (like the TTL training and evaluation program) and I had a large, visible presence in the community. I was sacrificing a opportunity that provided me continuous growth opportunities, respect, and enjoyment for the chance to become one of the select 100,000 MBAs that graduate each year.

What I learned

Structured, standardized learning no longer appeals to me. I learn more from conversations over a few beers than I do listening to a lecture. The ability to execute is more important than the ability to memorize business trivia. Doing something just because it’s uncomfortable is not a good reason. I thought I was being courageous by stepping out of my comfort zone, but know I see I was just being an idiot.

In the end I realized that the expectations I listed in my decision post can be fulfilled by continuing to develop my testing career. I was already working towards all of them and making great progress. At this point in my life and career, the benefits of an MBA program didn’t outweigh the costs. Maybe someday I’ll regret not having those three letters after my name, but today is not that day.

Key Decision: Pursue my MBA

Decision

The key decision I made is to pursue my MBA degree. I’ve been wrestling with this decision for several years now. I’ve always enjoyed learning and studying but it just seems like there are so many reasons not to do this. I have two children and a wonderful wife that I want to spend time with, I just started a new job, I coach hockey, I’m heavily involved with uTest, etc.

There are also philosophical reasons pulling me away from the MBA. Is an MBA really the best way for me to learn? I’m mostly interested in testing so why not focus on testing courses from AST or personal mentoring and training from some of testings leaders. I should also quote Will Hunting :)

…you dropped a hundred and fifty grand on a f@#$% education you coulda’ got for a dollar fifty in late charges at the Public Library.

So why did I make this key decision? Well… because it wasn’t an easy, comfortable decision. Whenever I find an uncomfortable situation, I try to force myself through it. I don’t know if this is the right decision but I know I’m a bit scared so I know I will learn plenty whether my MBA related goals are met or not. Also, the financial risk is tolerable at UNCG.

Expectations

  • Develop a more complete understanding of business – I know plenty about IT and testing, but other aspects such as finance and marketing are mysteries to me
  • Gain experience working with people from different business areas and different cultural backgrounds
  • Influence the education and growth of others
  • Learn new ideas and new ways of thinking
  • More challenging and rewarding career options

Strength and weakness exercise

“What are your greatest strengths?” “What are your greatest weaknesses?” Two of the most common and dreaded interview questions. Most people think they know the answers to these questions, but I’m pretty sure most people are wrong; especially when it comes to their strengths. The things I’m not good at are fairly obvious, but discovering what I’m truly good has proven to be more difficult. I think this is because what I think are my strengths are what I want to be my strengths.

I’ve found that unbiased self-reflection is a difficult thing to do well, so in attempt to better identify my strengths and weaknesses, I’m going to try to document key decisions I make and what I want and expect to happen as a result of those decision. A year or so later the plan is to revisit these decisions and see if I was able to achieve the desired outcome.

I unknowingly started this exercise in March of 2012 when I decided to join uTest. I documented the reasons for making that decision as well as my goals and now, 18 months later what I expected to happen – develop a reputation as an expert tester – is starting to develop (at least within the uTest community).

I’m excited to see where this exercise takes me. To kick it off, my first key decision is to…. start keeping track of my key decisions :)

My First Testing Experience – Part 2

Continued from Part 1

I can vividly remember lying on my couch in my living room staring at the ceiling, my stomach in knots. I felt this huge weight on top of me making it hard to breath. I was pretty close to panicking. “How can I possibly do this? I have no idea what I’m doing. Why did I agree to this project? Do they have any idea what they’re asking? This is so unfair. This is CRAZY!” These were the thoughts racing through my mind. Before I could think one through, another would jump on the pile. That feeling of hopelessness, of being completely overwhelmed was my first and worst moment as a software tester.

I’d like to think the PM understood the magnitude of the project, that somehow she saw this amazing potential inside me just waiting for an opportunity to prove itself. But I knew she didn’t really understand what she was asking. In her mind this was a small task that any mid-level developer should be able to do. After all, it’s just testing, how hard can that be? That just added to the pressure. I was dealing with unrealistic expectations about an undefined project that required an under-appreciated amount of skill and effort.

After breathing into a paper bag for 10 minutes or so, I calmed down. I reminded myself the best way to climb a mountain is one step at a time. I started by making a spreadsheet documenting all the different migration tools we had built. I identified the base data type, the destination data type and a description of the transformation needed. One by one I went down the list, talking to the developers, searching for any documentation, looking at the old system, learning about the new one, basically doing the work a BA would have done.

Once I had a solid understanding of the specifications I was working with, I started testing. I started with the simplest cases and the ones that I was most familiar with. You are probably thinking I should have done the opposite and started with the hardest, most complicated and risky transformations. If I was doing it over today, that is the approach I would take, but considering I was a complete rookie tester, I needed to start with a small, achievable task to build some confidence and establish my testing strategy.

My strategy was pretty simple. I gathered sample source data, ran it though the migration and verified the results on the other side. Then I started adjusting the source data, trying every scenario I could think of looking for ways to make the tools fail. As I worked through my list, reporting bug after bug, it became clearer to everyone just how large this task was. Expectations slowly became more aligned with reality and the measurement for success became more reasonable.

After a few months of work, we were ready to start migrating content. It wasn’t perfect and several bugs made it through, but they were minor and due to unexpected bad source data. The migration ran for months as different sites came online. The migration tools did their job and by all accounts, the testing aspect of the project was successful.

This experience isn’t what pushed me into testing, but it was my first introduction. It gave me a new appreciation for what testing is and how valuable it can be. It gave me confidence in myself that I could accomplish a task that I really had no business attempting. It’s an experience that I can now look back on and be proud of.

Webinar – Should testers report every bug they find?

In December of 2012, Ryan Lamontagne and I got into a good discussion on the uTest forms about whether testers should report every bug they find. We decided to kick it up a notch and debate it live in a webinar!

http://forums.utest.com/viewtopic.php?f=13&t=4430

Webinar – Three more uTest Panel webinars

It’s been a busy past few weeks. In addition to picking up two new enterprise customer accounts (uTest TTL work) I was a panelist for three more uTest webinars.

Maximizing Your Benefit From The uTest Forums
http://forums.utest.com/viewtopic.php?f=55&t=4985

Maximizing Exploratory Testing Methods
http://forums.utest.com/viewtopic.php?f=55&t=4984

How to be a Quiet Tester That Customers Shout About
http://forums.utest.com/viewtopic.php?f=55&t=4986

 

Testing with the Microsoft Surface RT

About the Device

The Microsoft Surface RT is a interesting device to test with. It has the new Windows 8 interface, but it also still has the traditional desktop interface because not all things can be accomplished on the Windows 8 side. This can make things a little confusing, especially if you are just starting to test with this device. So, here are some instructions to help you get ready to set up and back to testing in no time.

Installing Apps

When we are testing apps that are in development, they are not available in the Windows Store so we will need to manually install them. Usually, you will be given a zip file that contains all the files you need.

  1. Download the zip file
  2. Go to your Desktop and locate the zip file (Should be in your Downloads folder)
  3. You can’t install directly from a zip file so you need to extract the files
  4. Long tap/release (or right-click if you have a mouse) to open the context menu
  5. Tap on ‘Extract All…”
  6. Select the location where you want the files saved to and tap ‘Extract’
  7. The extracted folder will open
  8. Locate the Windows Powershell Script (should have a Notepad icon)
  9. Long tap/release (or right-click if you have a mouse) to open the context menu
  10. Tap ‘Run with PowerShell’
  11. You’ll see a series of prompts, accept all of them
  12. If this is the first time you’ve installed an app, you’ll be required to sign up for a development license. It’s free and it has just a few simple steps
  13. After the application is installed, the PowerShell windows will just close. There is no indication that it was installed, but when you go back to your home screen, you should see the app on the far right

Uninstalling apps

  1. On the home screen, drag the app icon down just a bit until a check mark shows up just above the icon
  2. Release the app and the action menu will open from the bottom of the screen
  3. Tap ‘Uninstall’ to remove the app

Error logs

The Surface RT essentially is a PC, so application errors and crashes will be logged in the Event Viewer just as they are on a regular Windows desktop or lap top.

How you access the Event Viewer on the Surface RT is more complicated than on a regular PC since there is no Start button or search function. Here is how you can add a link to the Event Viewer to your desktop:

  1. From the Home screen, tap on ‘Desktop’
  2. Long tap/release (or right-click if you have a mouse) to open the context menu
  3. Select Personalize
  4. In the ‘Search Control Panel’, search for Event Viewer
  5. Tap on Administrative Tools(or Navigate to Control Panel > System And Security > Administrative Tools)
  6. Long tap/release (or right-click if you have a mouse) on Event Viewer to open the context menu
  7. Tap Send To > Desktop (create Shortcut)

Now that you have the event viewer open, you can view error messages and save the error files to attach to your bug reports.

  1. On the left pane, tap on Windows Logs >> Application
  2. On the right pane, tap on ‘Filter Current Log…’
  3. Select ‘Error’ and tap ‘OK’
  4. Select the error log(s) you want and on the right pane, tap ‘Save Selected Events’
  5. Save the events wherever you like and you can attach them to your bug reports

Taking Screenshots

Screenshots are pretty easy to take. You need to press the home button (Windows icon on the front of the Surface) and the down volume button (left side of the Surface) at the same time.
The file will be stored in your Photos > Pictures library > Screenshots folder

Taking Video

A well-created external video is usually the best way to go as I’ve done in all the above videos. Check out this post for some tips to improve your external videos.
I haven’t found a way to create mirrored videos yet, but I’ll update this if I do.

 

Webinar – Finding bugs in mobile devices

I was able to join Kayla Cox and Todd Smith for a uTest webinar to talk about testing mobile devices and how to find high-value bugs.

Since my microphone was terrible (and I might have been mumbling a little ) here is a summary of the points I made in our discussion.

Crashes
Understand that not all crashes are valuable.
Out of memory crash may be due to other apps using up 90% of your memory and the app you are testing just pushed you over the limit. The best way to know for sure, is to have a clean test bed. Restart your phone after you install a new app, and make sure no other apps are running in the background.

When you do get a memory related crash, use a memory management app to help you see where your memory usage spikes. Being able to identify a reproducible memory crash is usually a high-value bug

Connection Issues

  • Kill your connection while data is being transferred
  • Unplug your wi-fi router/modem
  • Turn on airplane mode
  • Turn of wi-fi on your device
  • Turn off cellular data on your device
  • Find places near you that have low or no signal and test there

Interaction with native and popular apps

  • Share something via email with no email set up
  • Log in using Facebook account with/without the Facebook app installed
  • Interrupt testing with phone calls, text messages, FaceTime calls etc
  • If the app changes the phone settings, make sure it does it correctly. Change it back manually in settings and see how the app responds

Investigation and Documentation
There are many topics on how to write good bug reports but there are a few points worth reiterating

  • Provide exact reproduction steps
  • Do root cause analysis – don’t report symptoms. I once saw 3 testers reported 3 different symptoms of the same bug. On the surface they all looked like different bugs, but a little analysis showed they were all caused by the same step they all overlooked.