uTest University Webinar – Live Exploratory Testing

To go along with my course on Exploratory Testing, I had the opportunity to do a live webinar demonstrating how I do exploratory testing. The community management team did some great marketing and over 300 people signed up! Unfortunately, we found out at the start of the webinar that the uTest GoToMeeting license only supported 100 participants so a lot of folks were not able to attend. It was quite a humbling experience to have so many people interested in watching how I test.

As you can see from the video, the app I chose was quite buggy. That coupled with my poor internet connection presented some interesting challenges. It was difficult to maintain my train of thought and to investigate a single issue when other issues kept popping up, but I think it gave a good representation of what exploratory testing can be like sometimes.

Looking back at the video I need to improve in the following areas:

  • Speak slower and clearer – I was trying to articulate my every thought and this lead me to speak in quick, incomplete sentences. It might have been beneficial in showing how choppy my thought process can be, but I think it might also made it difficult to follow what I was thinking.
  • Repeat the question – During the Q&A portion, I didn’t do a good job of stating the question I was answering. I read the question and kinda mumbled it before jumping into the answer. I did OK on some questions, but I need to remember to do a better job of this next time.

http://university.utest.com/recorded-webinar-exploratory-testing-lucas-dargis/

uTest University Course – What is Exploratory Testing?

I recently authored a course with my good friend Allyson Burk for the uTest University on Exploratory testing. This is a heavily viewed course because it’s part of the “Getting Started at uTest” course track which most new uTesters work through. Since this course is a foundational piece of new uTester’s development, I spent a large amount of time researching and editing to make this as accurate and consumable as possible. So far it’s been well-received earning a 5-star average rating – Not too shabby.

https://university.utest.com/what-is-exploratory-testing/

It even got front-page billing on the University home page:
University Home Page - ET

 

Below is the content of the course:

INTRODUCTION

Today’s development world is much different than it was 10 or even 5 years ago. Product development is fast-paced and must meet the high expectations of users. As development practices have changed, so too have approaches to testing. Many testers are finding that Exploratory Testing (ET) is an effective way to test in these circumstances. The adoption and use of ET has rapidly grown, to the point where it is arguably the most popular testing approach used today.

In this course we’ll start by learning what ET is and how it differs from scripted testing (ST). Then we’ll look at why you should use ET and finally, we’ll wrap up by showing you how to get started. So let’s make like Magellan and start exploring!

THE ‘TRADITIONAL’ APPROACH TO TESTING

Before we look at ET, it might help if we first talk about a different, more traditional approach to testing so we can use that as a reference point to make some comparisons.

Scripted Testing (ST) is a two-step approach to testing. First the tests are written; they are planned, designed and documented. Second, the tests are executed. These two activities are done independently of each other and in many cases, the person who writes the tests is different than the person who executes them.

Generally, the tester executing the tests has some knowledge of the product, or the tests include the information needed to execute them. This is important because without that knowledge or information, the tester might not be able to execute the test or interpret its results.

WHAT IS EXPLORATORY TESTING?

Now lets compare that with ET which is simultaneous learning, test design and test execution. In other words, the tester is designing his or her tests and executing them at the same time. As an exploratory tester, your next action (the next test) is influenced by your previous actions, your observations of the product’s behavior, and your own thought process.

ET also assumes that a significant portion of the testing will be spent learning about the product. As you explore, you become more aware of how the product functions and its expected behavior. You can use that knowledge to design new and better tests. It also helps improve the analysis of the test’s results.

ET table

It is important to make the distinction between ET and other types of unscripted testing because some testers mistakenly believe that all unscripted testing is simply poking the product randomly to see what happens. Performing a series of random actions is called monkey testing and in some cases it may be a valid approach, however this is quite different from ET. With ET actions are the opposite of random — they are deliberate, driven by human thought and reasoning. Your approach is continually refined as new information is gathered and analyzed.

When an explorer goes to an uncharted region of the world, they spend months preparing. They go with a goal in mind and they rely on their abilities to adapt to changing situations. Similarly, an exploratory tester must prepare. They too have a goal and the skills needed to adjust their course. It’s true that monkey testing may occasionally find useful information, but it’s found unexpectedly. It’s the difference between discovery and exploration; luck versus skill.

WHY DO EXPLORATORY TESTING?

One of the most defining qualities of humans is our ability to think. It’s what makes us unique. It allows us to analyze a situation and make decisions, or come up with new ideas, or find new solutions to a problem. We have the capability to learn and continuously improve.

Henry Ford once said “If you always do what you’ve always done, you’ll always get what you’ve always got.” Executing scripted tests over and over will generally produce the same results. Exploratory testers use their cognitive abilities to continually improve the value of their work. They explore and adapt, they learn and adjust. ET is designed to make the most of our intellectual abilities.

ET also takes advantage of the differences between testers. Each tester’s previous experience, skills, and thought process (among other things) causes him or her to view thing in a unique way. Different testers may come up with different ways to test the same function. It may be more beneficial to have three testers test the same function in three unique ways than it is to have all three test it in exactly same way.

In many cases we need to use ET because there is no other alternative. For example, consider a situation where the tester isn’t familiar with the product under test and scripted tests are not available. In this case, it’s up to the tester to study the product, design and execute their tests. As we’ve already seen, this is the very definition of ET.

It can take a significant time to read, comprehend, and execute each step in a test case. This is especially true if you don’t already know the product, or the TC uses language or terms you’re not familiar with. When a tester is tasked to find bugs quickly, they need to be searching for bugs, not reading test cases. They need the freedom to follow promising leads, not the constraints of predefined instructions.

HOW DO I GET STARTED?

If there is one thing all new testers (including new exploratory testers) should do, it’s to start by thinking about the product in general terms; try to see the big picture. Instead of initially focusing on one specific thing, first try to understand the context in which you are working.

Some questions to consider are:

  • Is this a product in development or is it already in production?
  • What is the purpose of the product?
  • Who are the users and how are they going to use it?

Jumping right in and banging on things might produce a bug or too, but if you hope to get the most out of ET, initial preparation and understanding your context is vital.

Now lets see how ET might look in practice. Imagine you’re a brand new tester, your boss comes to you and on your first day and says, “Here you go, this is the latest version of our app. Please begin testing and report any bugs you find.” There are no test cases and no documentation. What do you do? An exploratory tester would do something like this:

1. Get a notebook (or a digital word processor) to take notes as you go.

2. Explore the app as if you just downloaded it and want to use it yourself. If it is not an app you would typically use, then imagine you are the target market for the app.

Take a moment to really get in the mindset of a typical user. Some questions you can ask yourself are what is the goal of this app? Who would benefit from that? How do they benefit?

Let’s say this is an app to show up-to-date stock market information.

Goal of the app: Having stock market data at your fingertips.

Who benefits: Someone who is financially savvy or wants to be and has available income to be investing or has interest in other people’s investments.

How do they benefit: They benefit by the data being timely, accurate, easily accessible and displayed in a way that they can understand quickly.

Don’t worry about finding any bugs right now. You may stumble on them, but this is really just getting used to the app. Jot down anything you find that you want to explore further later.

3. Once you get a feel for the app start going back to the areas that interested you and you thought might be a place of vulnerability in the app. This knowledge about vulnerability is going to come with experience. Don’t worry if you don’t have any experience yet because you are about to get some!

4. One by one, work through each area you’ve earlier identified, exploring every function in that area. Think of what a real user might do. Come up with with some use cases or scenarios and execute those. Then think of variations and execute those. Use the results of your tests to help you come up with new ideas.

5. Focus on one bug at a time, but always be on the lookout for hints of other bugs or suspicious areas. In your notebook, quickly make a note of these areas and how to get back to them. This way you can come back and explore each one later. You could very well end up with 4 or 5 bugs just from investigating the initial bug.

6. Once you’ve exhausted that area or function of the app, move on to your next point of interest. As you repeat this process, remember what you’ve learned so far and use that information to influence your tests.

As you can see from this narrative, you are simultaneously learning, designing tests, and executing the tests. These are the core pieces of ET. Understand this and you’re on your way to becoming an exploratory tester.

CONCLUSION

Different testing needs call for different testing approaches and there are many situations where ET can prove most beneficial. ET is the inverse of scripted testing because it relies on human intellect as opposed to simply following instructions. ET is a process of continual refinement and improvement where testers adapt to situations and the information they’ve gathered. Now that you’ve been introduced to ET, our hope is that will continue to explore exploratory testing and that you can use these skills to provide the most value possible.

ADDITIONAL READING

My Testing bucket list

I realize this might appear to be a vain post, but the intent is to set some goals for myself and make them public in order to help me be more motivated to achieve them. I’ll probably be adding/removing items as my career progresses and my goals change.

  • Win a uTest tester of the year award
  • Mentor a new uTest TTL until they become an ETTL – Prashanti
  • Present at a testing conference
  • Have an A-list tester know who I am
  • Complete all the AST-BBST courses
    • Foundations course
    • Bug Advocacy
    • Test Design
    • Instructors
  • Instruct a AST-BBST course
    • Assistant instructor – Foundations 101-AC (January 2015)
    • Lead instructor
  • Publish an article in various testing publications/blogs
    • http://blog.utest.com/
    • http://www.testingcircus.com/
    • http://www.softwaretestpro.com/

 

 

uTest blog post – Three Things Testers Can Learn From Wine Experts

Originally posted on the uTest blog: http://blog.utest.com/2014/08/22/three-things-testers-can-learn-from-wine-experts/

Also posted here: http://blogs.yourzephyr.com/?p=3987

When I’m not testing, one of my favorite hobbies is alcohol. Wait…that didn’t come out right. What I meant was my hobby is learning about winSommelier_e_Tastevine, beer and sprits. Yeah, that sounds better.

While I do love a cold beer in the summer, a single-malt scotch when I’m feeling sophisticated, or an 1855 classified Bordeaux on special occasions, I think I spend more time studying booze than I do drinking it. I really enjoy learning about the various Appellation d’origine contrôlée (AOCs) in France, and the differences between a Pinot Noir from California and one from Burgundy. I sound pretty smart, huh?

As any cultured, refined wine connoisseur such as myself knows, the true masters of the bottle are called sommeliers. These fine folks are highly trained adult beverage experts who often work in fancy, fine-dining restaurants, setting the wine list, caring for the cellar and working with customers to help them select the perfect wine.

So what could a tester possibly learn from a someone obsessed with booze? Good question! I have three answers.

Be passionate

I have yet to find people who are more passionate about what they do than Master Sommeliers. Need proof? Watch the movie Somm (available on Netflix). The tremendous amount of dedication and effort these people pour (wink, wink) into their work is simply astounding.

A sommelier must be constantly learning and exploring. Each year, a new vintage of every wine is created. That means thousands of new wines are added to the multitude that already exist…and a sommelier is expected to be familiar with with all of them. And you thought the IT world was constantly changing!

There will always be a new product to test, a new approach to learn, a new idea to debate. Testers who are passionate about testing are excited about these new developments as they are opportunities to grow and improve.

Be a servant

From the Demeanor of the Professional Sommelier:

It is important for sommeliers to put themselves in the role of a server; no job or task on the floor is beneath the role of a sommelier; he or she does whatever needs to be done in the moment to take care of the guest.

Sommeliers are at the top of the service food chain. They are highly trained, knowledgeable, and professional people, yet they are also the most humble servants. They realize that their qualities alone are worthless. They must be used in the service of a customer for their value to be realized.

Testers too need to remember that we don’t get paid because we think we’re great testers. We get paid because the person who is paying us values our work. Put your client’s perception of value and quality ahead of your own.

Be an expert

A sommelier would never walk up to your table and say, “I recommend this bottle I found in the back.” They are the ultimate experts in wine selection and food pairing. He or she asks questions: What do you like? What have you had before? What is your budget? They learn about the customer and use that information to help them find exactly the right bottle.

Likewise, a tester should be knowledgeable in the testing field. A good tester doesn’t just randomly go banging on things — they too take a more thoughtful approach. What areas are the most error prone? What parts of the product are the most important? What information would be most useful at this stage in the product’s life cycle? They learn about the product and the circumstances under which they are testing to ensure they provide the most value possible.

Take pride in your work. Understand that testing is not a low-skilled job; it is a highly cognitive profession with demands on your professionalism, communication skills, and attention to detail. It takes a lot of effort, study and experience to become an expert (or so I’m told), but that should be the goal of every tester.

uTest blog post – Certified and Proud?

I recently wrote a post for the uTest blog about my certification story. It was well received earning a good number of comments and social shares.

http://blog.utest.com/2014/08/04/certified-and-proud-a-testers-journey-part-i/
http://blog.utest.com/2014/08/05/certified-and-proud-a-testers-journey-part-ii/

 

A Gold-rated tester and Enterprise Test Team Lead (TTL) at uTest, Lucas Dargis has been an invaluable fixture in the uTest Community for 2 1/2 years, mentoring hundreds of testers and championing them to become better testers. As a software consultant, Lucas has also led the testing efforts of mission-critical and flagship projects for several global companies.

Here, 2013 uTester of the Year Lucas Dargis here shares his journey on becoming ISTQB-certified, and also tackles some of the controversy surrounding certifications.

In case you missed it, testing certification is somewhat of a polarizing topic. Sorry for stating the obvious, but I needed a good hook and that’s the best I could come up with. What follows is the story of my journey to ISTQB certification, and how and why I pursued it in the first place. My reasons and what I learned might surprise you, so read on and be amazed!

Certifications are evil

Early in my testing career, I was a sponge for information. I indiscriminately absorbed every piece of testing knowledge I could get my hands on. I guess that makes sense for a new tester — I didn’t know much, so I didn’t know what to believe and what to be suspicious of. I also didn’t have much foundational knowledge with which to form my own opinions.

As you might expect, one of the first things I did was look into training and certifications. I quickly found that the pervasive opinion towards certifications (at least the opinion of thought leaders I was learning from) was that they were at best a waste of time, and at worst, a dangerous detriment to the testing industry.

In typical ignoramus (It’s a word, I looked it up) fashion, I embraced the views of my industry leaders as my own, even though I didn’t really understand them. Anytime someone would have something positive to say about certification, I’d recite all the anti-certification talking points I’d learned as if I was an expert on the topic. “You’re an idiot” and “I’d never hire a certified tester” were phrases I uttered more than once.

A moment of clarity

Then one fine day, I was having a heated political debate with one of my friends (I should clarify…ex-friend). We had conflicting views on the topic of hula hoop subsidies. He could repeat the points the talking heads on TV made, but when I challenged him, asking prodding questions trying to get him to express his own unique ideas, he just went around in circles (see what I did there?).

Like so many other seemingly politically savvy people, his views and opinions were formed for him by his party leaders. He had no experience or expertise in the area we were debating, but he sure acted like the ultimate authority. Suddenly, it dawned on me that despite my obviously superior hip-swiveling knowledge, I wasn’t that much different from him. My views on certifications and the reasons behind those views came from someone else.

As a tester, I pride myself on my ability to question everything, and to look at situations objectively in order to come to a useful and informative conclusion. But when it came to certifications, I was adopting the opinions of others and acting like I was an expert in an area I knew nothing about. After that brief moment of clarity, I realized how foolish I’d been, and felt quite embarrassed. I needed to find the truth about certifications myself.

Now let me pause here for a second to clarify that I’m not saying everyone should go take a certification test so they can have first-hand experience. I think we all agree that it’s prudent to learn from the experience of others. For example, not all of us have put our hand in a fire, but we all know that if you do, it will get burned.

It’s perfectly fine to let other people influence your opinions as long as you are honest about the source of those opinions. For example, say:

Testers I respect have said that testing certificates are potentially dangerous and a waste of time. Their views make sense to me, so it’s my opinion that testers should look for other ways to improve and demonstrate their testing abilities.

However, if you’re going to act like an expert who has all the answers, you should probably be an expert who has all the answers.

Studying for the exam

I took my preparation pretty seriously. Most of my studying material came from this book which I read several times. I spent a lot of time reading the syllabus, taking practice exams and downloading a few study apps on my phone. I also looked through all the information on the ISTQB site, learning all I could about the organization and the exam.

Since I knew all the anti-certification rhetoric about how this test simply measures your ability to memorize, I memorized all the key points from the syllabus (such as the 7 testing principles). But I also took my studying further, making sure I understood the concepts so I could answer the K3 and K4 (apply and analyze) questions.

Headed into the test, I felt quite confident that I was going to ace it.

Be sure check out the second part of Lucas’ story at the uTest Blog.

————————————————————————————————-

his is the second part of tester and uTest Enterprise Test Team Lead Lucas Dargis’ journey on becoming ISTQB-certified. Be sure to check out Part One from
yesterday.

The test

After about 3 minutes, I realized just how ridiculous the test was. Some of the questions were so obvious it was insulting, some were so irrelevant they were infuriating, and others were so ambiguous all you could do was guess.

Interestingly, testers with experience in context-driven testing will actually be at a disadvantage on this test. When you understand that the context of a question influences the answer, you realize that many of the questions couldn’t possibly have only one correct answer, because no context was specified.

You are allotted 60 minutes to complete the test, but I was done and out of the building in 27 minutes. That I finished quickly wasn’t because I knew all the answers — it was rather the exact opposite. Most of the questions were so silly, that all I could do was select answers randomly. Here are two examples:

“Who should lead a walkthrough review?” – Really? I was expected to memorize all the participants of all the different types of meetings, most of which I’ve never seen any team actually utilize?

“Test cases are designed during which testing phase?” – Umm…new tests and test cases should be identified and designed at all phases of the project as things change and your understanding develops.

According to the syllabus, there are “right” answers to all the questions, but most thinking testers, those not bound by the rigidness of “best practices,” will struggle because you know there is no right answer.

Despite guessing on many questions, I ended up passing the exam, but that really wasn’t a surprise. The test only requires a 65% to pass, so  a person could probably pass with minimal preparation, simply making educated guesses. I left the test in a pretty grumpy mood.

 

The aftermath

For the next few days, I was annoyed. I felt like I had completely wasted my time. But then I started thinking about why I took the test in the first place, and what that certification really meant. As a tester striving to become an expert, I wanted to know for myself what certifications were about. Well — now I have first-hand experience with the process. I’m able to talk about certifications more intelligently because my opinions and views about them are my own, not borrowed from others.

Former tennis great Arthur Ashe once said, “Success is a journey, not a destination. The doing is often more important than the outcome.” I think this sums up certifications well. For me, there was no value in the destination (passing the exam). I’m not proud of it, and I don’t think it adds to or detracts from my value as a tester. However, I felt there was value in the journey.

Through my studying, I actually did learn a thing our two about testing. I was able to build some structure around the basic testing concepts. I learned some new testing terms that I have used to help explain concepts such as tester independence. Studying gave me practice analyzing and questioning the “instructional” writing of others (some of which I found inaccurate, misleading or simply worthless). The whole process gave me insight into what is being taught, which helps me better understand why some testers and test managers believe and behave the way they do.

But more importantly, I became aware of the way I formulate opinions and of how susceptible I am to the teachings of confident and powerful people.

The lessons I learned from testing leaders early in my career freed me from the constraints of the traditional ways of thinking about testing, but in return, I took on the binds associated with more modern testing thinking. Ultimately, I came to a conclusion similar to that of many wise testers before me, but I did so on my terms, in a way that I’m satisfied with.

One day, I hope to have a conversation with some of the more boisterous certification opponents and say, “I decided to get certified, in part, because you were so adamantly against it. I don’t do it out of defiance, but rather out of a quest for deeper understanding.”

If you made it this far, I thank you for sharing this journey with me. Make a mention that you finished the story in the Comments below, and I’ll send you a pony.

Key Decision: Become a full-time uTester

Decision

So I’m about 7 months late in writing this, but the key decision I made back in January was to leave my job as the test lead for Brock Solutions and become a full-time uTester.

Working at Brock, I enjoyed the people I was working with and was reasonably content with the company. But he product was the most untestable and convoluted piece of software I’ve ever seen, which resulted in extremely challenging work. It was difficult to add value as most of the day was spent trying to get the system ready to test. The work wasn’t what I thought it was going to be when I signed up and at the end of the day I didn’t enjoy my job.

In early January, Steve Moses, a Sr. project manager at uTest contacted me. He said he had an exciting opportunity to be a dedicated test lead for one of uTest’s largest clients. It was a 6-month contract with the possibility of an extension. It really wasn’t much of a decision. How could I turn down representing my favorite company, the one I spend my free time working for anyway?

Expectations

  • Build my reputation as as a top uTester and leader in the community
  • Develop a better understanding of e-commerce
  • Gain experience working with people from industries
  • Influence the education and growth of others
  • Find more challenging and rewarding career options

How to Write a Quality Bug Report

I worked with Aaron Weintrob to put together a uTu course on how to write quality bug reports. The intended audience of this course is new uTesters with little or no testing experience. We kept it relatively short and simply highlighted the key areas. We may create additional courses for various sections where we can dig in a little deeper. Here is a link to the course. Below is the course’s content:

INTRODUCTION

Writing a good bug report is one of the most talked-about topics in the testing world. The art of creating a well-written bug report requires a balanced combination of testing and communication skills. This course provides advice and tips geared towards helping you create bug reports that are informative and actionable, thus improving their value to the customer.

WHAT MAKES A GOOD BUG REPORT?

Most testers understand the role of a bug report is to provide information, however a “good” or valuable bug report takes that a step further and provides useful information in an efficient way.

To help us get started writing valuable bug reports, we are going to focus on a few key areas:

  • The Title
  • Actions Performed (Steps)
  • Expected and Actual Results
  • Attachments

THE TITLE – THE GOOD, THE BAD, THE UGLY!

The title is the face of your report. It’s the first thing anyone sees and it’s importance cannot be overstated. A good title helps reduce duplicate issues and can quickly convey a summary of the bug.

It’s best to avoid generic problems in the title. For example, these should never be used:

  • XYX is not working properly
  • Issue with XYZ
  • XYZ is corrupted/does not look right

The above example titles add little value in describing the problem. By nature, every report is describing something that is not working as it should. Be specific about what makes it “not working.”

Instead of: Sorting is not working properly.
Try: Sorting is happening in reverse order.

Instead of: Issues with GUI on navigation bar.
Try: Navigation bar is wrapping to a second line.

Often times, bugs are migrated into the developer’s database that may contain hundreds, if not thousands, of other issues. Imagine trying to search this database for “navigation bar”. That search will return every issue related to the navigation bar. Searching for “wrapping to second line” is much more specific making it easier to find the bug. Your bug report needs to survive (and be useful) beyond the current test cycle; a strong title will help it through it’s journey.

ACTIONS PERFORMED – ADVICE FOR EXPLAINING YOUR STEPS

This is the body of your report. The goal of this section is to show the reader how to reproduce the bug. Since this area usually contains the most information, it’s important to keep it concise and easy to read. Always number your steps and kept them short and to the point.

Tip: Using a prerequisite can reduce the number of steps.
Instead of listing out every step to login in, start your steps with: “Prerequisite: User is logged in”

Tip: Find the direct path to the bug
Often times, testers will stop at the point where they found a bug and log their last few actions. However, the most helpful bug reports are those that distill the report down to the core reproduction steps.

It’s a good exercise to reproduce the bug by following the steps you’ve just outlined. This will help ensure you’ve included everything the customer will need to reproduce it as well.

Sometimes digging a little deeper below the surface of the bug can add additional value. Here are some examples of how adding a bit more effort or thought will produce a higher quality report.

Example 1: Provide additional useful information
Scenario: You find that a video does not play.
Good: Mention if it happened on all videos and not just the one mentioned in report.
Better: Specify if the issue is reproducible on more than one browser or device.
Best: Upload a speed test showing that bandwidth was adequate when testing was happening.
Lesson: Try to identify and answer follow-up questions before the customer asks them.

Example 2: Report the bug, not a symptom of the bug
Scenario: We are testing an Address input field. We find that the Address field allows “1234567890″ and it also allows “!@#$%^&*()_+”
Lesson: These are two different symptoms of the same bug. Closer inspection would reveal that the real issue is the Address field isn’t being validated at all. The problem may be more serious than the first symptom you find.

EXPECTED AND ACTUAL RESULTS – WOULDA, COULDA, SHOULDA

Now that you have described how to reproduce the bug, you need to explain the problem and the desired behavior.

Tip: When describing expected results, explain what should happen, not what shouldn’t happen.
Instead of: The app shouldn’t crash.
Try: The user is taken to XYZ screen.

Tip: When describing actual results, describe what did happen, not what didn’t happen.
Instead of: The user wasn’t taken to the XYZ screen.
Try: The user remained on the ABC screen.

ATTACHMENTS – WHAT TO DO AND WHAT NOT TO DO

Attachments add to the bug’s value by offering proof of the bug’s existence, enabling the customer to reproduce it or helping the developer fix it. Each attachment should add to the value of the bug in at least one of these three ways.

The following are some tips and guidelines to keep in mind when adding attachments:

IMAGES

  • Adding images is a quick way to add context to your bug. Consider adding an image even if you also have a video.
  • Highlight the area(s) of interest in your image.
  • Attach the image files directly to the report. Don’t put images in a Word document or a zip file.
  • Use images to illustrate static issues.

VIDEOS

  • Video confirms your steps were accurate at the time the issue was created. For example, a screen grab of an error message isn’t as useful as seeing what went into creating that error message.
  • Actions in the video should match the steps listed in the bug report.
  • Videos should be trimmed to only show the bug.
  • Provide video if the steps are complex.
  • External/live videos can be more impactful than mirrored videos because you can see hand gestures or you touching a button on the screen.

LOG FILES AND OTHER TIPS

Avoid proprietary file types (like .docx). Use .txt instead.
Avoid compressed (.zip) files unless specifically asked for or approved by the TTL, PM, or customer.

UTEST ETIQUETTE – GENERAL OVERVIEW OF PROPER BEHAVIOR

It is important to remember that you are representing the TTL, the test team, and all of uTest when you work on the test cycle. Your fellow testers rely on you to write a good title for your bug report so they won’t file a duplicate bug. TTLs depend on clean, good reports to ensure the customer receives value from the cycle. uTest needs quality work from everyone so we can continue to work in the field we all love.

ADDITIONAL READING

Here are some valuable discussions about bug reports from the uTest Forums:

Two contributions to the uTest University

Back in December of 2013, uTest officially launched the uTest University (blog post) which is intended to be a single source for testers of all experience levels to access free training resources. This is a neat opportunity for testers to contribute to the growth and development of the testing community by creating courses and writing articles. The university also offers  Author Page

My first course was derived from a uTest forum’s post I I wrote back in June of 2013. I was on a cycle where the customer required logs from Charles Web Debugging Proxy be attached to every bug report, but none of the testers (myself included) or the knew what that was or how to use it. I spent some time learning how to use the tool and then put together a tutorial to share with the rest of the team. Fast forward 8 months later several other customers started required the same thing. To make the information a bit easier to find the tutorial was turned into a uTu (uTest university) course:
How to Set Up Charles Web Debugging Proxy for iOS Devices and Windows 8

My second course came at the request of the uTest Community Management team. They needed a tutorial for new testers to show them how to create videos (screencasts) of their bugs. They specifically wanted it based around the free tool Screencast-O-Matic. I had actually never used that tool before, so I spent some time getting familiar with the tool. I also compiled a list of suggestions and tips based on things I see frequently in the videos of other testers. The result is:
How to Set Up and Use Screencast-O-Matic

 

A Tester’s Haiku

From time to time uTest puts on fun contests for the community. This month the objective was to come up with the best testing-related haiku. Over 90 testers participated, coming up with some great poems ranging from serious to silly to clever.

There was one haiku that got more attention than most so I felt it warranted some further discussion. It was one of the first submissions and it was well-written so the uTest CM team used it to promote the contest. It was mentioned it on the forums and posted it on Twitter. It was nominated as one of the 10 finalists and went on to win the contest, earning the vote of 21 other testers. Here is the poem in question:

Gifted are testers
Fixing a world of errors
Coded by others

It follows haiku syllable structure and has a nice flow, but I feel it’s misleading and potentially damaging. I’ll quickly address each line.

Gifted are testers
Can’t argue with that 🙂

Fixing a world of errors
This shouldn’t be a  great revelation, but in general, testers do not fix errors; We provide information and dispel misconceptions. One of the way’s we do this is by identifying errors. Some might try to make an argument that part of resolving a problem is finding it, but testers who understand what we do to provide value, realize the importance of making the distinction between finding an fixing a bug. These are two different actions that required two different skillsets.

Coded by others
This line really bothers me because it promotes an us vs. them mentality. Even though someone else (the developer) did write the code, it doesn’t help to point the finger at them. It’s a testers job to identify mistakes in other people’s work and that puts us in a delicate situation. It needs to be handled without judgment or condemnation. Bugs are an inevitable part of software development and since quality is the responsibility of the entire team, we need to work together, take responsibility together and avoid blaming each other.

Now, I’m not trying to say the author of this haiku meant to say that testers literally write the code to fix bugs or that developers and testers are enemies and I don’t mean to pick on him/her, but the fact that so many other testers thought this haiku was a good representation of testing troubled me. I’m sure this can be partially attributed to uTest’s endorsement and also because of language issues, but these are two themes that I see often and wanted to briefly address them.

My Entry
In case you were wondering, I did write a haiku. While I was a finalist, I didn’t make it into the top 3. Here’s my entry:

found and reported
a customer enlightened
value provided

Decision Review: Pursue my MBA

My first Key Decision was to Pursue my MBA. Let’s see how that decision worked out shall we?

After only one semester it became quite clear that pursing my MBA was a poor decision. Hmm… so far I’m 0 for 1. You might think I quit before I even gave it a chance, but I struggled just to finish out the semester. After a lot of reflection on what exactly went wrong, I was able to narrow it down to three things. The impact on my family, the educational aspect was disappointing, and the opportunity costs were too high.

Time with family

Going in, I knew that class time would reduce the amount of time with my family, but that didn’t fully sink in until it became real. A 3 hour class, sandwiched between a 45 minute commute meant that two days a week, I wouldn’t get to see my wife or kids. How could I possibly be so cruel as to deny my family the pleasure of my company? My absence did put a lot of additional burden on my wife and the kids missed wrestling with me, but mostly I’m just selfish; I missed them too much.

Disappointing

After my first class I was super excited. The instructor was fun and engaging. He asked open ended questions and joined myself and others in debate and discussion. Unfortunately that didn’t last long. As we approached our first exam it became clear that grad school is merely a continuation of undergrad. You’re still graded and evaluated on what you know, not on how you think, reason, or your ability to learn and execute. Multiple-choice Scantron tests…SERIOUSLY!? The entire dynamic of the class changed. Boring lectures attempting to “teach” the “right” answers to unimportant questions.

I was really looking forward to learning from my fellow students, from their experiences in industries and business areas new to me. Sadly, most only cared about their grade. “Is that going to be on the test” was the most common question asked. The lack of interest in true self-improvement and overall “quality” of the students admitted to the program was disheartening.

To be fair, I only took one class at one university, but as UNCG is a highly ranked university, I have to suspect that my experience isn’t all that unique.

Opportunity costs

Similar to how my time focusing on school reduced my family time, it also took the place of other career-related opportunities, specifically uTest. The additional 20 hours a week meant that I had to completely abandon uTest. My good friend Rex helped me realize what an expensive trade-off that really was. I had spent two years building my reputation as a tester and done so quite successfully. I was invited to best projects, I was able to participate in various company initiatives (like the TTL training and evaluation program) and I had a large, visible presence in the community. I was sacrificing a opportunity that provided me continuous growth opportunities, respect, and enjoyment for the chance to become one of the select 100,000 MBAs that graduate each year.

What I learned

Structured, standardized learning no longer appeals to me. I learn more from conversations over a few beers than I do listening to a lecture. The ability to execute is more important than the ability to memorize business trivia. Doing something just because it’s uncomfortable is not a good reason. I thought I was being courageous by stepping out of my comfort zone, but know I see I was just being an idiot.

In the end I realized that the expectations I listed in my decision post can be fulfilled by continuing to develop my testing career. I was already working towards all of them and making great progress. At this point in my life and career, the benefits of an MBA program didn’t outweigh the costs. Maybe someday I’ll regret not having those three letters after my name, but today is not that day.