I worked with Aaron Weintrob to put together a uTu course on how to write quality bug reports. The intended audience of this course is new uTesters with little or no testing experience. We kept it relatively short and simply highlighted the key areas. We may create additional courses for various sections where we can dig in a little deeper. Here is a link to the course. Below is the course’s content:
INTRODUCTION
Writing a good bug report is one of the most talked-about topics in the testing world. The art of creating a well-written bug report requires a balanced combination of testing and communication skills. This course provides advice and tips geared towards helping you create bug reports that are informative and actionable, thus improving their value to the customer.
WHAT MAKES A GOOD BUG REPORT?
Most testers understand the role of a bug report is to provide information, however a “good” or valuable bug report takes that a step further and provides useful information in an efficient way.
To help us get started writing valuable bug reports, we are going to focus on a few key areas:
- The Title
- Actions Performed (Steps)
- Expected and Actual Results
- Attachments
THE TITLE – THE GOOD, THE BAD, THE UGLY!
The title is the face of your report. It’s the first thing anyone sees and it’s importance cannot be overstated. A good title helps reduce duplicate issues and can quickly convey a summary of the bug.
It’s best to avoid generic problems in the title. For example, these should never be used:
- XYX is not working properly
- Issue with XYZ
- XYZ is corrupted/does not look right
The above example titles add little value in describing the problem. By nature, every report is describing something that is not working as it should. Be specific about what makes it “not working.”
Instead of: Sorting is not working properly.
Try: Sorting is happening in reverse order.
Instead of: Issues with GUI on navigation bar.
Try: Navigation bar is wrapping to a second line.
Often times, bugs are migrated into the developer’s database that may contain hundreds, if not thousands, of other issues. Imagine trying to search this database for “navigation bar”. That search will return every issue related to the navigation bar. Searching for “wrapping to second line” is much more specific making it easier to find the bug. Your bug report needs to survive (and be useful) beyond the current test cycle; a strong title will help it through it’s journey.
ACTIONS PERFORMED – ADVICE FOR EXPLAINING YOUR STEPS
This is the body of your report. The goal of this section is to show the reader how to reproduce the bug. Since this area usually contains the most information, it’s important to keep it concise and easy to read. Always number your steps and kept them short and to the point.
Tip: Using a prerequisite can reduce the number of steps.
Instead of listing out every step to login in, start your steps with: “Prerequisite: User is logged in”
Tip: Find the direct path to the bug
Often times, testers will stop at the point where they found a bug and log their last few actions. However, the most helpful bug reports are those that distill the report down to the core reproduction steps.
It’s a good exercise to reproduce the bug by following the steps you’ve just outlined. This will help ensure you’ve included everything the customer will need to reproduce it as well.
Sometimes digging a little deeper below the surface of the bug can add additional value. Here are some examples of how adding a bit more effort or thought will produce a higher quality report.
Example 1: Provide additional useful information
Scenario: You find that a video does not play.
Good: Mention if it happened on all videos and not just the one mentioned in report.
Better: Specify if the issue is reproducible on more than one browser or device.
Best: Upload a speed test showing that bandwidth was adequate when testing was happening.
Lesson: Try to identify and answer follow-up questions before the customer asks them.
Example 2: Report the bug, not a symptom of the bug
Scenario: We are testing an Address input field. We find that the Address field allows “1234567890″ and it also allows “!@#$%^&*()_+”
Lesson: These are two different symptoms of the same bug. Closer inspection would reveal that the real issue is the Address field isn’t being validated at all. The problem may be more serious than the first symptom you find.
EXPECTED AND ACTUAL RESULTS – WOULDA, COULDA, SHOULDA
Now that you have described how to reproduce the bug, you need to explain the problem and the desired behavior.
Tip: When describing expected results, explain what should happen, not what shouldn’t happen.
Instead of: The app shouldn’t crash.
Try: The user is taken to XYZ screen.
Tip: When describing actual results, describe what did happen, not what didn’t happen.
Instead of: The user wasn’t taken to the XYZ screen.
Try: The user remained on the ABC screen.
ATTACHMENTS – WHAT TO DO AND WHAT NOT TO DO
Attachments add to the bug’s value by offering proof of the bug’s existence, enabling the customer to reproduce it or helping the developer fix it. Each attachment should add to the value of the bug in at least one of these three ways.
The following are some tips and guidelines to keep in mind when adding attachments:
IMAGES
- Adding images is a quick way to add context to your bug. Consider adding an image even if you also have a video.
- Highlight the area(s) of interest in your image.
- Attach the image files directly to the report. Don’t put images in a Word document or a zip file.
- Use images to illustrate static issues.
VIDEOS
- Video confirms your steps were accurate at the time the issue was created. For example, a screen grab of an error message isn’t as useful as seeing what went into creating that error message.
- Actions in the video should match the steps listed in the bug report.
- Videos should be trimmed to only show the bug.
- Provide video if the steps are complex.
- External/live videos can be more impactful than mirrored videos because you can see hand gestures or you touching a button on the screen.
LOG FILES AND OTHER
Avoid proprietary file types (like .docx). Use .txt instead.
Avoid compressed (.zip) files unless specifically asked for or approved by the TTL, PM, or customer.
UTEST ETIQUETTE – GENERAL OVERVIEW OF PROPER BEHAVIOR
It is important to remember that you are representing the TTL, the test team, and all of uTest when you work on the test cycle. Your fellow testers rely on you to write a good title for your bug report so they won’t file a duplicate bug. TTLs depend on clean, good reports to ensure the customer receives value from the cycle. uTest needs quality work from everyone so we can continue to work in the field we all love.
ADDITIONAL READING
Here are some valuable discussions about bug reports from the uTest Forums: