New uTest blog post – Certified and Proud?

I recently wrote a post for the uTest blog about my certification story. It was well received earning a good number of comments and social shares.

http://blog.utest.com/2014/08/04/certified-and-proud-a-testers-journey-part-i/
http://blog.utest.com/2014/08/05/certified-and-proud-a-testers-journey-part-ii/

 

A Gold-rated tester and Enterprise Test Team Lead (TTL) at uTest, Lucas Dargis has been an invaluable fixture in the uTest Community for 2 1/2 years, mentoring hundreds of testers and championing them to become better testers. As a software consultant, Lucas has also led the testing efforts of mission-critical and flagship projects for several global companies.

Here, 2013 uTester of the Year Lucas Dargis here shares his journey on becoming ISTQB-certified, and also tackles some of the controversy surrounding certifications.

In case you missed it, testing certification is somewhat of a polarizing topic. Sorry for stating the obvious, but I needed a good hook and that’s the best I could come up with. What follows is the story of my journey to ISTQB certification, and how and why I pursued it in the first place. My reasons and what I learned might surprise you, so read on and be amazed!

Certifications are evil

Early in my testing career, I was a sponge for information. I indiscriminately absorbed every piece of testing knowledge I could get my hands on. I guess that makes sense for a new tester — I didn’t know much, so I didn’t know what to believe and what to be suspicious of. I also didn’t have much foundational knowledge with which to form my own opinions.

As you might expect, one of the first things I did was look into training and certifications. I quickly found that the pervasive opinion towards certifications (at least the opinion of thought leaders I was learning from) was that they were at best a waste of time, and at worst, a dangerous detriment to the testing industry.

In typical ignoramus (It’s a word, I looked it up) fashion, I embraced the views of my industry leaders as my own, even though I didn’t really understand them. Anytime someone would have something positive to say about certification, I’d recite all the anti-certification talking points I’d learned as if I was an expert on the topic. “You’re an idiot” and “I’d never hire a certified tester” were phrases I uttered more than once.

A moment of clarity

Then one fine day, I was having a heated political debate with one of my friends (I should clarify…ex-friend). We had conflicting views on the topic of hula hoop subsidies. He could repeat the points the talking heads on TV made, but when I challenged him, asking prodding questions trying to get him to express his own unique ideas, he just went around in circles (see what I did there?).

Like so many other seemingly politically savvy people, his views and opinions were formed for him by his party leaders. He had no experience or expertise in the area we were debating, but he sure acted like the ultimate authority. Suddenly, it dawned on me that despite my obviously superior hip-swiveling knowledge, I wasn’t that much different from him. My views on certifications and the reasons behind those views came from someone else.

As a tester, I pride myself on my ability to question everything, and to look at situations objectively in order to come to a useful and informative conclusion. But when it came to certifications, I was adopting the opinions of others and acting like I was an expert in an area I knew nothing about. After that brief moment of clarity, I realized how foolish I’d been, and felt quite embarrassed. I needed to find the truth about certifications myself.

Now let me pause here for a second to clarify that I’m not saying everyone should go take a certification test so they can have first-hand experience. I think we all agree that it’s prudent to learn from the experience of others. For example, not all of us have put our hand in a fire, but we all know that if you do, it will get burned.

It’s perfectly fine to let other people influence your opinions as long as you are honest about the source of those opinions. For example, say:

Testers I respect have said that testing certificates are potentially dangerous and a waste of time. Their views make sense to me, so it’s my opinion that testers should look for other ways to improve and demonstrate their testing abilities.

However, if you’re going to act like an expert who has all the answers, you should probably be an expert who has all the answers.

Studying for the exam

I took my preparation pretty seriously. Most of my studying material came from this book which I read several times. I spent a lot of time reading the syllabus, taking practice exams and downloading a few study apps on my phone. I also looked through all the information on the ISTQB site, learning all I could about the organization and the exam.

Since I knew all the anti-certification rhetoric about how this test simply measures your ability to memorize, I memorized all the key points from the syllabus (such as the 7 testing principles). But I also took my studying further, making sure I understood the concepts so I could answer the K3 and K4 (apply and analyze) questions.

Headed into the test, I felt quite confident that I was going to ace it.

Be sure check out the second part of Lucas’ story at the uTest Blog.

————————————————————————————————-

his is the second part of tester and uTest Enterprise Test Team Lead Lucas Dargis’ journey on becoming ISTQB-certified. Be sure to check out Part One from
yesterday.

The test

After about 3 minutes, I realized just how ridiculous the test was. Some of the questions were so obvious it was insulting, some were so irrelevant they were infuriating, and others were so ambiguous all you could do was guess.

Interestingly, testers with experience in context-driven testing will actually be at a disadvantage on this test. When you understand that the context of a question influences the answer, you realize that many of the questions couldn’t possibly have only one correct answer, because no context was specified.

You are allotted 60 minutes to complete the test, but I was done and out of the building in 27 minutes. That I finished quickly wasn’t because I knew all the answers — it was rather the exact opposite. Most of the questions were so silly, that all I could do was select answers randomly. Here are two examples:

Who should lead a walkthrough review?” – Really? I was expected to memorize all the participants of all the different types of meetings, most of which I’ve never seen any team actually utilize?

Test cases are designed during which testing phase?” – Umm…new tests and test cases should be identified and designed at all phases of the project as things change and your understanding develops.

According to the syllabus, there are “right” answers to all the questions, but most thinking testers, those not bound by the rigidness of “best practices,” will struggle because you know there is no right answer.

Despite guessing on many questions, I ended up passing the exam, but that really wasn’t a surprise. The test only requires a 65% to pass, so  a person could probably pass with minimal preparation, simply making educated guesses. I left the test in a pretty grumpy mood.

 

The aftermath

For the next few days, I was annoyed. I felt like I had completely wasted my time. But then I started thinking about why I took the test in the first place, and what that certification really meant. As a tester striving to become an expert, I wanted to know for myself what certifications were about. Well — now I have first-hand experience with the process. I’m able to talk about certifications more intelligently because my opinions and views about them are my own, not borrowed from others.

Former tennis great Arthur Ashe once said, “Success is a journey, not a destination. The doing is often more important than the outcome.” I think this sums up certifications well. For me, there was no value in the destination (passing the exam). I’m not proud of it, and I don’t think it adds to or detracts from my value as a tester. However, I felt there was value in the journey.

Through my studying, I actually did learn a thing our two about testing. I was able to build some structure around the basic testing concepts. I learned some new testing terms that I have used to help explain concepts such as tester independence. Studying gave me practice analyzing and questioning the “instructional” writing of others (some of which I found inaccurate, misleading or simply worthless). The whole process gave me insight into what is being taught, which helps me better understand why some testers and test managers believe and behave the way they do.

But more importantly, I became aware of the way I formulate opinions and of how susceptible I am to the teachings of confident and powerful people.

The lessons I learned from testing leaders early in my career freed me from the constraints of the traditional ways of thinking about testing, but in return, I took on the binds associated with more modern testing thinking. Ultimately, I came to a conclusion similar to that of many wise testers before me, but I did so on my terms, in a way that I’m satisfied with.

One day, I hope to have a conversation with some of the more boisterous certification opponents and say, “I decided to get certified, in part, because you were so adamantly against it. I don’t do it out of defiance, but rather out of a quest for deeper understanding.”

If you made it this far, I thank you for sharing this journey with me. Make a mention that you finished the story in the Comments below, and I’ll send you a pony.

Key Decision: Become a full-time uTester

Decision

So I’m about 7 months late in writing this, but the key decision I made back in January was to leave my job as the test lead for Brock Solutions and become a full-time uTester.

Working at Brock, I enjoyed the people I was working with and was reasonably content with the company. But he product was the most untestable and convoluted piece of software I’ve ever seen, which resulted in extremely challenging work. It was difficult to add value as most of the day was spent trying to get the system ready to test. The work wasn’t what I thought it was going to be when I signed up and at the end of the day I didn’t enjoy my job.

In early January, Steve Moses, a Sr. project manager at uTest contacted me. He said he had an exciting opportunity to be a dedicated test lead for one of uTest’s largest clients. It was a 6-month contract with the possibility of an extension. It really wasn’t much of a decision. How could I turn down representing my favorite company, the one I spend my free time working for anyway?

Expectations

  • Build my reputation as as a top uTester and leader in the community
  • Develop a better understanding of e-commerce
  • Gain experience working with people from industries
  • Influence the education and growth of others
  • Find more challenging and rewarding career options

How to Write a Quality Bug Report

I worked with Aaron Weintrob to put together a uTu course on how to write quality bug reports. The intended audience of this course is new uTesters with little or no testing experience. We kept it relatively short and simply highlighted the key areas. We may create additional courses for various sections where we can dig in a little deeper. Here is a link to the course. Below is the course’s content:

INTRODUCTION

Writing a good bug report is one of the most talked-about topics in the testing world. The art of creating a well-written bug report requires a balanced combination of testing and communication skills. This course provides advice and tips geared towards helping you create bug reports that are informative and actionable, thus improving their value to the customer.

WHAT MAKES A GOOD BUG REPORT?

Most testers understand the role of a bug report is to provide information, however a “good” or valuable bug report takes that a step further and provides useful information in an efficient way.

To help us get started writing valuable bug reports, we are going to focus on a few key areas:

  • The Title
  • Actions Performed (Steps)
  • Expected and Actual Results
  • Attachments

THE TITLE – THE GOOD, THE BAD, THE UGLY!

The title is the face of your report. It’s the first thing anyone sees and it’s importance cannot be overstated. A good title helps reduce duplicate issues and can quickly convey a summary of the bug.

It’s best to avoid generic problems in the title. For example, these should never be used:

  • XYX is not working properly
  • Issue with XYZ
  • XYZ is corrupted/does not look right

The above example titles add little value in describing the problem. By nature, every report is describing something that is not working as it should. Be specific about what makes it “not working.”

Instead of: Sorting is not working properly.
Try: Sorting is happening in reverse order.

Instead of: Issues with GUI on navigation bar.
Try: Navigation bar is wrapping to a second line.

Often times, bugs are migrated into the developer’s database that may contain hundreds, if not thousands, of other issues. Imagine trying to search this database for “navigation bar”. That search will return every issue related to the navigation bar. Searching for “wrapping to second line” is much more specific making it easier to find the bug. Your bug report needs to survive (and be useful) beyond the current test cycle; a strong title will help it through it’s journey.

ACTIONS PERFORMED – ADVICE FOR EXPLAINING YOUR STEPS

This is the body of your report. The goal of this section is to show the reader how to reproduce the bug. Since this area usually contains the most information, it’s important to keep it concise and easy to read. Always number your steps and kept them short and to the point.

Tip: Using a prerequisite can reduce the number of steps.
Instead of listing out every step to login in, start your steps with: “Prerequisite: User is logged in”

Tip: Find the direct path to the bug
Often times, testers will stop at the point where they found a bug and log their last few actions. However, the most helpful bug reports are those that distill the report down to the core reproduction steps.

It’s a good exercise to reproduce the bug by following the steps you’ve just outlined. This will help ensure you’ve included everything the customer will need to reproduce it as well.

Sometimes digging a little deeper below the surface of the bug can add additional value. Here are some examples of how adding a bit more effort or thought will produce a higher quality report.

Example 1: Provide additional useful information
Scenario: You find that a video does not play.
Good: Mention if it happened on all videos and not just the one mentioned in report.
Better: Specify if the issue is reproducible on more than one browser or device.
Best: Upload a speed test showing that bandwidth was adequate when testing was happening.
Lesson: Try to identify and answer follow-up questions before the customer asks them.

Example 2: Report the bug, not a symptom of the bug
Scenario: We are testing an Address input field. We find that the Address field allows “1234567890″ and it also allows “!@#$%^&*()_+”
Lesson: These are two different symptoms of the same bug. Closer inspection would reveal that the real issue is the Address field isn’t being validated at all. The problem may be more serious than the first symptom you find.

EXPECTED AND ACTUAL RESULTS – WOULDA, COULDA, SHOULDA

Now that you have described how to reproduce the bug, you need to explain the problem and the desired behavior.

Tip: When describing expected results, explain what should happen, not what shouldn’t happen.
Instead of: The app shouldn’t crash.
Try: The user is taken to XYZ screen.

Tip: When describing actual results, describe what did happen, not what didn’t happen.
Instead of: The user wasn’t taken to the XYZ screen.
Try: The user remained on the ABC screen.

ATTACHMENTS – WHAT TO DO AND WHAT NOT TO DO

Attachments add to the bug’s value by offering proof of the bug’s existence, enabling the customer to reproduce it or helping the developer fix it. Each attachment should add to the value of the bug in at least one of these three ways.

The following are some tips and guidelines to keep in mind when adding attachments:

IMAGES

  • Adding images is a quick way to add context to your bug. Consider adding an image even if you also have a video.
  • Highlight the area(s) of interest in your image.
  • Attach the image files directly to the report. Don’t put images in a Word document or a zip file.
  • Use images to illustrate static issues.

VIDEOS

  • Video confirms your steps were accurate at the time the issue was created. For example, a screen grab of an error message isn’t as useful as seeing what went into creating that error message.
  • Actions in the video should match the steps listed in the bug report.
  • Videos should be trimmed to only show the bug.
  • Provide video if the steps are complex.
  • External/live videos can be more impactful than mirrored videos because you can see hand gestures or you touching a button on the screen.

LOG FILES AND OTHER TIPS

Avoid proprietary file types (like .docx). Use .txt instead.
Avoid compressed (.zip) files unless specifically asked for or approved by the TTL, PM, or customer.

UTEST ETIQUETTE – GENERAL OVERVIEW OF PROPER BEHAVIOR

It is important to remember that you are representing the TTL, the test team, and all of uTest when you work on the test cycle. Your fellow testers rely on you to write a good title for your bug report so they won’t file a duplicate bug. TTLs depend on clean, good reports to ensure the customer receives value from the cycle. uTest needs quality work from everyone so we can continue to work in the field we all love.

ADDITIONAL READING

Here are some valuable discussions about bug reports from the uTest Forums:

Two contributions to the uTest University

Back in December of 2013, uTest officially launched the uTest University (blog post) which is intended to be a single source for testers of all experience levels to access free training resources. This is a neat opportunity for testers to contribute to the growth and development of the testing community by creating courses and writing articles. The university also offers  Author Page

My first course was derived from a uTest forum’s post I I wrote back in June of 2013. I was on a cycle where the customer required logs from Charles Web Debugging Proxy be attached to every bug report, but none of the testers (myself included) or the knew what that was or how to use it. I spent some time learning how to use the tool and then put together a tutorial to share with the rest of the team. Fast forward 8 months later several other customers started required the same thing. To make the information a bit easier to find the tutorial was turned into a uTu (uTest university) course:
How to Set Up Charles Web Debugging Proxy for iOS Devices and Windows 8

My second course came at the request of the uTest Community Management team. They needed a tutorial for new testers to show them how to create videos (screencasts) of their bugs. They specifically wanted it based around the free tool Screencast-O-Matic. I had actually never used that tool before, so I spent some time getting familiar with the tool. I also compiled a list of suggestions and tips based on things I see frequently in the videos of other testers. The result is:
How to Set Up and Use Screencast-O-Matic

 

A Tester’s Haiku

From time to time uTest puts on fun contests for the community. This month the objective was to come up with the best testing-related haiku. Over 90 testers participated, coming up with some great poems ranging from serious to silly to clever.

There was one haiku that got more attention than most so I felt it warranted some further discussion. It was one of the first submissions and it was well-written so the uTest CM team used it to promote the contest. It was mentioned it on the forums and posted it on Twitter. It was nominated as one of the 10 finalists and went on to win the contest, earning the vote of 21 other testers. Here is the poem in question:

Gifted are testers
Fixing a world of errors
Coded by others

It follows haiku syllable structure and has a nice flow, but I feel it’s misleading and potentially damaging. I’ll quickly address each line.

Gifted are testers
Can’t argue with that :)

Fixing a world of errors
This shouldn’t be a  great revelation, but in general, testers do not fix errors; We provide information and dispel misconceptions. One of the way’s we do this is by identifying errors. Some might try to make an argument that part of resolving a problem is finding it, but testers who understand what we do to provide value, realize the importance of making the distinction between finding an fixing a bug. These are two different actions that required two different skillsets.

Coded by others
This line really bothers me because it promotes an us vs. them mentality. Even though someone else (the developer) did write the code, it doesn’t help to point the finger at them. It’s a testers job to identify mistakes in other people’s work and that puts us in a delicate situation. It needs to be handled without judgment or condemnation. Bugs are an inevitable part of software development and since quality is the responsibility of the entire team, we need to work together, take responsibility together and avoid blaming each other.

Now, I’m not trying to say the author of this haiku meant to say that testers literally write the code to fix bugs or that developers and testers are enemies and I don’t mean to pick on him/her, but the fact that so many other testers thought this haiku was a good representation of testing troubled me. I’m sure this can be partially attributed to uTest’s endorsement and also because of language issues, but these are two themes that I see often and wanted to briefly address them.

 

Decision Review: Pursue my MBA

My first Key Decision was to Pursue my MBA. Let’s see how that decision worked out shall we?

After only one semester it became quite clear that pursing my MBA was a poor decision. Hmm… so far I’m 0 for 1. You might think I quit before I even gave it a chance, but I struggled just to finish out the semester. After a lot of reflection on what exactly went wrong, I was able to narrow it down to three things. The impact on my family, the educational aspect was disappointing, and the opportunity costs were too high.

Time with family

Going in, I knew that class time would reduce the amount of time with my family, but that didn’t fully sink in until it became real. A 3 hour class, sandwiched between a 45 minute commute meant that two days a week, I wouldn’t get to see my wife or kids. How could I possibly be so cruel as to deny my family the pleasure of my company? My absence did put a lot of additional burden on my wife and the kids missed wrestling with me, but mostly I’m just selfish; I missed them too much.

Disappointing

After my first class I was super excited. The instructor was fun and engaging. He asked open ended questions and joined myself and others in debate and discussion. Unfortunately that didn’t last long. As we approached our first exam it became clear that grad school is merely a continuation of undergrad. You’re still graded and evaluated on what you know, not on how you think, reason, or your ability to learn and execute. Multiple-choice Scantron tests…SERIOUSLY!? The entire dynamic of the class changed. Boring lectures attempting to “teach” the “right” answers to unimportant questions.

I was really looking forward to learning from my fellow students, from their experiences in industries and business areas new to me. Sadly, most only cared about their grade. “Is that going to be on the test” was the most common question asked. The lack of interest in true self-improvement and overall “quality” of the students admitted to the program was disheartening.

To be fair, I only took one class at one university, but as UNCG is a highly ranked university, I have to suspect that my experience isn’t all that unique.

Opportunity costs

Similar to how my time focusing on school reduced my family time, it also took the place of other career-related opportunities, specifically uTest. The additional 20 hours a week meant that I had to completely abandon uTest. My good friend Rex helped me realize what an expensive trade-off that really was. I had spent two years building my reputation as a tester and done so quite successfully. I was invited to best projects, I was able to participate in various company initiatives (like the TTL training and evaluation program) and I had a large, visible presence in the community. I was sacrificing a opportunity that provided me continuous growth opportunities, respect, and enjoyment for the chance to become one of the select 100,000 MBAs that graduate each year.

What I learned

Structured, standardized learning no longer appeals to me. I learn more from conversations over a few beers than I do listening to a lecture. The ability to execute is more important than the ability to memorize business trivia. Doing something just because it’s uncomfortable is not a good reason. I thought I was being courageous by stepping out of my comfort zone, but know I see I was just being an idiot.

In the end I realized that the expectations I listed in my decision post can be fulfilled by continuing to develop my testing career. I was already working towards all of them and making great progress. At this point in my life and career, the benefits of an MBA program didn’t outweigh the costs. Maybe someday I’ll regret not having those three letters after my name, but today is not that day.

Key Decision: Pursue my MBA

Decision

The key decision I made is to pursue my MBA degree. I’ve been wrestling with this decision for several years now. I’ve always enjoyed learning and studying but it just seems like there are so many reasons not to do this. I have two children and a wonderful wife that I want to spend time with, I just started a new job, I coach hockey, I’m heavily involved with uTest, etc.

There are also philosophical reasons pulling me away from the MBA. Is an MBA really the best way for me to learn? I’m mostly interested in testing so why not focus on testing courses from AST or personal mentoring and training from some of testings leaders. I should also quote Will Hunting :)

…you dropped a hundred and fifty grand on a f@#$% education you coulda’ got for a dollar fifty in late charges at the Public Library.

So why did I make this key decision? Well… because it wasn’t an easy, comfortable decision. Whenever I find an uncomfortable situation, I try to force myself through it. I don’t know if this is the right decision but I know I’m a bit scared so I know I will learn plenty whether my MBA related goals are met or not. Also, the financial risk is tolerable at UNCG.

Expectations

  • Develop a more complete understanding of business – I know plenty about IT and testing, but other aspects such as finance and marketing are mysteries to me
  • Gain experience working with people from different business areas and different cultural backgrounds
  • Influence the education and growth of others
  • Learn new ideas and new ways of thinking
  • More challenging and rewarding career options

Strength and weakness exercise

“What are your greatest strengths?” “What are your greatest weaknesses?” Two of the most common and dreaded interview questions. Most people think they know the answers to these questions, but I’m pretty sure most people are wrong; especially when it comes to their strengths. The things I’m not good at are fairly obvious, but discovering what I’m truly good has proven to be more difficult. I think this is because what I think are my strengths are what I want to be my strengths.

I’ve found that unbiased self-reflection is a difficult thing to do well, so in attempt to better identify my strengths and weaknesses, I’m going to try to document key decisions I make and what I want and expect to happen as a result of those decision. A year or so later the plan is to revisit these decisions and see if I was able to achieve the desired outcome.

I unknowingly started this exercise in March of 2012 when I decided to join uTest. I documented the reasons for making that decision as well as my goals and now, 18 months later what I expected to happen – develop a reputation as an expert tester – is starting to develop (at least within the uTest community).

I’m excited to see where this exercise takes me. To kick it off, my first key decision is to…. start keeping track of my key decisions :)

My First Testing Experience – Part 2

Continued from Part 1

I can vividly remember lying on my couch in my living room staring at the ceiling, my stomach in knots. I felt this huge weight on top of me making it hard to breath. I was pretty close to panicking. “How can I possibly do this? I have no idea what I’m doing. Why did I agree to this project? Do they have any idea what they’re asking? This is so unfair. This is CRAZY!” These were the thoughts racing through my mind. Before I could think one through, another would jump on the pile. That feeling of hopelessness, of being completely overwhelmed was my first and worst moment as a software tester.

I’d like to think the PM understood the magnitude of the project, that somehow she saw this amazing potential inside me just waiting for an opportunity to prove itself. But I knew she didn’t really understand what she was asking. In her mind this was a small task that any mid-level developer should be able to do. After all, it’s just testing, how hard can that be? That just added to the pressure. I was dealing with unrealistic expectations about an undefined project that required an under-appreciated amount of skill and effort.

After breathing into a paper bag for 10 minutes or so, I calmed down. I reminded myself the best way to climb a mountain is one step at a time. I started by making a spreadsheet documenting all the different migration tools we had built. I identified the base data type, the destination data type and a description of the transformation needed. One by one I went down the list, talking to the developers, searching for any documentation, looking at the old system, learning about the new one, basically doing the work a BA would have done.

Once I had a solid understanding of the specifications I was working with, I started testing. I started with the simplest cases and the ones that I was most familiar with. You are probably thinking I should have done the opposite and started with the hardest, most complicated and risky transformations. If I was doing it over today, that is the approach I would take, but considering I was a complete rookie tester, I needed to start with a small, achievable task to build some confidence and establish my testing strategy.

My strategy was pretty simple. I gathered sample source data, ran it though the migration and verified the results on the other side. Then I started adjusting the source data, trying every scenario I could think of looking for ways to make the tools fail. As I worked through my list, reporting bug after bug, it became clearer to everyone just how large this task was. Expectations slowly became more aligned with reality and the measurement for success became more reasonable.

After a few months of work, we were ready to start migrating content. It wasn’t perfect and several bugs made it through, but they were minor and due to unexpected bad source data. The migration ran for months as different sites came online. The migration tools did their job and by all accounts, the testing aspect of the project was successful.

This experience isn’t what pushed me into testing, but it was my first introduction. It gave me a new appreciation for what testing is and how valuable it can be. It gave me confidence in myself that I could accomplish a task that I really had no business attempting. It’s an experience that I can now look back on and be proud of.

Webinar – Should testers report every bug they find?

In December of 2012, Ryan Lamontagne and I got into a good discussion on the uTest forms about whether testers should report every bug they find. We decided to kick it up a notch and debate it live in a webinar!

http://forums.utest.com/viewtopic.php?f=13&t=4430