Category: Education

Quality is Customer Value – My Quest for the uTest MVT Award

One thing I respect about the uTest management is their continual pursuit of ways increase customer value. It’s an essential business objective to ensure the health and growth of our company. ‘Value’ should be the middle name of any good tester. “Lucas Value Dargis”. Sounds pretty cool huh?

I had just finished my 26th uTest test cycle. I had put an extra amount of focus and effort into this cycle because there was something special at stake. Occasional uTest offers a MVT award which is given to the most valuable tester of the cycle. The selection process takes several things into account including the quality of the bugs found, clear documentation, participation, and of course, customer value.

The MVT award not only offered a nice monetary prize, but it’s also a way to establish yourself as a top tester within the uTest community. I decided I was going to win that MVT award.

As usual, I started by defining my test strategy. I took the selection criteria and the projects scope and instructions into account and came out with these 4 strategic objectives:

  • Focus on the customer-defined ‘focus’ area
  • Report only high-value bugs
  • Report more bugs then anyone else
  • Write detailed, easy to understand bug reports
  • Be active on the project’s discussion board

When the test cycle was over I reflected on how well I’d done. I reported 9 bugs, more then anyone else in the cycle. Of those, 8 were bugs in the customer’s ‘focus’ area. The same 8 were also rated as very or extremely valuable. All the bugs were documented beautifully and I was an active participant in the discussion board.

There was no competition. No one other tester was even close. I had that MVP award in the bag. I was thinking of all the baseball cards I could buy with the extra Cheddar I’d won. I even called my mom to tell her how awesome her son was! You’ll can only imagine my surprise when the announcement was made that someone else had won the MVT award. Clearly there was some mistake right? That’s not how you spell my name!

I emailed the project manager asking for an explanation for this miscarriage of justice. The tester who won had fewer bugs, none of them were from the ‘focus’ area and they weren’t documented particularly well. How could that possibly be worth the MVT award? The PM tactfully explained that while I had done well in the cycle, the tester who won had found the 2 most valuable bugs and the customer deemed them worth the MVT award.

I was reminded that my adopted definition of quality is “Value to someone who matters” and suddenly it all fell into place. It didn’t matter how valuable I thought my bugs and reports were. It didn’t matter how much thought and effort I put into my strategy and work. At the end of the day a testers goal, his or her mission, should be to provide “someone who maters“ with the most value possible. I’m not that “someone who matters”. That “someone” is our customer.

It was a hard pill to swallow, but that lesson had a strong impact on me and it will be something I’ll carry with me moving forward. Congratulations to the MVT, I hope you enjoy all those baseball cards.

uTest Review From a New uTester

Back in March of 2012, I was listening to a podcast from SoftwareTestPro.com and they were talking about this thing called uTest. I decided to check it out.

What is uTest?

My first impression was “oh great, this is just another rent-a-tester website”, but as I looked a little closer I realized that this this company was much more powerful then I thought. uTest takes advantage of crowdsourcing (or in-the-wild testing as they call it) by distributing the work across people all over the world. In a matter of hours a website or app can be tested by dozens of different device configurations by real world users as well as professional testers.

Companies who have something they need tested hire uTest and their army of testers to perform the testing tasks. Testers are paid per bug found and per test case wrote or executed. The more valuable the customer finds the bug or test case, the more money the tester is paid.

This provides tremendous value for both companies and testers:

Companies get their products tested much quicker and cheaper then they could with in-house testers. They can have 100 testers working for one week vs. having 5 tester test for 20 weeks. Plus, they only pay for work they find valuable.

Testers get the opportunity to work on many different types of testing tasks, projects, environments and basically get exposed to things not available in their daily job. Getting paid is nice too.

Obviously there are downsides to this testing approach. For example, uTest testers don’t know or have time to learn your business or all of the intricacies of your product, so uTest shouldn’t be used when a deep level of understanding is needed. However, there are many projects where just having a hundred different eyes on on the product is exactly what is needed.

My First Test Cycle

The first project (called a test cycle) I worked on was an unpaid “interview” sandbox test cycle. This is an opportunity to try out to become a uTest tester. The task was to test NBA.com. There really wasn’t much in the way of requirements or instruction, except that you could only report 4 bugs and they had to be within the NBA.com domain. We would be evaluated on the value of our bugs and our ability to follow directions.

Since this was an audition, and I was limited to only 4 bugs I knew that each bug was very important for my evaluation. It’s hard to judge someone’s ability based on just 4 bugs so I needed to make their job easier. uTest rates bugs on a value scale: somewhat, very, and exceptionally. I wanted to be the top ranked tester in the cycle so in order to do that I thought about the assignment, the constraints, and the customer and came up with a plan.

My plan was to find different types of bugs in different locations within the site that would show my technical knowledge and testing expertise. Solid communication skills are a must in an online-based work environment so I’d need to make sure my bug reports where clear and descriptive.

Through my testing, I found dozens of bugs and I could have easily just logged the first 4 I found and been done with it. But I was very careful to make sure that I only reported those that fit into my strategy and represented my abilities as accurately as possible.

My value as a tester can be found in my strategy; how I went about finding and reporting bugs to accomplish my goal. All testing should be driven by a mission or it serves no purpose. In this case my mission was to be the highest rated tester so all my testing was done with that in mind. The actual work I did was really just “operational effectiveness” and should be expected for any good tester.

Results

  • I received the highest rating out of all 107 testers in the cycle.
  • I was the only tester who had all 4 bugs bug reports rated higher then ‘somewhat valuable’ ( I had the lone ‘exceptionally valuable’ bug report)
  • My overall uTest rank was higher then 33% of the 55,000 uTesters (Not too bad for only having 4 bug reports submitted)
Overall, I’d say my strategy worked out nicely. I don’t point this out to brag (well, not entirely), but to show the importance of a mission and a formulating a strategy to accomplish it.

Lessons Learned

From a customer’s point of view the power of crowdsourcing is amazing. The shear volume of bugs we uncovered in just a few days shocked me, especially considering how heavily trafficked that site is.

For testers, the opportunity to see the work of 100 other testers is priceless. There were several very solid testers in the cycle who had fantastic bug reports. I was able to glean some useful knowledge and techniques from reading through their reports.

The tester talent pool is sadly shallow. It surprised me just how clueless the majority of the testers were. Of the 219 bugs that were reported, 71 were rejected because the tester didn’t follow the simple instructions. It was obvious that the majority were clicking around aimlessly and reporting anything they could find.

The gap between a good tester and a bad tester is wider then I thought. Thankfully uTest takes this into account. Only testers ranked in the top 30% of the sandbox test cycle are allowed to join a paid test cycle. This allows uTest to filter out the majority of the poor testers. Also, lower ranked paid testers are limited in the amount and complexity of projects they can participate in while highly ranked testers are offered many more test cycles. This helps ensure projects have the best possible testers, keeps poor testers out, and gives new testers the opportunity to prove themselves.

I’m a now a raving fan of this company and crowd sourcing (or team sourcing as uTest calls it)!

Learn more about uTest at utest.com.
You can also check out my uTest profile here.

5 Qualities of Horrible Bosses

I was reading through this article on the 5 Qualities of Remarkable Bosses and thought it would be fun to list 5 qualities of horrible bosses. If you took each of these qualities and do the opposite, you would be well on your way to becoming a remarkable boss.

Keep communication to a minimum

Keep your eployees in the dark. They don’t need to know the direction of the group or company. This also enforces that you are more important then they are. Plus anything they need to know they’ll hear through the grapevine anyway so why bother?

You don’t want your employees to get a big head and think they are important so don’t ever show interest in their work. You should, however micromanage their work because they’re doing it all wrong.

Be reactive

Planning ahead is the worst thing you can do. You should only react to problems and needs as they arise. There are several of old sayings that prove this:

  • The squeaky wheel gets the grease
  • Let tomorrow worry about itself

This keeps everyone on their toes because they’ll never know whats coming up next. It also makes you look like the hero because you solved all these problems. Why on earth would you want to solve a problem if no one will ever know that you solved it?

Keep employee turnover rate high

It’s best to constantly have new blood in your group. You don’t want your employees to ever feel secure in their job because people do better work when they know their time is limited. Make sure there is a toxic work environment and  ignore team morale (or actively lower it) to encourage voluntarily turnover. Don’t worry about the costs associates with training new employees, they won’t be around long enough to be fully trained anyway.

Keep all power to yourself

You are the boss because you know the most.  You need to be involved in everything. Every meeting, every email, every decision. Don’t ever let one of your team members to make a decision or lead a project. This makes you look weak and incapable plus you can’t trust any of them. Occasionally make an obviously wrong decision and stick with it no matter what, just to remind everyone who’s boss.

Sanction Incompetence

What better way to keep your power intact then to ensure you have incompetent employees. Don’t ever send your employees to training or give them projects that would increase their skill set. Hiring low-end employees is cheap and they are great scapegoats to blame when things don’t go well so always keep a few on hand.

What are some of the qualities of horrible bosses you’ve seen?

Strategies for a Testing Career – Part 1

I am reading through a book of articles titled “On Strategy”. It is published by the Harvard Business Review and the context deals with business strategy. What I have found is that even though the intended audience is business managers, the lessons can easily be applied to individual testers. As I read and study each article, I will write a post summarizing the content of the article and how testers can apply the lessons to develop their own strategy.

What is Strategy?

by Michael E. Porter

Operational Effectiveness Is Not Strategy

Operational Effectiveness (OE) means performing activities better, faster and with fewer inputs and defects then your rivals. The quest for productivity, quality, and speed has spawned a remarkable number of management tools and techniques. Enormous advantages can be gained from OE, but from a competitive standpoint, the problem with OE is that the best practices that drive it are easily emulated. When competitors both strive to achieve the best possible OE, the competition between them produces absolute improvement in OE (everyone is more effective), but relative improvement for no one and the more indistinguishable the competitors become. The pursuit of OE is not a bad thing, in fact it is a necessary part of strategy, but OE itself is not a strategy

As testers, we too pursue common tools, techniques, and procedures to do our job more effectively. Quality Center, Test Manager, test plans, test cases, bug tracking, and defect metric are just a few examples of the many ways the testing industry has enable testers to chase OE. As all testers attempt to become that sought-after tester they too fall into the trap of trying to become that one ideal tester. We all are want to become QC experts, and automation specialists, but as we all pursue that goal, we all become more and more similar which is contrary to our goal of standing out from the crowd.

So what is strategy? How do I as a tester establish my strategic position?

Strategy Rests on Unique Activities

Strategy is all about being different from your competition. Our competition as testers would be anyone who has the same goal as we do. For example all other testers that are applying for that job we want or the opinionated blogger who has the ear of the testing community that we envy.

Strategic positioning means performing different activities from rivals, or performing similar activities in different ways.

A perfect example of doing different activities is Jame Bach. Almost everything he does and teaches is exactly the opposite of what most “factory” testers do. While your average tester is pursuing certifications and following test cases, James supports open learning and exploratory testing. As a result, he is one of the most well know testers in the world. No doubt he is an amazing tester, but the reason he stands out is because he is so different from everyone else. He’s loud, sarcastic, opinionated (none of this is meant as a slight by the way) and people listen.

Almost every tester out there is experienced in Quality Center. Knowing how to write test cases or track bugs is expected and it doesn’t set you apart, but there are ways to do those similar activities in different ways. Consider learning about lesser know bug tracking software such as Kiln, or possibly create something totally new yourself. Try to find a new way to perform exploratory testing within an Agile testing environment.

A Sustainable Strategic Position Requires Trade-offs

A strategic position is not sustainable unless there are trade-offs. Trade-offs occur when activities are incompatible. For example, if my strategy is to be a white-box manual tester, then I should ignore learning about Selenium automation and focus on manual testing and coding skills. Pursuing both would split my available time and effort effectively making me a lesser white-box tester. Trade-offs mean more of one thing necessitates less of another.

Sustainable means that over time, your strategic position will erode if you try to do everything. You may be able to get by in the short term, but as you spend more and more time on non-strategic activities, your core skills will suffer and it will become apparent as others (who made the trade-offs) surpass you.

Fit Drives Both Competitive Advantage and Sustainability

Strategy is all about combining activities to give you that unique strategic position. Fit describes how those activities tie into each other. One activity’s cost can be lowered by the way other activities are performed. Similarly, an activity’s value to your strategy can be increases by effectively “fitting” it with other activities.

I want to become known as an expert tester. My strategy to accomplish that, in part, includes becoming knowledgeable in my field and sharing that knowledge with others. Learning and blogging “fit” because they both contribute directly to my goal, and each complements and improves the other. Learning give me something to blog about, and blogging helps me to better remember and reflect upon what I’ve learned.

Conversely, we need to be aware activities that don’t “fit” our strategy. If I decide that I want to be a context-based exploratory tester, will getting a certification from ISTQB contribute to that goal? does it “fit” with the other activities of my strategy?

Rediscovering Strategy

There are two distinct views on strategy. One is the idea of competing head-to-head with your rivals on the same things, the other is establishing a unique strategic position that provides a competitive advantage. Here are some characteristics of both views.

Head-to-Head

  • One ideal competitive position in the industry
  • Benchmarking activities to achieve best practices
  • Outsourcing and partnering to improve efficiencies
  • Focus on a few success factors, critical resources, and core competencies
  • Flexibility and rapid responses to competitive and market changes

Unique Strategic Position

  • Activities tailored to the strategy
  • Trade-offs focus activities
  • Fit across activities
  • Focus on the entire system, not individual parts
  • Operational effectiveness is a given

There are two points I took away from this article that I think are extremely valuable. First, we as testers need a strategy. Strategy isn’t something that should be left to managers of  big companies, it can and should be applied to individuals. And second, imitating what others are doing is not a strategy, it’s an anti-strategy. The only way to give ourselves an advantage over our competition is to have a unique strategic position based on trade-offs and carefully aligned activities.