Tag: Testing

2012 Year In Review

2012 was a career year for me. For the past 8 years I’ve just had a job. I didn’t really enjoy what I was doing and didn’t put much thought into how I could or should develop my career. Things changed quickly early in the year as several opportunities came together. Here are a few of those highlights:

New Jobs
I found the best job I’ve ever had, working as the principle tester at a semiconductor manufacturer. Before I arrived, there was no formal testing in the IT department. I was tasked with introducing testing in one group and then over time, grow it throughout the organization. I’ve been able to test the new “Flagship” application which is a few weeks away from our first Release. So far we have received rave reviews on all aspects of the application and the development process.

I was able to expand my testing skills by learning the nuances of SPA (single page application) testing. This has been a fun and challenging experience, mostly because it isn’t done much yet so there few resources out there geared specifically toward SPA testing.

I was also able to dabble in automation testing for SPAs. Since this type of application is client-side heavy, the true value comes from exercising it through the browser. Many automation solutions and supporters prefer testing the code directly (via APIs or a test harnesses), bypassing the browser. That has made this learning process more of a struggle for me then I had expected.

If you follow my blog at all you’ll know I’m also a uTest fan boy and freelance tester. I’ve already wrote a blog post about my uTest experiences this year, so I’ll just give an updated summary:

  • Team Test Lead
  • Became a solid iOS tester
  • Worked with and learned from testers all over the world
  • 200 cycles/425 bugs
  • 94% bug approval rate/44% high-value rate
  • Gold Rated (99.75%)

Improved Online Presence
One of the most valuable and educational aspects of this year was my decision to join and contribute to the testing community. I started this blog to chronicle my career development. I only found the time for 11 posts but I was able to post regularly… well kind of.

I spent most of my time focused on the uTest community. I became a uMentor, a forum moderator and one of the most active forum members. My topics have generated hundreds of responses and over 20,000 views. I’m now seeing more and more new uTesters step up and contribute to the growth of the forum and the uTest community which is fantastic!

I was also featured on the uTest blog a few times:

I have learned so much from writing about testing, teaching new testers, and learning from others. I’d say that focusing on developing my online presence has had the largest impact in my growth as a professional tester.

Scrum Mastery
In addition to testing, I’m also extremely interested in software development processes and improving efficiency, specifically the Scrum development framework. Since I had a few years of Scrum experience, I volunteered to be a Scrum Master for the “flagship” product I mentioned above. As word of that project’s success got around, I became a champion for Scrum in our organization. I was able to coach POs, Developers, Customers, and Managers and am currently Scrum Mastering 2 projects. I was asked to give an “Introduction to Scrum” presentation to our department during one of our Lunch & Learn sessions and since then one group has started their own project using Scrum.

To complement and improve my real-world experiences, I attended a Scrum Master course and then passed both the Scrum Alliance and Scrum.org Scrum Master certifications.

So cheers 2012; you’ve been swell. I look forward to meeting you 2013. I know you have many fun and challenging experiences in store.

Testing buzz words that annoy me

Maybe I’m just too sensitive (I have been watching a lot of romantic comedies lately), but there are certain “testing” words that really bother me. Either they are way overused or they are used incorrectly or whatever. I just felt I needed to vent :)

So, here are my top 5 testing words (phrases) that annoy the heck out of me:

It depends -When does it ever NOT depend? how is this helpful? Would you just take a stance for crying out loud?

QA -This is used way to often and it is usually used incorrectly. QA stands for Quality Assurance. Quality Assurance is process oriented and focuses on defect prevention. It is a term typically used in manufacturing. We are in the defect identification and information providing business. We are not QA people, we don’t do QA. We are testers, we test!

Sapient – Yeah, I get it. You used a smart sounding word to describe testing activities so that testers sound smart. Good one. Go away….

Heuristics – I’m still recovering from how impressed I was by sapient.

Craft – I really don’t know why this one bothers me so much but I absolutely CRINGE anytime I read or hear it. Nails on a chalkboard for me.

So, what testing buzz words annoy you?

Disclaimer:
This is intended to be a fun rant. If you use these words and I’ve offended you, my reaction would depend on several factors. Maybe you should QA this post relying on your sapient abilities. Following a heuristic, take the opportunity to hone your craft. AHHHHGGGG!!!

STPCon Fall 2012 – My Review

I recently was able to attend STPCon in Miami. It was my first conference and I really enjoyed it. Here are some of my highlights.

People I met

I finally got to meet a few members of the uTest staff. Jessica, Matt, Mike, and Chris were all there slinging swag and preaching the uTest gospel. It was nice to get some face time with those folks. Unfortunately we didn’t have a uTest meet up. Maybe next time.

I did get to meet a former uTester, Joseph Ours. I talked with him for a while about his experiences with uTest and how he has made the transition into management and consulting. Joe is a knowledgeable and well-spoken guy. I really enjoyed getting to know him. Too bad he’s not an active uTester anymore, I would love the chance to work with him.

Hands-on vs. Lecture format

There was a track of 7 classes that focused on hands-on practicals. It seemed like a cool idea to me, so I spent the first 3 sessions in hands-on classes. While they were interesting and it was fun to test with other testers, I didn’t really learn anything. It was more like “Here’s a program, lets think of ways to test it”. I do that every day when I test with uTest. I wasn’t there to test, I was there to learn how to become a better tester.

While I can see the value of the hands-on classes for people who don’t have the luxury of contently testing new things with new people, for me it wasn’t the best use of my limited and expensive time. (I was there on my own dime)

After I realized that, I switched over to the lecture/discussion sessions and really found a lot of value there. Most of those sessions weren’t lectures but more of someone sharing their experience or suggesting ideas on how to do things better. The dialog between the presenters and the audience was also great. There was lots of idea sharing and discussions.

Favorite Sessions

Mike Lyles of Lowes gave a two part talk about how Lowes created a Test Center of Excellence (TCoE). It was extremely useful for me to see what a mature testing organization looks like and how it functions. It helped me start to develop my long term vision for my testing organization.

I also enjoyed Joseph Ours’s talk about “Redefining the purpose of software testing”. Simply put, he was making the argument that testing teams should be looked at as information providers, not gate keepers or decision makers. I have been making this argument for a while now but had been looking at it the wrong way. I thought it just was the definition of what testers should do. His explanation showed me that it actually can be an effective way to explain the value that testing provides. Testing is a service. We provide information to make informed decisions.

One lady in the audience challenged Joe saying that she is the test lead as well as the gate keeper and that in her organization it works fine. Joe and her debated a bit but she wasn’t convinced. After the presentation Joe and I chatted about that point more and realized that we need to think of roles and people separately. A testers role is to discover and provide information. A decision makers role is to make decisions based, in part, on that information. Usually those role are divided between two different people (or groups of people) but in some cases, it may be the same person.

In her case, she led the testing team, but she also had additional business knowledge and the authority necessary to be able to decide when the product was fit for delivery.

Not so great

The Vendor showcase was disappointing. There were only 10 or so booths. I talked to most of them in under an hour and spent the rest of the time trolling around the uTest booth sharing my uTest testimony with anyone who would listen 🙂

Overall

I’m glad I was able to make it down to STPCon. I got to meet some great people, learn a few things, and get a better understanding of our industry in general. It was a valuable experience and am looking forward to attending next year. Maybe I’ll see you there.

uTest – My First 100 Cycles

So I’m a little late, I’m actually at 138 cycles, but I wanted to give an update on my uTest experience now that I’ve got 100 cycles under my belt.

Accomplishments

When I first signed up with uTest I set a few goals. I really had no idea how realistic they were, but you have to at least have something to shoot for right?

By the end of 2012 (9 months from when I started) I wanted to:

  1. Earn my gold badge in Functional testing
  2. Become a TTL (Test Team Lead)
  3. Develop a strong reputation within the uTest community

Gold badge

I got my functional badge within 30 days of my first test cycle. At first this was actually a disappointment. I was really looking forward to the challenge of having to work hard for that badge.

After the initial shock wore off I decided that it actually was an accomplishment to be proud of. I realized that uTest has created an efficient system that allows strong testers to bubble up to the top quickly which makes sense when you think about it. Why would they want to hold good testers back? You want your best testers out front setting an example and representing the company.

TTL

Three months in, I got the email I had been anxiously waiting for; I had been invited to be a TTL! A rite of passage to become a TTL is you have to manage a sandbox class. Working that cycle was the most difficult thing I’ve done at uTest so far. Working with and evaluating 100 rookie testers, triaging their 400+ bugs and test cases in just a week is grueling to say the least. But the sense of accomplishment at the end made it all worth it. Since then, it’s been a fantastic experience working closely with the PMs while helping guide the testers so they can develop and become successful.

Reputation

My approach to this goal was to focus on the uTest form. The forum is an amazing community full of talented testers from all over the world. Every day there is are several interesting conversations going on. We constantly learn new things while challenging and encouraging each other.

After a few months of active participation, I was asked to become a uMentor and a forum moderator. Basically my role is to spark discussions and debates as well as write educational “mentoring” posts.

Before I started working at uTest maybe 10 other people knew I was a tester (That includes my mother). Now literally hundreds of people read my posts and engage in awesome discussions with me every week. So far, the conversations I’ve started have generated over 13,000 views.

I’m not sure I’ve fully achieved this goal yet, but I’m on my way.

Testing Statistics

To quantify my time at uTest so far and to brag a little 🙂 here are my digits:

  • Products Tested: 70
  • Test Cycles: 138
  • Bugs Filed: 342
  • Bug Acceptance: 93.2%
  • High Value Bugs: 47%
  • Functional Rating: 99.3% – 99.7%

You can see my uTest profile here: https://my.utest.com/platform/profile/LucasDargis

uTest Perception

Without a doubt, joining uTest has been the biggest factor in the development of my career. I have grown more in the last 7 months then I ever expected. I was able to achieve all my goals and then some.

Here is a short list of the amazing opportunities uTest provides testers:

  • The opportunity to work for every type of company. You are able to see how testing is done from start-ups to big Fortune 100 companies. Every cycle adds to your experience and helps open your mind to the many different ways testing is approached.
  • The opportunity to work with and learn from people from all over the world. I’ve made friends in the UK, India, Romania, Brazil, etc. The global exposure is priceless.
  • The uTest forum is relatively new and the number of active contributors is still small, but since the uTest community is so large (60,000+ testers) the potential is huge.  There is an excellent opportunity for testers to get in on the ground floor and quickly establish themselves in a community that is evolving into an industry leader.
  • There are several leadership opportunities. You can become a TTL, a forum moderator, a uMentor, and a Gold-rated tester. Being active in discussions on the form or in your test cycle (providing help, answering questions) are informal ways to develop your leadership skills.

You may have noticed that I haven’t talked about money at all. That is because for me, getting paid at uTest is an extra bonus. I can always make money, but it’s these other aspects that provide the most value to me as a tester.

100 cycles down. Next stop, 1000 bugs!

Quality is Customer Value – My Quest for the uTest MVT Award

One thing I respect about the uTest management is their continual pursuit of ways increase customer value. It’s an essential business objective to ensure the health and growth of our company. ‘Value’ should be the middle name of any good tester. “Lucas Value Dargis”. Sounds pretty cool huh?

I had just finished my 26th uTest test cycle. I had put an extra amount of focus and effort into this cycle because there was something special at stake. Occasional uTest offers a MVT award which is given to the most valuable tester of the cycle. The selection process takes several things into account including the quality of the bugs found, clear documentation, participation, and of course, customer value.

The MVT award not only offered a nice monetary prize, but it’s also a way to establish yourself as a top tester within the uTest community. I decided I was going to win that MVT award.

As usual, I started by defining my test strategy. I took the selection criteria and the projects scope and instructions into account and came out with these 4 strategic objectives:

  • Focus on the customer-defined ‘focus’ area
  • Report only high-value bugs
  • Report more bugs then anyone else
  • Write detailed, easy to understand bug reports
  • Be active on the project’s discussion board

When the test cycle was over I reflected on how well I’d done. I reported 9 bugs, more then anyone else in the cycle. Of those, 8 were bugs in the customer’s ‘focus’ area. The same 8 were also rated as very or extremely valuable. All the bugs were documented beautifully and I was an active participant in the discussion board.

There was no competition. No one other tester was even close. I had that MVP award in the bag. I was thinking of all the baseball cards I could buy with the extra Cheddar I’d won. I even called my mom to tell her how awesome her son was! You’ll can only imagine my surprise when the announcement was made that someone else had won the MVT award. Clearly there was some mistake right? That’s not how you spell my name!

I emailed the project manager asking for an explanation for this miscarriage of justice. The tester who won had fewer bugs, none of them were from the ‘focus’ area and they weren’t documented particularly well. How could that possibly be worth the MVT award? The PM tactfully explained that while I had done well in the cycle, the tester who won had found the 2 most valuable bugs and the customer deemed them worth the MVT award.

I was reminded that my adopted definition of quality is “Value to someone who matters” and suddenly it all fell into place. It didn’t matter how valuable I thought my bugs and reports were. It didn’t matter how much thought and effort I put into my strategy and work. At the end of the day a testers goal, his or her mission, should be to provide “someone who maters“ with the most value possible. I’m not that “someone who matters”. That “someone” is our customer.

It was a hard pill to swallow, but that lesson had a strong impact on me and it will be something I’ll carry with me moving forward. Congratulations to the MVT, I hope you enjoy all those baseball cards.

uTest Review From a New uTester

Back in March of 2012, I was listening to a podcast from SoftwareTestPro.com and they were talking about this thing called uTest. I decided to check it out.

What is uTest?

My first impression was “oh great, this is just another rent-a-tester website”, but as I looked a little closer I realized that this this company was much more powerful then I thought. uTest takes advantage of crowdsourcing (or in-the-wild testing as they call it) by distributing the work across people all over the world. In a matter of hours a website or app can be tested by dozens of different device configurations by real world users as well as professional testers.

Companies who have something they need tested hire uTest and their army of testers to perform the testing tasks. Testers are paid per bug found and per test case wrote or executed. The more valuable the customer finds the bug or test case, the more money the tester is paid.

This provides tremendous value for both companies and testers:

Companies get their products tested much quicker and cheaper then they could with in-house testers. They can have 100 testers working for one week vs. having 5 tester test for 20 weeks. Plus, they only pay for work they find valuable.

Testers get the opportunity to work on many different types of testing tasks, projects, environments and basically get exposed to things not available in their daily job. Getting paid is nice too.

Obviously there are downsides to this testing approach. For example, uTest testers don’t know or have time to learn your business or all of the intricacies of your product, so uTest shouldn’t be used when a deep level of understanding is needed. However, there are many projects where just having a hundred different eyes on on the product is exactly what is needed.

My First Test Cycle

The first project (called a test cycle) I worked on was an unpaid “interview” sandbox test cycle. This is an opportunity to try out to become a uTest tester. The task was to test NBA.com. There really wasn’t much in the way of requirements or instruction, except that you could only report 4 bugs and they had to be within the NBA.com domain. We would be evaluated on the value of our bugs and our ability to follow directions.

Since this was an audition, and I was limited to only 4 bugs I knew that each bug was very important for my evaluation. It’s hard to judge someone’s ability based on just 4 bugs so I needed to make their job easier. uTest rates bugs on a value scale: somewhat, very, and exceptionally. I wanted to be the top ranked tester in the cycle so in order to do that I thought about the assignment, the constraints, and the customer and came up with a plan.

My plan was to find different types of bugs in different locations within the site that would show my technical knowledge and testing expertise. Solid communication skills are a must in an online-based work environment so I’d need to make sure my bug reports where clear and descriptive.

Through my testing, I found dozens of bugs and I could have easily just logged the first 4 I found and been done with it. But I was very careful to make sure that I only reported those that fit into my strategy and represented my abilities as accurately as possible.

My value as a tester can be found in my strategy; how I went about finding and reporting bugs to accomplish my goal. All testing should be driven by a mission or it serves no purpose. In this case my mission was to be the highest rated tester so all my testing was done with that in mind. The actual work I did was really just “operational effectiveness” and should be expected for any good tester.

Results

  • I received the highest rating out of all 107 testers in the cycle.
  • I was the only tester who had all 4 bugs bug reports rated higher then ‘somewhat valuable’ ( I had the lone ‘exceptionally valuable’ bug report)
  • My overall uTest rank was higher then 33% of the 55,000 uTesters (Not too bad for only having 4 bug reports submitted)
Overall, I’d say my strategy worked out nicely. I don’t point this out to brag (well, not entirely), but to show the importance of a mission and a formulating a strategy to accomplish it.

Lessons Learned

From a customer’s point of view the power of crowdsourcing is amazing. The shear volume of bugs we uncovered in just a few days shocked me, especially considering how heavily trafficked that site is.

For testers, the opportunity to see the work of 100 other testers is priceless. There were several very solid testers in the cycle who had fantastic bug reports. I was able to glean some useful knowledge and techniques from reading through their reports.

The tester talent pool is sadly shallow. It surprised me just how clueless the majority of the testers were. Of the 219 bugs that were reported, 71 were rejected because the tester didn’t follow the simple instructions. It was obvious that the majority were clicking around aimlessly and reporting anything they could find.

The gap between a good tester and a bad tester is wider then I thought. Thankfully uTest takes this into account. Only testers ranked in the top 30% of the sandbox test cycle are allowed to join a paid test cycle. This allows uTest to filter out the majority of the poor testers. Also, lower ranked paid testers are limited in the amount and complexity of projects they can participate in while highly ranked testers are offered many more test cycles. This helps ensure projects have the best possible testers, keeps poor testers out, and gives new testers the opportunity to prove themselves.

I’m a now a raving fan of this company and crowd sourcing (or team sourcing as uTest calls it)!

Learn more about uTest at utest.com.
You can also check out my uTest profile here.

My First Testing Experience – Part 1

I thought it would be fun to relive the event that really pushed me into testing. The first 5 years of my career were spent doing business analysis and development work. Apart from unit testing my own code, I had no testing experience or expertise. I was an average mid-level developer doing his thing when out of nowhere, I fell into the world of testing. This is Part 1 of that story.

My company was in the process of upgrading/migrating their web platform to SharePoint 2007. The project was broken into two teams; the platform team, and the migration team. The platform team was tasked with re-writing the entire platform (internet, intranet, and extranet sites) while the migration team was responsible for writing throwaway programs that would transform and move the company’s data from the old systems to the new one. I was a developer on the migration team.

The platform team had a typical setup. There were stakeholders, a product owner, a BA, requirements, developers, and several testers. The migration team however, was simply a handful of developers. Our job was to make sure the data was taken from the old system, transformed, and then migrated to the new system.

Since the old system wasn’t documented very well and the new system was still being developed we had little information to guide us. We had to dig into the old system’s data then talk to the developers to see what their new code expected then go write a tool to do the work. As the platform matured and changed, our migration tools had to evolve to accommodate those changes.

Many months went by and the platform and migration tools started to take shape. Every component on the platform had a specific tool, or part of a tool, that would handle its migration. Not surprisingly, the migration project grew very fast to the point where, in terms of lines of code, it was actually larger than the platform.

At this point there was a test manager and 5 testers working on the platform but no one was testing the migration tools. One day the project manager came to me and said:

“We are almost ready to start migrating data, but we need to make sure the migration tools work correctly. I’m gonna need you to go ahead and test them OK?”

Yeah, that was me (except I’m not a baby and my head isn’t shaped like a football).

“Let me get this straight, you want me, someone with no testing training or experience, to test an incredibly large and complex set of tools, without any documented requirements, by myself, all while the platform project, which is similar in size and complexity has had 5 people testing it for the past 6 months?”

“Yup”

“Wow…. Well, OK”

In part 2 panic sets in.

Strategies for a Testing Career – Part 1

I am reading through a book of articles titled “On Strategy”. It is published by the Harvard Business Review and the context deals with business strategy. What I have found is that even though the intended audience is business managers, the lessons can easily be applied to individual testers. As I read and study each article, I will write a post summarizing the content of the article and how testers can apply the lessons to develop their own strategy.

What is Strategy?

by Michael E. Porter

Operational Effectiveness Is Not Strategy

Operational Effectiveness (OE) means performing activities better, faster and with fewer inputs and defects then your rivals. The quest for productivity, quality, and speed has spawned a remarkable number of management tools and techniques. Enormous advantages can be gained from OE, but from a competitive standpoint, the problem with OE is that the best practices that drive it are easily emulated. When competitors both strive to achieve the best possible OE, the competition between them produces absolute improvement in OE (everyone is more effective), but relative improvement for no one and the more indistinguishable the competitors become. The pursuit of OE is not a bad thing, in fact it is a necessary part of strategy, but OE itself is not a strategy

As testers, we too pursue common tools, techniques, and procedures to do our job more effectively. Quality Center, Test Manager, test plans, test cases, bug tracking, and defect metric are just a few examples of the many ways the testing industry has enable testers to chase OE. As all testers attempt to become that sought-after tester they too fall into the trap of trying to become that one ideal tester. We all are want to become QC experts, and automation specialists, but as we all pursue that goal, we all become more and more similar which is contrary to our goal of standing out from the crowd.

So what is strategy? How do I as a tester establish my strategic position?

Strategy Rests on Unique Activities

Strategy is all about being different from your competition. Our competition as testers would be anyone who has the same goal as we do. For example all other testers that are applying for that job we want or the opinionated blogger who has the ear of the testing community that we envy.

Strategic positioning means performing different activities from rivals, or performing similar activities in different ways.

A perfect example of doing different activities is Jame Bach. Almost everything he does and teaches is exactly the opposite of what most “factory” testers do. While your average tester is pursuing certifications and following test cases, James supports open learning and exploratory testing. As a result, he is one of the most well know testers in the world. No doubt he is an amazing tester, but the reason he stands out is because he is so different from everyone else. He’s loud, sarcastic, opinionated (none of this is meant as a slight by the way) and people listen.

Almost every tester out there is experienced in Quality Center. Knowing how to write test cases or track bugs is expected and it doesn’t set you apart, but there are ways to do those similar activities in different ways. Consider learning about lesser know bug tracking software such as Kiln, or possibly create something totally new yourself. Try to find a new way to perform exploratory testing within an Agile testing environment.

A Sustainable Strategic Position Requires Trade-offs

A strategic position is not sustainable unless there are trade-offs. Trade-offs occur when activities are incompatible. For example, if my strategy is to be a white-box manual tester, then I should ignore learning about Selenium automation and focus on manual testing and coding skills. Pursuing both would split my available time and effort effectively making me a lesser white-box tester. Trade-offs mean more of one thing necessitates less of another.

Sustainable means that over time, your strategic position will erode if you try to do everything. You may be able to get by in the short term, but as you spend more and more time on non-strategic activities, your core skills will suffer and it will become apparent as others (who made the trade-offs) surpass you.

Fit Drives Both Competitive Advantage and Sustainability

Strategy is all about combining activities to give you that unique strategic position. Fit describes how those activities tie into each other. One activity’s cost can be lowered by the way other activities are performed. Similarly, an activity’s value to your strategy can be increases by effectively “fitting” it with other activities.

I want to become known as an expert tester. My strategy to accomplish that, in part, includes becoming knowledgeable in my field and sharing that knowledge with others. Learning and blogging “fit” because they both contribute directly to my goal, and each complements and improves the other. Learning give me something to blog about, and blogging helps me to better remember and reflect upon what I’ve learned.

Conversely, we need to be aware activities that don’t “fit” our strategy. If I decide that I want to be a context-based exploratory tester, will getting a certification from ISTQB contribute to that goal? does it “fit” with the other activities of my strategy?

Rediscovering Strategy

There are two distinct views on strategy. One is the idea of competing head-to-head with your rivals on the same things, the other is establishing a unique strategic position that provides a competitive advantage. Here are some characteristics of both views.

Head-to-Head

  • One ideal competitive position in the industry
  • Benchmarking activities to achieve best practices
  • Outsourcing and partnering to improve efficiencies
  • Focus on a few success factors, critical resources, and core competencies
  • Flexibility and rapid responses to competitive and market changes

Unique Strategic Position

  • Activities tailored to the strategy
  • Trade-offs focus activities
  • Fit across activities
  • Focus on the entire system, not individual parts
  • Operational effectiveness is a given

There are two points I took away from this article that I think are extremely valuable. First, we as testers need a strategy. Strategy isn’t something that should be left to managers of  big companies, it can and should be applied to individuals. And second, imitating what others are doing is not a strategy, it’s an anti-strategy. The only way to give ourselves an advantage over our competition is to have a unique strategic position based on trade-offs and carefully aligned activities.