There are some reviews out there from testers claiming uTest is a scam and that you can’t make any money. I’ve also seen a few uTest customers complain about the quality of the testing and uTest’s sales/negotiating practices. These concerns are valid and I can understand why some people have the impression that uTest is not what it claims to be. In order to address these perceptions, we need to look at them from two points of view; The view of the customer, and the view of the tester.
The uTest customer
Complaint #1: The quality of the testing was lower than expected
I have no problem saying that there are a lot of bad testers at uTest. It’s true, there is no point in denying it. These testers only report low-value bugs, they don’t follow instructions, and generally disrupt the test cycle. Sometimes it is because they’re inexperienced testers, sometimes they’re just plain bad. Unfortunately this is one of the downsides of crowdsourcing. To uTest’s credit, they do realize this and are continuously working to improve the skills and abilities of uTesters. They also identify and remove problem testers.
On the other hand there are some absolutely awesome testers at uTest. These top testers consistently provide the customer with excellent service and high-value bugs. uTest does a pretty good job of identifying the strong testers and ensuring that they are the ones working on your projects. Keep in mind that there are literally hundreds of projects, dozens of Project Managers and thousands of testers from all over the world, so every test cycle is going to be different.
So what’s a customer to do? First you need to manage your expectations. Understand the limitations and benefits of uTest and make sure they align with your testing needs. Second, you need to be involved. Yes, uTest is a service, but the quality and success of your project is a direct result of your participation and influence.
Here is an excellent article from Elena Houser on how customers can get the most from their uTest (or any crowdsourced) testing service. It is a MUST read for any potential uTest customer: http://trancecyberiantester.blogspot.com/2012/10/crowdsourced-testing-lessons-learned.html
Complaint #2: The uTest sales process is shady
In full disclosure, I’m not a uTest customer so I’ve never gone through this process myself. However, I am a uTest TTL (Team Test Lead) and so I have worked with many different customers. I’ve seen customers who are extremely satisfied and those constantly complain. It doesn’t take long before you start to see a pattern.
I really could just do a copy/paste from above. Again, this comes down to having correct expectations and being involved. Customers who follow Elena’s advice will find that uTest’s services are well worth the money and effort. Those who don’t will be disappointed with their results.
Another point worth mentioning is that uTest is a start-up company. They have only been around a few years yet they are growing and changing incredibly fast. In just the last year I’ve seen some impressive improvements. I have no doubt that as the company matures and customer needs and expectations are better understood, the sales experience will improve and mature as well.
The uTest tester
Complaint #1: uTest is a scam
As I mentioned above, uTest is a start-up. The company grew faster than many people expected and as a result it went through some obvious growing pains. Everything about the company was (and still is) evolving. The payment process was still being worked out, bug reporting and evaluation was confusing, and in general the tester experience gave some the impression that uTest was either a scam or just unprofessional.
Admittedly, the first tester interface was terrible. It was slow, buggy, and difficult to use, which is quite ironic for a software testing company. This problem has now been addressed. uTest recently launched their new tester platform and it is so much better (read more here). Their are now several reliable ways for testers to receive payment, and there is an entire team of employees solely dedicated to the welfare of the testers. These are just a few examples of how uTest is working to improve its image and show testers that uTest is a legitimate company and a great place to work.
Complaint #2: You can’t make any money
I recently read a review from a uTester that he had reported 87 bugs but he only was paid for 16 of them. The other 71 bugs were rejected. He felt that bugs were intentionally rejected in order to avoid paying the testers. He’s not the only one to complain that testers are not adequately compensated for their efforts. Fortunately it is because of a few misconceptions.
Testers need to understand the uTest bug payment model. Customers pay uTest a set price for an agreed upon amount of work. It is up to the customer to accept or reject the bugs reported by the testers. uTest then pays the testers for the bugs (and other work) the customer accepted. Since the customer pays a flat fee no matter how many bugs are reported or accepted, they have no financial incentive to reject individual bugs. Bugs are rejected for valid reasons, not to avoid paying for them.
The other important point is testers are not paid for their efforts (there are some exceptions), they are paid primarily for the value they provide. Testers who provide the customer with high-value bugs make a lot of money. Testers who report low-value or “junk” bugs make very little money.
The bottom line is good testers can make good money working at uTest. Poor testers will be frustrated.
Conclusion
uTest is not a scam. It is a legitimate company and an amazing one at that. While uTest is not perfect, most criticisms can be answered if you look at the entire situation objectively.
Interesting. I’m a new uTest tester – just been a member for a month. I don’t think it’s a scam – I have high hopes for it long term- but I do think some of what they tell new testers need to be taken with a pinch of salt. They tell you that if you pass the Sandbox program in the top 30% – which I did-pass the Quia quiz with 100% – which I did – , and introduce yourself on the forum – which I did – you should then start receiving invites to paid cycles – which I haven’t. But fair enough – the fact that I have done these things isn’t automatically going to create suitable projects.
Also, I got an e-mail from uTest Research asking me to complete a brief survey about the chat feature. “We’re asking a select group of testers to participate in a very brief survey that will help us to continue to refine the feature.” Come again? I’ve only just joined, and have not yet had an invite to a paid cycle. Does that make me a member of a select group? This e-mail was presumably sent to everybody.
But then as said it’s early days both for uTest and my time with them, and I do feel optimistic about it long-term.
Hi Robert,
Mobile testing is really big right now, so you really need to have a popular mobile device like an iPhone 5 or a Galaxy IV. As for the survey email, that’s marketing for you. I think the entire uTest community was part of that “select group” 🙂
thanks for your comments
Lucas thanks for the tip ref. mobile. I need a new one and am being a bit indecisive about what to go for. Think you have just helped me make the decision. Have also just seen a good mobile app testing course from a company here in Denmark which I will be signing up for.
Best regards Rob Ingleby.
Hi Robert, are you still working at utest? How is it going now? I joined utest few months ago, and so far I am liking it. I haven’t gotten a nice rating yet because I don’t report that many bugs, but I get paid for what I do. My only complaint is that, there should be an hourly salary too, because sometimes we spent hours and hours on something, let’s say to find a few bugs, and as in my case, sometimes they get rejected, so I dont get paid for my efforts and time. I really wish that along with the current pattern of paying, they include maybe a small hourly rate too, i beleive that will surely get the testers more motivated and involved in their work.
Hi Im interested in uTest… Is it available here in the Philippines?
Hi Jonathan,
uTest is available globally, however paid work is based on supply and demand and varies by area. You can learn more here.
My final verdict on uTest as I ask them to de-activate my account- nice idea, they just don’t have the work. Fair enough – they never guaranteed it. If I have a complaint about them, it’s that I wasted a whole weekend of my life on their Sandbox program. WHY do they ask people to do this if there is no point to it?
uTest/Applause actually has a tremendous amount of work, but most of the work needs to be completed by a specific set of testers. For example if the project calls for testers in the US with iPhones, testers in the UK with Android phones won’t be invited to the project. The projects we run and their requirements are constantly changing, so you might not have been invited to the project today, but you might get invited to a different project tomorrow.
Another determining factor is your rating and reputation. Experienced, highly rated testers get significantly more work than new unknown testers. This makes sense since it’s in the customer’s best interest to provide them with testers who have proven they do good work. It can take a while for a new tester to get experience and build up their rating and reputation, but I assure you it’s well worth the effort.
As a uTester for about two months now, I can say that it is most definitely NOT a scam. You get in what you put out. If you want to report a whole bunch of bugs that are not terribly valuable, then you are going to get payouts that aren’t terribly high. Think about it from the customer’s perspective: what bugs are going to block the user from using their app and what bugs are going to go unnoticed? I am extremely pleased with uTest, and if you put in the effort, you will be, too. Lucas is a great leader at uTest and I assure you he has nothing to gain from writing this article.
Thanks Michael, but actually I do have something to gain; hits on my blog 🙂
Lucas , my last word then I’ll
leave you alone.(Gives you another
hit on your blog!) One thing I DO accept
it’s a U.S company so that”s where
most of the work will be.Fair enough I
need to find the alternatives on this side
of the pond. I’ve got no complaint against
uTest, it”s just all this “What to expect
in the first month” stuff I was sent
after sailing through the Sandbox etc
led me to expect too much. If you put in
the effort,report valuable bugs,you get out
what you put in?Chance to do any of these
things would have been nice.
Rob,
What I think really ramped up my invites to tests was going to the forum and looking at upcoming paid opportunities and applying to them specifically. This got me rated as a tester and now after about three months I am invited to about a dozen cycles daily.
Give it a go 🙂
I like this post. I don’t think utest is a scam either. I can tell you that almost two years ago when I jumped into this endeavor, I knew absolutely nothing about testing! However, with the promise of “on the job” training and interest in technology in general, I jumped in feet first. I was a little leery at first, but after watching a few utester spotlight videos of people just like me, I felt better. That and the fact that I owned several different devices in the first place. After completing my sandbox program, which was nothing to shake a stick at, as it was a live site with not many bugs for sure! I went on to get invited to a couple ongoing test cycles for a couple of companies, and ended up getting a “favorite tester” badge. That company I still do work for on a regular basis. Right now, I tend to get more invites than I have time to do as a Gold rated tester, and I have picked out a number of companies that I like doing work for. All in all you get paid for what you want to put into it. Who knows, perhaps you will learn a new skill in the process! 🙂 In conclusion, if you sign up and only have a Windows XP desktop and a flip phone you aren’t going to get much ,so perhaps this set up is not for you; because there are a whole lot of testers with every device known to man out there, and they are going to get the cycles!
You stated: “The bottom line is good testers can make good money working at uTest.”
So what is good money? 2K per month 3K? 5K? 8K?
Could you provide the following:
1. The top monthly earners average monthly earnings.
2. Average monthly earnings of someone who’s exactly the 90th percentile earner.
thx!
That statement is intentionally vague, because every tester’s situation is completely different. However, I can point out this official statement:
source: http://university.utest.com/getting-started/
Thx for the response. I saw their statement. I personally am making around 1300 per month right now. In my mind, given how much work it takes to make that, I think 4K would be about the max you could make. It is true that getting a lot of junky little tests slows you down a lot. If I could only get more bigger cycles.
I’ve been doing it steadily for 2 months and am a high silver rated tester.
I love your question, this had been in my mind all the time and I don’t know who to ask, since its not good to talk about salaries, but still, we want to have a little idea. I have read your next comment too, where you mentioned your average salary, thanks for that, now I want to ask:
1. how many hours a day you work at utest to make this much?
2. how much is an average salary for bronze, silver and gold rated testers?
3. I see your comment is 2 years old, i want to know, how is it going now, in regards to work, and money both.
Kindly answer my questions, since I really want to have an idea and want to decide is it something I want for myself or not? I will greatly appreciate any suggestions from everyone. (Amna’s little background about me, I am a state at home mom, with a degree in software,who can only work when kids are at school, or maybe an hour or two, at midnight).
Hi there.
Is uTest only for testers that have a background in technology? I can’t seem to find information on what kind of background/education/expertise is required to be a uTest tester.
No it’s not. Anyone can join and many people with no prior technical or even testing experience have done quite well at uTest.
uTest is definitely NOT a scam. I’ve been testing there for about 3 years. You probably won’t make 100K doing it but it is possible to make really nice money doing it.
It is the kind of thing where you have to work really hard to get yourself known among the project managers so they will help promote you. Once you get know, you get many, many invites.
So actually I think eventually you could even make 100K doing it per year.
Oh and yes. They actually pay you what they say they will, when they say they will.
It is on Forbes top 500 companies to watch or something like that for 3 years in a row.
Also, please note, the more devices you have, the more invites you get. iPhone 5 and 6 and Samsung Galaxy s4 and s5 are at a premium. You will get tons of work if you have an iPhone and a S5. Well worth the investment of buying the phone outright. Don’t even need a plan really. Can test using WIFI for most tests.
Just to be clear, there are more factors that are taken into account; It’s not simply the device you have. Your location, your availability, your performance on previous cycles, your reputation within the community and many other things are used to determine how many projects you are invited to. But to your point, having a device that is in demand is a prerequisite to getting work. If all you have is an iPhone 3G, you won’t receive many cycle invites.
To be clearer, my point still stands. Not sure why you’re arguing against someone who’s on your side.
The question was really “Is uTest real”. And it’s a fact that having an iPhone 5 or 6 like I said will get you tons of invites. S5’s too. Half my invites are on iPhones and I can’t even accept them very often – too much work right now.
How strict is the time commitment once you have “accepted” a project? I would be interested in doing this as a side job and as a way to get involbed in the industry, but it might be difficult to commit a definite block of time to a project up front.
For the most part, there is no time commitment required. You can test as much or as little as you want. However, if you commit to completing a test case (predefined set of tests), you typically have a set amount of time to complete it.
Hello Lucas
I just saw Utest in my facebook feed and decided to check it out. At the moment I’m in college, but I’m also learning to code, partly because I would like to earn a living while not hugely disrupting my studies. My majors are math and physics, so I will have to learn to code for career purposes in any case. I also have been interested in coding for a long time.
Which leads me to my question. Would Utest be a good place for a beginner like me to learn more about coding and its real world applications (as a tester). I’m not so hugely concerned about the pay (not that it would be unwelcome mind you 😉 ), as I would prefer to make earnings as a contractor. I just want to know how suitable Utest is for me as a beginner coder/developer and how suitable I would be for it.
Working for uTest wouldn’t hurt your growth as a coder, but it won’t directly improve your coding skills either. Unless you are on the automation team, uTesters don’t write code. However, simply being exposed to many different products at various stages in their lifecycle can be a huge benefit to you as an IT professional. You’ll get a sense of what something looks like when it is early in development as opposed to something that is almost ready to be released to the public. You’ll also get a good idea of the types of bugs that are introduced during development which might help you avoid them in your own code.
Hi !! I just heard about that today from one of my girls mom who just started two months ago. And I am wondering about even not good English writer can do this one too ? I come from other country and live in U.S. 11 years and want to do something for my spare time. Pls tell me something about how it works. Thank you
All work must be done in English, however we have many testers who have a different primary language. You can get all the information you need about how Applause/uTest works here: http://www.utest.com/
Utest/Applause is a great place for utest/applause only..
They get tons of free testing out of people, because you only get paid for finding defects.
So if you get a fairly well fleshed out application, you may spend a day or 2 and only find a few small defects. It also leads to testers only looking to land the big one, and not bothering with minor defects.
Oftentimes you are limited in how many defects you can even submit.
I tried it for awhile, determined I could make more delivering pizza or working at McDonalds per hour.
Thanks for the comment JP. I’m sorry you didn’t have a more positive experience, but it may be that you didn’t quite understand how we do things. I’ll address each of your concerns and hopefully you’ll be able to give it another shot.
– Tester are absolutely paid for finding bugs, but that is not the only thing they are paid for. We also pay for completing test cases, for writing exploratory notes, for completing tester reviews and in many other ways.
– You are not required to accept any cycle. If you feel a product is not worth the time to test (payouts are too low, you don’t like the product or whatever) you can decline the invite. Newer testers need to prove themselves so and they are invited to relatively few cycles, some of which may be less profitable than others. Our best testers get many invites including our more selective and lucrative projects.
– All new testers are limited to 5 unapproved bugs. This limit is removed after you get some experience and increase your rating. This restriction is in place to prevent new uTesters who don’t understand how things work from reporting a bunch of “junk” bugs that only results in more work for everyone.
– There are MANY uTesters from the US and around the world who test for Applause as their full-time job and make a very respectable income on-par or better than any traditional QA/Tester position. However, not everyone is cut out to be a successful uTester. I’m working on a post outlining the traits needed to be successful, so be on the lookout for that.
Hi Lucas,
thanks for the blog post – I have joined uTest yesterday and got a little overwhelmed by the amount of articles, forum posts, and links I have to go through before even getting a peak at a test cycle! 🙂 But your post has reassured me that if I put in the effort, it might well be worth it. I do not have much software testing experience, but I am familiar with test cases and logging bugs as I have been a game tester for a couple of months now. I have been invitied to the Sandbox cycle, so wish me luck with uTest…
Welcome to the community Pola! I’d encourage you to hang out in the utest forums as much as you can. Feel free to message me there if you have questions 🙂
I had the same experience Pola!
But trying now to solve a test cycle, didn’t work yet but we can try the next one and slowly get into it 😉
Hi,
What about devices such as iPad 3G and iPod Touch 6G with wifi, is there much demand for testing with these?
Thanks
Andrew.
Typically, the newest and most popular devices/OSs are in the highest demand. Occasionally there are needs for older devices but not often. I’d say the device demand would be in this order: Phones, tablets and then other devices (iPods, wearables, etc).
How does utest feel about testing with emulators?
One of the main differentiators for uTest/Applause is that we have real people, using real devices in the real world. Unless the cycle explicitly says it’s OK to use simulators, they are strictly forbidden and may be grounds for your removal from the community.
Utest is not a SCAM and a powerful tool for hundreds of customers to unwind the defects from varied perspectives across the globe. The only complain of mine on Utest is that with some customers it is very challenging to get to know what they are expecting. They clearly will not call out what they are looking for. So many testers accepting the cycle and spend lots of time and energy to report the defects. At the end it is at the customer’s team discretion that what defects will be approved and what not.
My above statements are not from a pay value stand point. There are tons of testers who are passionate in testing and expect the same when it comes to the payout even.
I would like to conclude that this platform is worth for those people who have loads of time and no source of income. If the utest wants the quality testers to join them, they have to review their policies with respect to the time and payout for those valuable utesters. Else it just remains as is now. This is a great concept but unless there is some professionalism & value added to it, and unless the utest enforce the customers to provide all the details that they need as part of cycle overview, i definitely doubt that the utest can survive for long.
What specific professional and value-added changes do you feel are necessary?
Why do you think it’s necessary for uTest to require customers to provide a certain amount of detail?
It is not a scam, i have been working there. But uTest can be extremely unfair at many times. The project managers especially. In cycles when customers do not accept the issues, the project managers randomly mark the issues as somewhat valuable, even when there are extremely and very valuable bugs available and so much effort to find bugs result in $2 per bug (which is extremely unfair). Secondly, they keep on reducing the payout of bugs, many of the cycles have $0.75 as their somewhat valuable bug and so much of hardwork results in nothing. So project managers to be fair enough, you can’t just randomly mark someone’s so much hard work and good bugs as somewhat valuable (If you are told to save money like this then it is something else)
When a customer decides not to approve/reject their issues, and this happens for many different reasons, the TTL or PM approve the bugs using their best judgment, in accordance to how they believe the customer would value them. I assure you it isn’t random. However if you ever feel that you or other tester were treated unfairly, you should definitely reach out to Community Management testers@utest.com and they will work with you.
Cycles with low payouts on somewhat valuable bugs typically have a high payout on very and exceptionally valuable bugs. This is designed to discourage testers from reporting low-value bugs and to encourage you to spend your time testing and looking for high-impact bugs that the customer will actually care about. Keep in mind that every cycle is different and you are able to see the bug payouts before accept. If the payouts are too low for you, simply decline the cycle and wait for one that pays more.
testers@utest.com people are racist. Once I asked about why certain test cases have amount reduced with the same steps. I got a reply from some Rebecca Carroll via RT, “The amount will be adjusted because of the country you are located in”
Just because I live in a third world country, does this mean I am going to get such racist replies?? She even suggested to ask about payout in the chat (Although uTest does not allow that) So racist people sitting over there.
My current sentiment is that in one of the cycles I recently worked at (it’s a second cycle), there were truly, for a lack of better term, crap bugs that are either obviously works as designed or not bugs at all that got accepted with “very valuable”, and then the latter bugs just got “somewhat valuable” acceptance even though most of them are detailed stuff that really affects the flow of the application. I even found a lot of duplicate bugs that got accepted, but in the first cycle I got like 2 rejections (which I disputed) because it’s a “duplicate”. I do hope people genuinely search for bugs instead of just coughing out nonsense for the sake of money.
uTest is definitely not a scam, been a member of the uTest community for 22 days now and I got my first payment 15 days after joining uTest. My only comments is related to turnaround time with regards to acceptance and rejection of bugs and testcases. I have experience with 1 cycle test that my bugs and testcases are still pending eventhough the cycle has been closed for a week already. I understand though that we need to wait for atleast 30 days after cycle is closed for use to know the acceptance and rejection of bugs/testcases . I am just a little excited and my tendency for me after reporting bugs or completing testcases is to know as soon as possible if they are approved or not. It’s so exciting to actually look at test cycle and clicking My Earnings 🙂 but if your reported bugs/testcase are still pending then you cannot see anything at your My Earnings tab. But this is not really a big concern since I know base on experience that uTest will eventually decide on your reported bugs/testcases. I would really like to say “Thank you uTest” for giving us opportunities to expand our knowledge in QA/Testing and also to earn additional income 🙂 I really like uTest community. Salute to all uTest management and members.
Thanks for the comment Rogelio, I’m glad you’ve had a positive experience so far!
Hi Lucas. Nice article you have here. I have a question. How long does it usually take for you to get your earnings? In my uTest page it says the pending payment is actually paid already, but it’s been three days and I still haven’t seen anything reflected in my account. Has this ever happened to you? I know that uTest isn’t a scam in the least, but I’m quite worried as this is my first payment.
As for uTest, it definitely is a nice sideline. You get to see some products before they are officially in the market, and it makes me giddy to be able to have the experience. It’s been great so far and I hope it continues to be this way.
I’m sorry for my delayed response. I hope you finally got paid 🙂
If you have any issues with payments, please contact testers@utest.com and they’ll be happy to help you out!
uTest is not a scam. I just got my first bonus payment today aside from my total payout for finding a very valuable bug. Thanks uTest!
I went out there to see what it was about. The generic word, testing, is used constantly. I looked all around the site, I receive their newsletter, I’ve been to the FAQs, I have read this blog, but testing is still testing. What the heck is being tested and how do you test it. I’m not exactly a dummy. I have a degree in sound engineering, but this must be some kind of cloaked brotherhood of testers.
Do I see a light?
Hi Missy,
Please sign up for an account and then start here: https://www.utest.com/courses/getting-started
Our tester community now has over 250,000 members and would love one more 🙂 it just takes a little work and patience on your part to get started. Don’t get discouraged!
I have regisered on utest and a few other crowdsorce testing sites but have mostly abandoned it in favour of online language tutoring. The classes are set, and regular so you know what you are getting each day and can plan for it. I had to go through a formal onboarding process however to get the contract. I used to be a tester for a large enterprise and I prefer that to crowd source testng. I have the developer specs, i can see the code chages and i know i am test writing and executing from a plan. I also have a regression suite as well. Crowd sourcimg works for the business who can have unlimited resources at their disposal but for the individual it’s not a good way to earn money in my opinion. With perhaps over 50 other testers doing the same thing and possibly finding the same fault and getting paid per defect found it’s working for peanuts.
just signed up and then quit process after they asked for DID number to my Iphone. Don’t think they should get that many details to be able to gather test data for users. Also on their blog post section has suspiciously high amount of Russian posters. These days I think its better to be safe than sorry. Don’t wanna be an unwitting Russian puppet. https://www.utest.com/forums
We collect iPhone UDIDs because they are used to give your device access to apps that are in development and not available in the app store. It’s just the unique ID of your device, it’s not sensitive at all.
Applause/uTest has testers from all over the world which is one of my favorite things about this company.
I am a pretty good tester, but my only personal device is a laptop with Windows 10. Reading through the posts, I get the feeling there is little, if any, testing opportunity with what I have. Could you comment?
Also, no posts for more than 1.5 year. Is the community still active and testing?