5 Traits of Successful uTesters – Part 4

Originally posted at: https://www.utest.com/articles/5-traits-of-successful-utesters—part-4

Passion

The path from a rookie to a top uTester is not an easy one. As we have seen in previous posts (Part 1, Part 2, Part 3), uTesters are challenged every day. Our work environments are continuously changing and we are faced with ambiguity from every direction. Sometimes it feels like there’s no work and other times we can’t keep up. uTesters have many reasons to look for “easier” work opportunities.

However, the most successful uTesters are able to thrive despite these challenges. Instead of being discouraged, they are excited. The have come to realize that this is exactly what brings out the best in them because they are passionate about what they do.

I’ve seen many testers in traditional testing roles who do everything they are “supposed” to do. They show up every day and they do their work reliably. They may even do a pretty good job at it. But they lack any real interest or enthusiasm. Without passion, they never will grow and realize their potential.

In the same way, testers who come to uTest just for the money or because “it’s a job” may make some money, but ultimately they won’t be very successful. There are literally thousands of other hopeful uTesters but only the most passionate raise to the top.

So what specifically does a successful, passionate uTester do? Well, watch them! Find several testers who have had continued success (not simply short-term success) and see what they do.

As you identify different acts of passion from the topuTesters, I’d encourage you to post them in the comments section below!

5 Traits of Successful uTesters – Part 3

Originally posted at: https://www.utest.com/articles/5-traits-of-successful-utesters—part-3

Versatility

Several years ago, I was a tester on a small development team. We built and maintained several products that did completely different things, yet they were quite similar. They were all .NET web apps and, per company policy, only had to work on IE 9. This meant that my ability to test was limited because there was no need or opportunity for me to learn how to test other browsers, platforms or technologies.

When new testers join uTest, they often find themselves in a similar situation, but for a different reason. They have a laptop and their one personal phone listed on their profile and that’s it. Their ability to test is limited by the devices they own.

Conversely, Versatility (both in terms of ability and tool availability) is an important attribute that all successful uTesters have.

Ability to Test Anything

uTest is constantly at the forefront of technology. Companies come to us to test the latest and greatest the IT world has to offer. From beta OS to pre-release smart phones. From wearables and IoT (Internet of Things) to smart cars and smart homes. Sometimes before anyone even knows how to develop on these new technologies, companies come to us to help them develop their testing strategy.

The most successful uTesters are the ones that are asked to join these unique and challenging projects because they are the ones who, through experience, have proven that they are capable of meeting those challenges. They understand the core fundamentals of testing theory and use that knowledge along with their past experiences to devise ways to test things no other tester has seen before.

Tool Availability

The most brilliant painter in the world could do little more than a finger painting without a collection of the right paintbrushes and knowledge of the techniques needed to use them effectively. In the same way, the most successful uTesters have the correct tools. With collections of laptops, phones, watches, and dozens of other devices, these tester’s desks look like they just robbed an electronics store.

For example, take a look at my friend Rob’s profile picture:

But simply having the tools isn’t enough. I have a stack of paintbrushes and my mother won’t even take any of my artwork home! The best uTesters know every inch of each device. They configure them to mimic different real-world scenarios. They know the different techniques to take videos or capture log files. They are able to do things most testers didn’t know was even possible and sometimes even surprise our customers.

While some uTesters may realize a small measure of success even with limited abilities and tools, the best uTesters owe their success to their own versatility. They are to accomplish any testing task given to them because they have the abilities and tools necessary to get the job done.

For you new uTesters, what can you do to improve your Versatility?

Also, be sure to check out Part 1, Part 2 and Part 4 of this series.

5 Traits of successful uTesters – Part 2

Tolerance of Ambiguity

Welcome to part 2 of this series where we are discussing the traits required to be a successful uTester. Last time we saw that the best uTesters are challenged with constant change and are still successful. Now we’ll see how important it is for uTesters to be tolerant of ambiguity and uncertainty.

There are 3 areas I’d like to discuss: Exploratory testing, problems with the cycle and problems with the product under test.

Exploratory Testing

The majority of work that uTesters do is exploratory testing. One of the key aspects of exploratory testing is that the testers knows very little about the product they are testing. By its very definition, exploratory testing is an ambiguous process. It’s usually the first time the tester has used the product and there are no test steps to follow.

Some testers (particularly new testers or even experienced ones who usually follow written tests) have difficulty testing when they receive little or no guidance. It can be challenging and even overwhelming to receive a product with the simple instruction of “Test it”. This is exactly what happens at uTest every day and the best uTesters know how raise to that challenge.

For a demo of how I perform exploratory testing, check out this video course.

Cycle Problems

While PMs and TTLs do their best to make cycle overviews as clear and accurate as possible, mistakes occasionally do happen. For example, there may be a discrepancy between the in-scope devices listed in the cycle overview and the devices the tester can select from when reporting a bug or claiming a test case. Or perhaps In-scope section directly contradicts the Out-of-scope section.

Poor uTesters might simply give up on the cycle, or worse, use these mistakes as an excuse to do whatever they want. However the best uTesters realize that no cycle is perfect and that it’s important to be flexible. They know that taking the time to ask clarifying questions can help them be successful and can improve the experience for the test team as well as the customer. These are the types of testers that TTLs and PMs want on their cycles.

Product Issues

As testers, it is often our job to find problems with the products we test. Interestingly, some poor testers have a difficult time when they encounter problems. When they can’t complete an action or find a find an issue that blocks them from testing, they respond as a typical user might; they get frustrated, complain and even quit. They forget that it’s their job to experience and document that frustration before it reaches the end-user.

The best uTesters expect there to be uncertainty in the products we test and they’re ready for it. They may evaluate the product as a user, but they don’t behave like one. Instead of being discouraged by the unexpected, they analyze the problem and provide the the customer with feedback so that they can improve the product.

I’d love to hear your thoughts on any of these points. If you are a top uTester, what other types of uncertainty do you face?

5 Traits of successful uTesters – Part 1

Originally posted at: https://www.utest.com/articles/5-traits-of-successful-utesters—part-1

The Ability to Cope with constant Change

There are many traits that testers must have in order to be successful. A quick Google search for “traits of a successful tester” will produce several articles on the subject. However in this article series, I wanted to focus on the traits that are of particular importance to uTesters.

At Applause (uTest) our approach to testing is quite unique. Testers who had success at a typical testing shop are not guaranteed to have success here. Conversely, people with no prior testing or even IT experience, often become highly respected and sought after uTesters… providing that have these particular qualities.

This will be a 5-part series and in this Part 1, we are going to discuss the importance of the ability to cope with constant change.

Probably one of the most unique aspects of being a uTester is that we test an incredibly large and ever-changing set of products. Instead of working for one company and testing the same product(s) over and over, uTesters test hundreds of different products for hundreds of customers every day. Some of the top uTesters have literally tested over a thousand different products.

The top uTesters work on several products at any given time. In order to be successful, they need to be constantly analyzing the context that they’re working in. Which product am I testing? What is the state of the product? What is the goal of this cycle? If a uTester fails to test within the correct context, they’ll find that their work provides little or no value.

Just as every product is different, so too are the needs and expectations of every customer. One customer may have extremely specific reporting standards. They require bug titles to follow a detailed format, they have attachment requirements and expect several pieces of additional data. However, the next customer may not have any requirements and is just happy to be receiving bug reports at all.

uTesters need to be able to quickly switch from context to context and from one set of customer expectations to another, often several times per day. This is no simple task, but it’s the first trait all the best uTesters share.

uTest TTL Handbook

The Fundamentals for All Test Team Leads

This handbook explains the basic concepts surrounding the uTest Test Team Lead role.

Developed by uTest Community Management  – 2/11/2014

Introduction

The term TTL is a uTest-derived acronym that stands for ‘Test Team Lead.’ TTLs are hand-selected members of the uTest community who work side-by-side with uTest Project Managers (PMs) to facilitate the flow and outcome of test cycles. A few traits and required skills of good TTLs include:

  • Hard skills: high-quality testing output with proven track record, attention to details, expertise managing test cycles based on customer requirements across multiple technologies, Experience with uTest and an understanding of test cycle operations, and device/industry specific expertise
  • Soft skills: excellent communication skills, professionalism, culturally sensitive, fair and helpful
  • Sense of urgency: attentive to test cycle details and activity, focuses on responsiveness, escalates issues accordingly to the PM or CM, constructive not confrontational
  • Business acumen: understands what’s important to the customer with the ability to analyze and communicate the information across the testing community when necessary

Although the tasks required per test cycle may differ, the goal is the same: to increase the value that uTest brings to the customer by maximizing the output and minimizing the noise level.

Distribution of TTL Work

A TTL is vetted and trained before being placed onto paid projects. Paid projects are assigned by a PM. TTL payouts are always defined upfront, before any work is performed, and are kept between the TTL and the PM. Each assignment is different and depends on many factors, such as the TTL’s ability to triage bug reports or their ability to create test cases, to name two. Other criteria may include the TTL’s experience in a particular testing domain, industry, or mobile platform and access to those platforms to reproduce bugs. TTLs should utilize the TTL Availability Google Doc to enter your weekly availability and project preferences. Request access through the link provided.

Details of TTL Assignments

For many test cycles, the TTL’s main responsibilities are to monitor tester questions via test cycle chat and triage reports as they come in. TTLs should be in regular contact with the PM to ensure that the test cycle is headed in the right direction and that the customer is delighted with the results. Below are details of the responsibilities TTLs may be asked to perform:

  1. Platform Communications with Testers and Customers
    No matter the type of communication, keep in mind that testers must follow the uTester Code of Conduct at all times. TTLs should monitor the chat as closely as possible in order to address incoming questions and deliver outgoing announcements, especially in the first few hours that the test cycle launches.

    • Chat: Test cycles typically have real-time chat enabled, which allows TTLs to communicate with testers, customers, and project managers in real time. There are three components of chat: Questions about Scope, Announcements and Threaded/Topic Conversations.
      • Questions on Scope – for general questions on the cycle. This is usually related to scope questions, but can contain other general questions about the cycle such as questions on difficulty installing the build or claiming a test case.
      • Announcements – where TTLs should place their welcome message to the testers (see example welcome message in the addendum). Also, this is where the TTL would make any other announcements such as: announcing a new build, announcing a new out of scope section, making a correction to the information in the overview, reminding testers about the cycle close time, encouraging testers to get test cases submitted, refocusing the team on the focus areas.
      • Topic Conversations – discussions isolated to a specific topic. If there is a major development that the TTL thinks should have more attention than just an announcement, consider starting a new thread. These Topics can garner more attention because the TTL can edit the title. Threaded conversations are also good for topics like test case claims. Using these topics can keep the Questions about Scope section a bit cleaner.
        NOTE: All chat activities can result in an email sent to the testers and the customer depending on their settings, so please be mindful of the noise that chat can create.
    • Tester Messenger: Notes that TTLs add to Tester Messenger will result in an email directly to the tester who reported the issue and is added to the defect report for easy reference. This message is visible to the customer, PM, and any other testers on the test cycle. TTLs should use the Tester Messenger when:
      • There is a question about the defect that requires an answer from the reporting tester.
      • The testers will sometimes leave a message here for the TTL or the customer when the TTL, PM or customer leaves them a message in the tester messenger. Be aware that testers cannot initiate that exchange. In addition, the TTL receives an email any time a message was added to a bug that they have commented on. When testers leave a message for the customer, the customer should have visibility and may respond, but if historically they are not that responsive, the TTL can email the Project Manager, relaying the question, then respond to the tester in the Tester Messenger.
        NOTE: Be careful about exposing the TM’s contact information to the tester if responding to the tester via email (say, forwarding the response). The TTL is never to give out the TM information unless expressly asked to do so by the PM. If so, please send the information to the tester privately through email or chat to keep that information out of the tester messenger where it could be seen by other testers. It is preferred for uTest to be the go-between rather than to start opening doors between the testers and TMs.
      • If the question and answer are determined to be valuable to other testers by the TTL, share this information on the Chat/Scope & Instructions with all the testers.
    • Request More Info: If there is missing information or more clarification is required from a tester, the TTL can push a bug report to the “Request More Info” status, which shows that the bug report was reviewed but requires a response from the tester. Be aware that CM recommends using this status only for small portions of missing information. For a bug that has more significant issues, for example it does not follow the customer’s bug template, is clearly a placeholder, or is just poorly written, the bug should be rejected as “Did Not Follow Instructions.” TTLs are encouraged to assist the overall community in elevating the expectations of testers and eliminating consistently poor bug documentation performance by rejecting bugs that meet those criteria.
      NOTE: Sending reports into the “Request More Info” status automatically adds a note to the Tester Messenger and prevents testers from discarding these bugs. However, it is possible to visually note when a tester has responded through the Platform UI and through email notification as the status of the bug returns to “New”. In addition, this status assists the testers when they see their bug change status. Testers should be reminded to press “Send Requested Info” button. Please note that if the tester does not respond before the cycle closes, then their bug will be automatically rejected.
    • Customer Notes: Notes that you add to Customer Notes sends an email to the customer’s Test Manager and makes a copy of the message in the defect report for easy reference. This message is visible to the customer, PM, and any TTLs on the test cycle who have TTL permissions (but not to testers). TTLs should use the Customer Messenger when:
      • The defect needs to be exported to the customer’s tracking system
      • The defect has been exported to the customer’s tracking system and the TTL needs to leave a note to identify the corresponding defect ID# in the other system.
        NOTE: The key to keeping track of what needs to be exported versus what has already been exported is to be consistent. For example, the TTL leaves a Customer Note only when the defect has been exported to the customer’s tracking system. Then in that note, simply enter the corresponding defect ID# from the other system. If all of the defects will not be exported to the customer’s tracking system, or if the defect is being approved but is not being exported to the customer’s tracking system, the TTL can insert a message to denote that.

        • The TTL needs to communicate directly with the customer about this issue, such as to provide clarifying information that you may have discovered after your initial recommendation was provided or to respond to a request or question from a customer, as examples.
      • Direct Email: uTest Project and Community Managers place a high level of trust in our TTL community. Occasionally TTLs will be given access to confidential and/or proprietary information including tester email address. TTLs must ensure they treat this confidential information with the upmost sensitivity.
        NOTE: If you are emailing more than one tester at a time you must place email addresses into the BCC field. This will ensure that tester email addresses remain hidden from fellow community members.
  2. Triaging Issues
    If possible, triage reports as they come in (or in short intervals of time) to prevent a pileup of work. Doing so allows the TTL to easily spot duplicates and potentially identify areas of the application that may need further clarification from the customer (in these cases, contact the PM immediately). Additionally, testers will appreciate timely triaging to help direct them where to focus their testing efforts for subsequent testing. There are two levels of permissions for TTLs: Basic and Approval. The Basic level allows the TTL to send an approval recommendation for how an issue should be triaged, while Approval allows the TTL to effectively triage as a customer or PM. Basic-level permissions require the TTL to complete several pre- determined dropdowns that provide more information to the customer and PM – including whether the issue is valid, in-scope, reproducible, and a duplicate of a previously-reported issue (hereby labeled the TTL Recommendations Workflow). Please consider the following items in triaging issues:

    • Triage fairly and consistently: The age-old question is whether to offer testers a 2nd chance when they do not follow instructions, or use a ‘tough love’ approach that will signify the importance of high quality with every single interaction. In an effort to save time and resources, our preference is to offer the ‘tough love’ approach, unless the report is salvageable and could be valuable for the customer. Therefore bugs that violate the customer specified bug templates should be rejected as “Did Not Follow Instructions”, while high-quality bugs and reports that have minor mistakes can be salvaged using “Request More Info” tools (more on that below). Testers can discard their bugs up until the TTL triages their bugs, including sending it to the “Request More Info” state.
    • Bug Templates – Is the report in the proper format (if the customer asked for a specific format)?
      • Confirm that the bug title contains the appropriate information.
      • Confirm that the steps to reproduce are clearly documented as required by the customer or development. Testers should be urged to number their steps to make it easier on the customer to read and reproduce.
      • Confirm Expected Results and Actual Results are documented clearly and provide factual information.
      • Confirm that all appropriate attachments are included as required.
      • Confirm that all the additional environment information is included.
      • Confirm that all other customer specific format requirements are met.
      • Confirm that the bug is classified appropriately.
    • Bug Reproducibility – Is there sufficient evidence supplied in the bug report?
      • i. When possible, TTLs should attempt to reproduce each submitted issue. In some cases the TTL is not able to try to reproduce a defect before triaging it due to device or time constraints. In these instances, the TTL should ensure the tester has provided sufficient proof that it happened during their testing (logs, screenshots, videos, clear and complete steps, etc.).
      • ii. If the TTL is unable to reproduce the issue on the same device/OS as the tester, they may consider asking the tester to verify the issue is still happening to verify the steps written are full and complete. They may have either missed a step or the issue may have corrected itself. Given the bug has adequate information to verify the tester was seeing the issue at the time it was reported, non-reproducibility should not be a reason to reject a tester’s issue before the customer has reviewed it.
      • iii. If there is missing information or more clarification is required from a tester, the TTL can push a bug report to the “Request More Info” status, which shows that the bug report was reviewed but requires a response from the tester.
        Note: Sending reports into the “Request More Info” status automatically adds a note to the Tester Messenger and prevents testers from discarding these bugs. However, it is possible to visually note when a tester has responded through the Platform UI and through email notification as the status of the bug returns to “New”. In addition, this status assists the testers when they see their bug change status. Testers should be reminded to press “Send Requested Info” button. Please note that if the tester does not respond before the cycle closes, then their bug will be automatically rejected.
    • d. Bug Classification – Is the severity and bug type of the issue appropriate?
      • i. If unsure, clarify with the PM about severity and value expectations. For example, what do they consider “Critical” vs. “Low”?
      • ii. If the Severity is not appropriate, send the tester a message asking them to update it, or the TTL can update it themselves if they have the appropriate permissions in the platform. TTLs can also leave a message for the customer’s Test Manager to make the update when they are approving bugs.
      • iii. If the type (functional, UI, technical) is not appropriate, send the tester a message asking them to update it, or the TTL can update it themselves if they have the appropriate permissions in the platform. TTLs can also leave a Customer Message for the customer’s Test Manager to make the update when they are approving bugs.
    • e. Duplicate Bugs – Is the issue a duplicate of a previously-submitted issue?
      • i. If you believe it is, reject the issue as a duplicate. Be sure to cite the bug report # that the rejected issue is a duplicate of. The time stamp for each defect entered into the platform is critical. When deciding which defect is a duplicate and which defect to approve, bugs reported earliest take precedence.
      • ii. If the customer does not want the TTL to do any actual approvals or rejections, the TTL can recommend that the defect be rejected as reason Duplicate and provide the # of the original defect for reference.
    • f. Out of Scope – Does the bug appear to be in scope?
      • i. The scope is determined by the customer and should be clearly defined in the Scope and Instructions provided by the PM.
      • ii. If the bug is not in scope, recommend rejection using “out of scope” as the reason.
      • iii. If the bug is not in scope and the TTL has the ability to approve and reject, then simply reject the bug using the “out of scope” reason.
    • g. Bug Validity – Does the bug appear to be valid?
      • i. Most submitted defect reports will be clearly valid or invalid for one of the reasons above, but some will need further consideration. It may be worthwhile to try to reproduce anything you are in doubt of, to get a clearer idea of what issue is actually being reported.
    • h. Bug Approvals – if the TTL has the ability to approve bugs, they should be very careful that they approve the bug at the proper value. Do not accidentally approve a bug as Exceptional when it should be Very or Somewhat valuable.
    • i. Customer Approvals – How to deal with the reported issue that is a valid defect from a tester standpoint but not a valid issue from the customer’s perspective?
      • i. In this scenario, there are two actions that the TTL can take, depending on the individual circumstances:
        • i) Reject (or recommend rejection) as Working as Intended/Designed. This reason does not negatively impact the tester’s rating but they also will not get paid for the defect.
        • ii) If applicable, Approve (or recommend approval of) the defect but leave a Customer Note that indicates that it should not be imported into the customer’s tracking system. Defect value is normally set to “Somewhat valuable” for these.
      • ii. In the case where the test application is either overly complicated or the customer does not have a complete list of known issues/working as designed, it’s more fair to the testers to approve the defect so they are paid for the time they put in reporting it. Too many “Working as designed” rejections cause testers to shy away from testing and can cause participation rates to drop.
      • iii. If the average tester would consider the issue as working as designed, then a rejection is the appropriate strategy. Use your best judgment when making this decision.
    • j. Bug Export – If the bug is valid, does it need to be entered into the customer’s bug tracking system?
      • i. If so, and the TTL will be handling the bug tracking system import, there are two ways this can be done.
        • i) If the account is set up for it, you can use the “Export to Tracking” button on the bug report in the platform to automatically move the report into the customer’s system. This functionality is customer facing. If a TTL is given an account to access customer site they must be aware that they should not use that account to triage bugs. TTLs should only use their tester/TTL account to triage issues. Under no circumstances should TTLs approve their own bugs.
        • ii) If the account is not set up for it, you can manually copy the bug report from the uTest JIRA system to the customer’s bug tracking system. The most efficient way to do this is to click the Copy link at the top of the defect screen in the platform and paste it into WordPad, then copy the sections you need into the corresponding sections in the customer’s tracking tool (JIRA/Team Track/etc.).
  3. Triaging Test Cases
    Depending on the test cycle and PM, TTLs may be responsible for approving or rejecting Test Cases. TTLs should always open each Test Case submission and check for all requested materials before choosing to Approve or Reject. Please consider the following points in triaging test cases:

    • a. Did the tester include all requested documents or information with the Pass/Fail verdict?
      • i. If attachments were requested, confirm that there actually is an attachment. Then be sure to open them and verify they are in the correct format and contain the necessary information. If multiple submissions were made by a single tester, compare these documents across the various completed test cases to verify they are original and not simply duplicates with different titles and dates.
      • ii. If the tester was asked to do exploratory testing, make sure the exploratory notes are filled out completely and cover a sufficient amount of testing time dependent on the expectation set in the overview or based on the payout (i.e. a $10 test case will probably have a lower expectation of work than a $30 test case).
    • b. Were the testers required to sign up (usually via an external document) for a limited number of test cases? If so, check to be sure the tester’s submission you are approving is slotted for a test case and that the device/browser/etc. details match.
      • i. If the testers were required to sign up, were they instructed to only fill a specific number of slots? Keep an eye on testers who try to reserve more than their share of test case slots.
  4. Bug Fix Verification
    If the TTL is working with a customer who either has bug tracking integration with our platform or for whom you are manually transferring our defects, you may also be responsible for identifying the need for (and possibly kicking off) Bug Fix Verification (BFV) cycles. Clarify with the PM who will be responsible for BFV management before the test cycle starts. Keep an eye on the fix statuses of defects that have been moved from the uTest platform to the customer’s bug tracking system and either manage the BFV cycles in a timely fashion or notify the PM which issues need attention. If a bug is verified fixed by the testers via BFV, be sure to update the status to the corresponding bug in the customer’s tracking system. If the testers find that that the issue is still recurring, report this status in the other system and continue to monitor it there, kicking off another round of verification testing when warranted. Again, be sure to clarify with the PM who will be responsible for BFV management before the test cycle starts.
  5. Disputed Bugs
    If a bug is disputed by the tester, the TTL should review the comments and then speak to tester on Tester Messenger to notify the tester that their bug is being reviewed. If the TTL has control of rejections and approvals, they can determine a final status. If rejected, the bug changes to Under Review for the PM to make a final determination. If the TTL made a mistake in the initial triage of the bug, they can recommend to the PM that the dispute be resolved and the bug approved.
  6. TTL Testing on Test Cycles
    TTLs should discuss whether they can test on the same test cycle that they are TTLing with the PM for that test cycle before launch. If the PM agrees that the TTL can test on the test cycle, then the TTL should notify the testers through the welcome message in chat to reduce confusion for the testers.

Expectations and Escalations

The following list contains the expectations and escalation paths specific to all TTLs. The tester Code of Conduct applies in all cases for all testers and TTLs.

  1. Expectations: TTLs are expected to…
    • Respond as quickly as possible to questions in chat for the test cycle. The availability schedule for the TTL and the PM should be listed in the Scope and Instructions. While testers typically expect chat responses within an hour of submission, TTLs should attempt to respond with 2-4 hours during normal business hours and within 12 hours during the nights and weekends.
    • Closely monitor test cycles directly after it launches for questions in chat and issues with bug reports.
    • Monitor chat for appropriate tester behavior and escalate any issues that are observed.
    • Help reduce the noise created by testers and reduce the number of information requests that PM’s and TM’s must address by gaining a strong understanding of the application and the scope of the test cycle and taking the lead role in answering questions in chat or in the defects.
    • Triage defects and test cases during the test cycle. Feedback (including approvals, rejections, or requests for more information) to testers during the test cycle can result in more detailed information in the bug reports and higher quality bug reports and test cases available to the customer. This increase in response to testers can assist in improving the overall quality for subsequent bugs in the current test cycles and future test cycles.
    • Provide feedback to testers based on their knowledge of the test cycle, the application under test, the customer, the devices under test, and the expectations listed by the PM prior to test cycle launch. It is understood that this information may change test cycle to test cycle or even within a single test cycle.
    • TTLs should provide feedback to testers when expectations are not met in defects, test cases, and reviews. They may also provide feedback for exceptional work.
    • Be flexible, very flexible.
      NOTE: If a TTL is unable to commit to the expectations of a particular test cycle they must give the Project Manager sufficient notice so that the role can be successfully backfilled. Sufficient notice will vary by test cycle, at the very least a TTL must give 48 hour notice that they will be unable to perform the work required. However, if there is a longer on-ramping time associated with a cycle, then the TTL should inform the PM two weeks prior to exiting the role of TTL.
  2. Escalations:
    1. For any issues dealing with the test cycle, the TTL escalates to the primary Project Manager first. If there’s not a response within 24 hours for normal issues or 12 hours for urgent issues, then the TTL should send an email to PM.Escalate@utest.com with a subject line in the following format – [TTL Escalation for Customer Name and Test Cycle ID#] and details on the issue in the body of the message. A TTL should only directly contact the customer if permission has been granted by the Project Manager, either at the project or test cycle level.
      Example 1: Testers record questions in chat about the fact that there is no review tab active for the test cycle, though the scope refers to customer reviews when testing is completed.
      Example 2: For an e-commerce site, testers are asking whether the entire shopping cart function is out of scope or only the final purchase step.
    2. Any issues with testers behavior, Code of Conduct infractions, or otherwise should be escalated to CM with notification to the PM.
      Example 1: Tester discusses test case pay out rates in chat.
      Example 2: Tester disputes rejected bugs multiple times.
      Example 3: Tester misuses slotting spreadsheets.
    3. As mentioned above, if a TTL is unable to commit to the expectations of a particular test cycle they must give the Project Manager sufficient notice so that the role can be successfully backfilled. Sufficient notice will vary by test cycle, at the very least a TTL must give 48 hour notice that they will be unable to perform the work required. However, if there is a longer on- ramping time associated with a cycle, then the TTL should inform the PM two weeks prior to exiting the role of TTL.

TTL Bill of Rights

  • Receive notification of test cycle assignments prior to launch of the test cycle
  • Receive anticipated scope of work prior to TTL accepting each TTL assignment
  • Receive information about anticipated payout prior to TTL accepting each TTL assignment
  • Receive notification from the PM prior to the launch of the next test cycle if PM decides to swap TTLs for any reason
  • Receive a response from the PM within 24 hours for normal issues or 12 hours for urgent issues during the following timeframe: 8am – 8pm US Eastern time. If you don’t hear back, you may email to PM.Escalate@utest.com with the customer name and test cycle ID in the subject line
  • Expectation that testers will contact TTLs through the platform only, not through any other method like Skype, IM, Google+, etc.

uTest Business Model

For TTLs who have not experienced uTest for a long time, here is a brief outline of uTest’s business model. This knowledge should put things into perspective when you have outstanding questions. However, the Community Management Team is always available to address any open questions about the uTest business model and how it pertains to any TTL assignments – past, present, or future.

The uTest business model is built on these Customer Expectations – that uTest:

  • Provides consultation for test strategy, test cycle configuration, and test cycle optimization to achieve high-quality results
  • Provides various encapsulated industry and domain-level knowledge and experience
  • Provides testing expertise and diverse testing coverage
  • Provides high value defect reports
  • Responds quickly to customer and testers alike
  • Assures a low level of noise during each test cycle

Basic Business Concept:

  • Larger customers generally engage uTest for longer timeframes to ensure continuity of testing coverage and expertise. In general, Project Managers that cover larger customers have fewer total customers, as this allows for deeper focus on customers who generally have greater needs.
  • Smaller customers may engage uTest for a variety of time periods, whether it’s several test cycles or annual contracts. In general, Project Managers that cover smaller customers will have more customers, which generally results in slower response times.

Key Message for TTLs: No matter the size of the customer or the size of the test cycle, every customer should be delighted with the test cycle experience from activation to close.

TTLs and SDLC

A basic understanding of the Software Development Lifecycle (SDLC) is very useful for every TTL. The more confidently that a TTL understands the different stages of the SDLC the better that the TTL can prepare and set appropriate expectations for the testers on the test cycle. In general, applications that are tested earlier in the development cycle (like an alpha release) have code that is relatively untested, and testers can expect to find more bugs. On the other hand, applications that are being tested later in the development cycle will have more stable code with fewer bugs that are more difficult to determine and more critical to the live release of the software. Please refer to the Wikipedia definition of the SDLC to begin your research on this topic.

Conclusion

The term TTL is a uTest-derived acronym that stands for ‘Test Team Lead.’ In a Test Cycle, a TTL is the bridge that connects Project Management, Testers, Customers, and Community Management and assists in increasing the uTest value to the customer and the testers.

TTLs have 3 primary tasks in a test cycle:

  • Platform communications
  • Triaging Bugs
  • Triaging Test Cases

Be sure to confidently understand the standards and expectations listed in this handbook before engaging in any test team leadership activities. Because it cannot be repeated enough, the goal for the TTL is to increase the value that uTest brings to the customer by maximizing the output and minimizing the noise level. As a TTL, we are placing a great deal of trust in your technical and leadership abilities to help manage each test cycle. Therefore, please do not hesitate to reach out to the CM Team if you do not believe we are providing appropriate training or expectations for all parties involved in order for you to become successful and for our customers to extract the highest value from uTest. Lastly, feel free to reach out to the uTest Community Management Team if you have any questions or concerns with current or recent TTL assignments.

Recommended Next Steps

Once you feel comfortable with the information contained within the TTL Handbook you may request access to the TTL Entry Level Exam by emailing TTL@utest.com.

  • Participate in on-line training sessions as they are available.
  • Further training may be required:
    • Rules about exporting bugs:
    • CSV
    • Bug tracking systems
  • Communications with PMs
  • Communications with CM

Addendum

Chat Examples:

Example: — New Chat Topic – Welcome message:

When the test cycle launches, the TTL should introduce themselves as the Test Team Lead for this test cycle in the Chat Topics. This notification establishes a point of contact testers can reach out to with their questions and observations. A generic welcome message is provided below. Amend as appropriate:

Welcome to this<Project name> cycle. I will be your Test Team Lead on this project. At any time, please let me know if you have any questions or concerns. I will try and find an answer for you as soon as possible.

Please ensure that you read the scope and instructions carefully. Check the Out of Scope section and known issues list before starting your testing effort.

In addition, make note of other bugs raised during the cycle to assure that you do not raise duplicate bugs.

When raising issues, please ensure that the bug titles and descriptions are of the highest quality. They should be descriptive and clear. The bug title in particular should be descriptive enough to explain the area and issue so it is easy to spot when going through the bug list.

Please be sure to include all mandatory information for the bug as required.

Thanks a lot and happy testing

<TTL’s Name>

Goodbye Ryan

Back in June 2015, my good buddy an uTest Community Manager extraordinaire Ryan Arsenault decided to leave Applause in pursuit of other opportunities.

Ryan wrote a farewell post to the uTest community that showed what a classy guy he is; always willing to help or just chat with testers, even late in the evening and always working to keep our testers engaged and improving.

Thanks Ryan, I’ll miss you!

 

Which Apps Do You Use Most? A Chat With Lucas Dargis

The uTest community management team asked me to write a blog post discussing the different apps I have on my phone and how I use them. I guess they thought it would be interesting to show what apps an experienced tester has and how they’re used. I gave them a bit more then they bargained for 🙂

The post ended up being one of the most popular of the year, being viewed and shared thousands of times.

 

 

 

For some reason, the Community Management team thought some of you might be interested to know which apps apps I have on my home screen. It took me a week to figure out what a home screen was and then another 12 days to figure out what an app is, but after some extensive research at the local bus station library (they have a stack of Nickel Saver magazines) I found a nice 1989 Toyota Corolla with under 350,000 miles… SCORE!

Messages – This is a cool app that I just got last week. I mostly use it to communicate with outer space and my mom.

USA Today – I’m a news junkie, and this is my go-to news app. Here I can keep up with the the politics in Czechoslovakia (they make the best hockey pucks).

Photos – Where I store all my pictures of “cats in boxes”. Also I have a few pictures of some short people that look a lot like me, but I never can remember their names.

Camera – I have this humanitarian goal of being the person with the most selfies so this app gets A LOT of use.

Skype – I really liked the “bloooop” sound when I got a message. I hope I get another one some day.

ESPN – I mistakenly thought this app was for Emerging Smart Person’s Neckties. But apparently its some app for keeping up with your favorite snooker players. I just haven’t deleted it yet.

Netflix – You’re supposed to be able to get DVDs from Netflix. Once someone can explain to me what a DVD is I’ll probably use this app more.

Maps – My house has 3 bedrooms, how else am I supposed to find my way around?

Clock – This app confuses the heck out of me. I mostly keep it as a reminder that you are never too old to learn new things.

Pandora – The best place to listen to the same 8 songs over and over and over and over and over and over and over and over.

The Weather – The thing I like best about weather is it’s always there!

Settings – I really have no idea what this app is for, but the gear picture looks pretty serious. I’m thinking if I touch it, my phone might transform into a toaster oven so I just stay clear of that one.

FaceTime – I think this is some kind of wormhole app where you can see into different dimensions. I’ve been trying to throw pennies through it, but so far whoever (or whatever!) is on the other side just throws them back at me.

Notes – This is where I write down all my big ideas. I don’t think I’ve ever opened it.

White Noise – I like to take little power naps while I’m driving to the office. This app soothes me right to sleep.

Mobile PM – I’m a Project Manager and my boss made me install this app. He’s such a meanie!

At Bat – This app has a lot of numbers on it so I used it to practice counting.

Football – Both of my friends (well, my friend and his friend…. actually, it was my neighbor and his friend) told me this was a cool app. I think it has too much green on it so I only use it when I want to imagine I’m looking at grass.

Travel – This just freaks me out. How did all those apps get so small!!?? There is no way I can tap on those tiny things. But they do help when I practice being a giant. GRR!

Fitness Buddy – I installed this app in the hopes that it would encourage me to encourage my wife to exercise more, but unfortunately she threw her sandwich at me, so now I just ignore it… Unless I’m hungry.

Bio – Lucas Dargis is an Enterprise Project Manager for Applause. When he’s not replacing the cheese in the mouse traps in the basement he enjoys eating the cheese in the mouse traps in the basement.

Later Ben Melito added his own humor to the mix:

http://blog.utest.com/2015/09/08/which-apps-do-you-use-most-a-chat-with-our-community-manager/

2014 Year in Review

I’d love to write a witty post recapping the last year, but right now I just don’t have the time. So a brief summary will have to do for now.

Job Change

In February, I decided to leave Brock Solutions and work for uTest as a full-time consultant. I spent the next 8 months working for SalonCentric (a L’Oréal devision) as a business/test manager as they built and launched their new eCommerce website. That was wonderful time. I make some fantastic friends and learned a lot about testing eCommerce, particularly on the Demandware platform.

I was promoted to an Enterprise Test Team Lead which is essentially a contracted Test Project Manager. I was assigned several customers and project all of which were delivered successfully and received praise and accolades.

uTest

University – I wrote or co-wrote several courses for the uTest University

Forums – I stayed quite active on the forums as usual. I’m almost at 2,000 posts.

  • I started a really fun series titled Analyze This where I posted a testing “truth” and we debated it. Although technically I started this in 2013, the majority of the posts happened in 2014. So far I’ve started 13 topics, all of which earned at least 2,000 views and several are approaching 10,000!
  • I also made a very controversial proposal to change the uTest rating system to eliminate rating hits. There’s a lot at stake so I understand uTest’s hesitance to move on it, but I still think this is the right thing to do.

Blog – I wrote a few posts for the uTest blog, two of which were selected for The Best of 2014: Top Posts from the uTest Blog. “Certified and Proud?” was one of uTest’s most popular posts of the year, earning the largest number of comments and social shares. Even a few A-list testers read and talked about it!

A.C.E – uTest started a new initiative to provide testers with organized, mentor-based training. I helped put together the pilot program and was one of the mentors for it. A large portion of the materials we used were things that I’ve written for uTest over the years. I’m now working on converting all of my previous articles and papers into more structured courses as the A.C.E. program continues to evolve and improve.

Awards – uTest created a Hall of Fame website and introduced quarterly awards based on votes from the community. In the two quarters that this as been going on, I’ve won 3 awards

  • Outstanding Forums Contributors
  • Outstanding Blogger
  • Outstanding uTest University Instructor

AST-BBST

In September, I successfully completed the Foundations Course. That was quite an experience. I hope to find some time to write about it one day.

In November, I passed the Instructors course and will be an assistant instructor for the first Foundations class of 2015.

uTest’s Testers of the Quarter (2014 Q4)

badgeTesterOfQuarteruTest recently announced the Testers of the Quarter winners for 2014 Q4.

This quarter I took home the hardware for:
Outstanding University Instructor because of the Exploratory Testing course I co-authored with Allyson Burk.

uTest’s Testers of the Quarter (2014 Q3)

Back in August 2014, uTest launched a brand-new community-wide recognition initiative badgeTesterOfQuartercalled Tester of the Quarter“. The goal of this quarterly program is to recognize and award the rock stars of our global community. The program is based on nominations and votes from members of the community.

Here are the categories:

  • Outstanding Forums Contributors
  • Outstanding Bloggers
  • Outstanding uTest University Instructors
  • Outstanding Tool Reviewers
  • Outstanding Project Managers of the Quarter
  • Outstanding Test Team Leads, Tester’s Choice
  • Outstanding Testers, Test Team Leads’ Choice
  • Outstanding Up-and-Comers, Test Team Leads’ Choice

Here’s the announcement for the winners for 2014 Q4:

Yours truly won the award for:

In addition to the quarterly awards, uTest also created a Hall of Fame to honor the top testers in our community…past and present.