As a beta manager you want to get as much relevant feedback from each tester as you can. Incentivizing and rewarding testers for their participation is a key part of getting that feedback. As a result, it can be tempting to attach a dollar amount to specific types of feedback, giving testers cash for each critical bug or for completing all your requirements. This way you have happy, involved testers and you don’t feel bad about asking them to spend time using your product.
While this seems reasonable on the surface, paying your testers for feedback changes the dynamic in a subtle but incredibly important way. It switches their focus from the product to the money. This creates a couple of key issues for you as the beta manager:
- They may lie to get into your test. If testers know that there is money involved, they may claim to meet your requirements just to get into your test. That “Senior Engineer” could really be a broke college student, and you could be collecting the opinions of people who aren’t really members of your target market or genuinely interested in your product.
- Trivial issues are reported to earn cash. Testers will be looking for problems, instead of reporting problems during their normal interactions with the product, so they may report small issues and try to make them sound more egregious than they are. This means more noise for your team as they try to find the truly critical bugs.
- The bugs they report may not be real. Some testers may take this a step further and report bugs that don’t even exist, wasting your development team’s time as they try to recreate bugs that aren’t real.
- They may give the product false praise. If testers are given a cash incentive for their participation, they may try to give you the feedback they think you want instead of their honest opinions in hopes that you’ll pay them more or bring them back for future paid tests.
- Their feedback loses its objectivity and authenticity. Even if testers think they’re being objective, they can’t be if money is involved. One of the benefits of feedback from beta testers is that they’re outside voices volunteering their opinions on the product. If you’re paying for their feedback, they become employees rather than volunteers. This calls into question the authenticity of any feedback they give.
When Paying Testers Can Work
If your only motivation is to find bugs, then paying for beta feedback can be useful. In that case you’re really just enlisting the public to be your QA testers, rather than running a beta test. A true beta project not only uncovers real bugs, but a wealth of other useful data that will help you release your product successfully.
Rewarding Testers Without Using Cash
We’re not suggesting that you don’t reward your testers for their hard work. Your testers put a lot of time and energy into your tests and they deserve a big thank you for everything they’ve done to help improve your product. That said, incentives should be looked at as a thank you gift after the test rather than the key motivation. Also, instead of cash we suggest giving products, t-shirts, gift cards, and/or personal thank you notes. This keeps your testers’ focus on the product, rather than profit.
For more tips check out this post on tester incentives. For other ways to encourage tester participation, consider downloading our eBook on Reaching 90% Participation.
