Strong tester participation is at the heart of every successful Customer Validation test. When you’ve recruited enthusiastic, target market testers to use your product, complete surveys, and submit feedback, you have everything you need to deliver those highly sought after recommendations for improvement.
But while everyone agrees that engagement is critical, only half of customer testing professionals say that they’re regularly tracking project participation. A major factor is the lack of processes and knowledge around measuring tester participation. High tester participation depends on consistent and continuous engagement throughout each phase of your project. Along the way, it’s critical to track just how — and how often — your testers are engaging in your test.
We’ve created a worksheet that outlines a structured approach for measuring project engagement using two types of metrics: frequency and activity. Download the worksheet or keep reading to learn how to apply these metrics and monitor your progress so you have the skills to establish your own baselines, identify opportunities for improvement now, and set realistic goals for the future.
Measuring Frequency of Engagement
Frequency metrics show when and how often your testers are engaging with your product and project. This data lets you identify where and when your testers’ energy is waning. It also allows you to anticipate and respond to tester drop-off with additional communication measures (like reminders) and guide your testers across the finish line.
Examples of Frequency Metrics
- Number of daily or weekly active users (DAU/WAU)
- Number of testers logging into the project space
- Number of comments or votes each week
- Number of testers who are completing surveys
- Number of testers who are submitting feedback
You can expect a steady decrease as your project progresses and your testers have surfaced most of the critical issues. Sharp declines early into your test, however, indicate something else (like friction with tools or unclear expectations) might be impacting your results.
Measuring Tester Activity
Activity metrics show how your testers are engaging with your project and give you a feel for the test’s overall performance. After all, submitting bugs isn’t the only way your testers engage during your project. Tracking activity metrics lets you identify and reinforce any weak points in your engagement strategy. Over time, they’ll serve as a vital tool for both increasing tester engagement and showcasing the value of your project (and program) to your stakeholders.
Examples of Activity Metrics
- Percentage of daily or weekly active users (DAU/WAU)
- Percentage of testers logging into the project space
- Percentage of comments or votes each week
- Percentage of testers who are completing surveys
- Percentage of testers who are submitting feedback
How Frequency and Activity Work Together
Together, your frequency and activity metrics paint a holistic picture of your project’s progress both at the moment and over time. Taking five minutes each day to make note of those numbers pushes a range of long-term, positive outcomes, from optimizing your engagement strategy to backing up the value of your beta testing efforts with data.
Below is an example of how you can track both your frequency and activity metrics.
Looking at the number of unique interactions your testers have with your project throughout the week gives immediate visibility into any changes or disruptions that might arise. At the end of the week, take those numbers and divide them by the total number of testers to calculate the percentage of testers engaging in each category.
Tracking engagement metrics across generic, disconnected tools like spreadsheets or email can get very time-consuming, very quickly. Learn how the Centercode Platform is helping professionals at Bose, Honda, Intuit, Autodesk, and countless other industry leaders break down information silos and keep their customer testing data organized and visible.
At the end of the week, take your daily engagement totals and plug them into your activity metrics chart. This allows you to set a baseline for progress over time, and return to it to gauge performance with each new project.
If you don’t yet have your own baseline (or you’re curious about engagement performance on a larger scale), you can compare your numbers to industry benchmarks. These are based on averages across a range of products and companies:
More Best Practices for Engagement
Was this helpful? Get a full suite of resources for increasing and maintaining tester engagement — from diagnosing common challenges, to guiding engaging testers at each phase of your project — by downloading the Tester Engagement Pocket Map.
