As a research analyst for Centercode, the best part of my job is using data and research practices to uncover insights on the Customer Validation industry. It’s my responsibility to collect and analyze data, then use it to provide recommendations that empower CV teams like yours to grow and thrive. (I know, it’s thrilling stuff!)
Last November, my team conducted a survey of professionals across all verticals of the Customer Validation industry for the third year in a row. With responses from over 240 professionals, we got a thorough look at who’s involved with testing, the challenges those individuals and teams are facing, as well as how they’re making their programs successful.
Analyzing industry data year after year reveals how CV program performance has evolved over time in interesting and exciting ways. In this blog post, I’m going to cover one of the most interesting findings from the 2019 industry survey: how using defined CV processes affects overall program performance.
Follow me as we look at the professionals who are working to define their test management processes and how it’s driving steady improvements to their CV programs.
Defined Processes vs. Reactive Processes
Let’s start with the breakdown of responses from professionals with defined and reactive processes.
Having defined processes means establishing a proactive course of action before you start testing. For example, CV program managers who have defined their processes use a consistent test plan and have a reliable workflow for recruitment set up before they initialize their projects.
Reactive processes are generated in reaction to the challenges of Customer Validation. For example, a reactive CV program manager still devotes time to planning and recruitment strategies, but they have no set course of action for unexpectedly low tester engagement. Without a process to handle it, they have to come up with a solution on the fly.
2018 vs. 2019 Results
2018’s CV industry report looked at how teams using defined and reactive processes spend their time. The data showed that regardless of the process type, the majority of respondents weren’t spending enough time on common beta management activities such as project planning, recruitment, communicating with testers, and organizing data and test results.
However, the 2019 industry survey highlighted some interesting shifts in the data. As you can see in the chart below, those with defined processes have moved away from not spending enough time on test management activities to spending the perfect amount of time. This is nearly a 20% increase in successful time management! Meanwhile, those with reactive processes have remained largely the same, with a slight shift towards spending too much time on those activities.
How to Move Towards Using Defined Processes
Customer Validation has many moving parts, and the factors that influence the effectiveness of a project (or an entire program) vary greatly. But by and large, the data shows that the biggest contributor to consistent success is having defined processes.
In a role where time and resources are already scarce and many are juggling numerous responsibilities, spending time planning and implementing repeatable processes can seem daunting and unimportant. But in reality, taking some time to give your projects more structure empowers you to predict outcomes and invest your time where it will make the most impact for your organization.
If you’re finding it difficult to shift your CV program towards repeatable successes, I’ve outlined a few simple steps to get you moving in the right direction.
- Write down your processes. Written documents provide a framework that you can use at the start of every test launch. They also serve as helpful reference resources for those who manage tests as just one part of their job (rather than as their primary job). Just make sure you’re adding notes and making tweaks as your processes grow and mature.
- Use templates to become proactive. Creating templates helps you streamline and repeat test execution. Surveys are a great place to start with this. By templatizing the way you measure tester satisfaction and sentiments, you allow yourself to concentrate on other value-driving activities like planning and reporting.
- Don’t forget your metrics. Tracking and analyzing key metrics enables you to identify gaps and make improvements on the next go-round. For example, maintaining high participation is the most common challenge while running CV tests. By looking at test participation rates and determining when your testers tend to drop off, you can plan proactive measures — like adding a mid-week reminder email to your cadence.
As a CV professional, there are many times when outside factors — product development constraints, time constraints, company culture, stakeholder priorities — will influence your program’s performance. Defining your processes is one way you can take control of your program in an otherwise unpredictable environment. By taking even small steps to increase the consistency of your customer insights and recommendations, you’re steadily closing the gap between your audience and your organization in a meaningful way.
Don’t stop here. Learn more about best practices and where the CV industry is headed, check out our annual Customer Validation Industry Report.
