Beta testing is much more than a box to check off on a list. While some companies only think of it as a last-ditch effort to tie up loose ends before launch, others are tapping user research to play a more fundamental role in the product development process.
In fact, today’s most forward-thinking companies are increasing their competitive edge by extending their beta programs in creative ways.
At this year’s Centercode User Group, Sonos Senior UX Research Lead Amy Dexter shared some of that creativity with her fellow customer testing professionals. Her presentation offered real-world examples of how blending traditional user research tactics with Customer Validation is increasing organizational program adoption.
Amy sat down with us for an interview to share more details about the beta team at Sonos and their journey into a proactive, cross-functional force with impact throughout the product development lifecycle.
In February, you gave an inspiring presentation at our Centercode User Group about the journey of the beta team at Sonos. Could you share the overarching theme of your talk?
Amy: At a high level, I shared three monumental changes that the beta team at Sonos has implemented in recent years.
- We have become less reactive and more proactive in our efforts to help the business scale, deliver new products, and improve the customer experience.
- We are repositioning the team to have more touchpoints along the product development lifecycle to deliver greater value to the organization.
- We have increased the number of teams with whom we collaborate to reinforce the importance of the customer experience.
Can you share the history behind the transformation? What sparked the changes?
Amy: Initially, the corporate beta team at Sonos had limited visibility into the activities of the broader organization. At the time, we would not begin working on a product until very close to launch, so we had little opportunity to learn and implement feedback from our testers to improve our customers’ experience.
Our team’s director always envisioned Sonos Beta partnering with many more functions within the organization. She made key changes that allotted the team influence over the product roadmap, allowed us to engage further upstream, and made our team more beneficial to the entire product development lifecycle – and in turn our customers.
I look for opportunities to utilize user research methodologies to learn from our beta community and increase the value our team adds across the organization.
You mentioned that you joined the beta team just last year. What is your role on the team, and how have you helped increase beta’s value to the organization?
Amy: I am the team’s user research lead. I look for opportunities to utilize user research methodologies to learn from our beta community and increase the value our team adds across the organization. For example, I noticed the team getting a lot of last-minute requests to learn from our beta community, which they usually accommodated despite not having the bandwidth.
Given this, I launched an omnibus survey program to learn from our beta community in a more controlled, proactive way, which helped relieve the team from undue stress. This program quickly became an integral part of the team’s mission to move further upstream and gain access to more customer touchpoints. Today, each survey receives more than 10K completed responses.
Anyone at Sonos can submit a research request for inclusion in the survey, which is distributed to our entire beta community every six weeks. These requests help teams form a point of view early in the product development lifecycle and better understand how we can evolve the experience through new features over time.
The wide range of requests submitted to the omnibus program, coupled with my expertise in user research, has led to more ways of learning from our beta community. This includes practices like concept testing and remote interviews.
I launched an omnibus survey program to learn from our beta community in a more controlled, proactive way, which helped relieve the team from undue stress. This program quickly became an integral part of the team’s mission to move further upstream and gain access to more customer touchpoints.
That’s impressive! Launching anything at a large organization can be especially difficult. What is helping your program succeed?
Amy: Although we were moving quickly, we were mindful of scale and sustainability. To scale, we created a detailed overview of the program – including an explanation of what it is, how to submit requests, and a calendar showing upcoming deadlines and survey deployment dates.
Additionally, we created a ticketing system that captures and tracks requests from anyone in the company. This encourages our stakeholders to plan ahead and be as thorough as possible with their requests.
From a sustainability perspective, we initially rolled out the program to a small number of teams, which enabled us to work out the kinks and establish some early wins. Delivering the program to a smaller group of people also allowed us to build internal champions who would eventually evangelize the program and train other groups.
Not many beta teams have members with user research experience. Is this necessary?
Amy: Of course I’m biased, but I believe having a user researcher on a beta team is critical when gathering customer feedback. A gatekeeper of user research helps ensure collected feedback is unbiased and actionable and that we are mindful of our beta community members’ time. Funneling requests through a trained research expert ensures the surveys are succinct, effective, and without bias, which maximizes the reliability and actionability of the results.
Lastly, what advice would you give a leader at a beta team who aspires to move their beta programs further upstream and become more proactive?
Amy: Becoming more tightly integrated with the product development process is an absolute must. Doing so requires regular interaction with the product organization. As you launch new programs, be sure to start small, establish the right expectations, and stick with your program guidelines even if stakeholders are initially reluctant to abide by them.
Becoming more tightly integrated with the product development process is an absolute must.
Although our ticketing system seemed unpleasant to some at first, capturing incoming requests enabled us to analyze the distribution of work across the team and justify future hiring in the right areas.
Thanks for sharing these insights, Amy. We’re excited to see how your beta program continues to thrive in 2019.
More Insights into Customer Validation
Want to hear the unique approaches other Customer Validation professionals are using to boost the value of their programs? Learn how four leading-edge companies are increasing stakeholder buy-in and leadership support in this on-demand webinar.
