Should you plan to launch a user testing project over the holidays? In this installment of Ask the Expert, Centercode's Sr. Test Manager Mike Fine talks about the oft-debated holiday test launch: when it works, when it doesn't, and what you should expect from your testers.
Got a question you’d like to see on the Centercode Blog? Shoot an email to help@centercode.com with the subject line, “Ask the Expert”!
Meet Today's Expert
Mike Fine is a Sr. Test Manager at Centercode and one of its co-founders. He manages dozens of tests each year for some of the largest manufacturers in tech. He also teaches LinkedIn Learning courses on the foundations of software testing; his nine courses have educated more than 60,000 learners. His superpower is his ability to juggle hundreds of tasks without breaking a sweat.
Should You Test Over the Holidays?
Thanks for your question, TC!
It's natural to assume that since so many people have time off during the holidays, it’s the perfect time to start a beta. And while there are benefits to it if you've got a certain audience (more on that in a bit), it's usually not a good idea to start a user testing project over a holiday break. Let's talk about why.
The biggest challenge is unpredictability. Family visits and planned vacations, negotiating school and work schedules, and even shiny new toys are all major distractions from a user testing project. This can lead to abrupt drops in test interest and participation. And without the right tools to help you out, engagement is plenty to handle without any additional curveballs.
There's also you, your team, and your stakeholders to consider. The unpredictability that can hinder the flow of incoming feedback can also negatively impact your ability to get the project support you need. If people are out of the office or scrambling to tie up loose ends before Q1, it's likely to impact your ability to get everything done on time.
When Holiday Testing Works
That doesn't mean that the last week of December, Hanukkah, or other special days are guaranteed to tank your test. Depending on your target audience, testing over the holidays can actually benefit you. For instance, many people are cooking and baking more than they would throughout the year, which makes it a great time to test cooking appliances. And while families tend to be busy, younger demographics usually have more time. Gaming testers do well during extended holiday breaks.
If You Have to Test Over the Holidays
We all know how this goes. Sometimes, delays or other changes to the launch timeline mean you have to run a user test over the holidays to keep everything on schedule.
In those cases, engagement automation gives you an amazing leg up. In addition to sending reminders about project timelines and due dates, engagement automation uses machine learning to adapt interactions based on the behaviors of each individual tester. It not only communicates with testers on your behalf, it learns how to reach out at the exact right moment, whether that's right when your testers wake up and check their email or just after the kids are asleep.
The best advice I can leave you is to have a discussion with your team and stakeholders before you send that launch email on December 23rd. Think about it really carefully: what will your target audience be doing during this time? Will they have the time and focus to provide the feedback you need? Experience has taught me that rescheduling your beta test to begin even a few days after a major holiday can do wonders for your testers' focus and the payoff of your efforts.
Good luck,
Mike
Enjoy Your Holidays
With engagement automation to keep your testers engaged and the rest of the delta testing system working to keep your feedback organized and prioritized, you can enjoy your holidays knowing that nothing's fallen through the cracks. Learn more about user testing automation over your morning tea or coffee with this ebook, the Delta Testing Quickstart Guide.
