Throughout the month, we’ve been looking at best practices for writing surveys — from avoiding psychological biases to tips for keeping your data squeaky clean. This week, we decided to ask our Customer Validation Managed Testing Services Team to weigh in.
Our Centercode Program Managers work closely alongside our customers to manage the ins and outs of Customer Validation, from communicating directly with testers to spinning feedback into recommendations. As survey pros themselves, they shared their thoughts on what makes a well-written survey and tips for maintaining engagement. They also tell a little bit about themselves (and share photos of their pets).
Tom Peelen
Director of Managed Programs
How long have you been at Centercode?
At the time of writing this, I’ve been with Centercode for 4 years, 3 months, and 2 days.
What’s your favorite album to listen to while you’re working?
I mostly listen to heavy metal, but I’ve been on a huge Tool kick the past few months. Their latest release, Fear Inoculum, is great.
What piece of outdated technology do you miss from back in the day?
I miss my Lite Brite.
Do you play video games? If so, what have you been playing lately?
I’m currently finishing up Assassin’s Creed Odyssey Fate of Atlantis DLC.
Two sides of Murphy
What’s one of the characteristics of a well-written survey?
In a well-written survey, every question has a purpose. You get much better results when you know exactly how the data you’re collecting will be used.
What’s a survey-writing mistake you see over and over?
Asking someone, “What problems did you encounter?” Don’t split your bug reports into multiple channels. It’s much easier to analyze and report on the problems your testers are running into by asking them to submit bugs as they find them, rather than waiting for a survey. Instead, use your surveys to find out how your testers feel about your product.
What’s the best way to approach unresponsive or disengaged survey respondents/testers?
Testers are usually volunteers, so be considerate of their time. Reminding yourself that they have a life outside of your test will allow you to be empathetic as you encourage them to re-engage.
Rob Diny
Program Manager
How long have you been at Centercode?
3 years
What’s your favorite album to listen to while you’re working?
Demon Days by the Gorillaz
What piece of outdated technology do you miss from back in the day?
Furbies
Do you play video games? If so, what have you been playing lately?
Classic WoW
What’s a survey-writing mistake you see over and over?
Going into a survey without a plan. Aligning your survey with your test goals in mind helps you steer clear of asking questions for the sake of asking questions.
What’s your #1 tip for building your surveys to meet stakeholder goals?
Make your main survey goal a priority. If your stakeholders want to ask questions that don’t align with your test, think about your testers’ energy pool carefully. Testers get tired and drop out of the survey when there are too many questions, even when those questions are written clearly. A side quest or tangent could burn testers out.
What’s the best way to approach unresponsive or disengaged survey respondents/testers?
It helps to remind them that active participation will be rewarded with priority on future opportunities. Many testers volunteer because they enjoy being part of the development process.
Mike Fine
Program Manager
How long have you been at Centercode?
18 years. I joined Centercode at the same time as the founders, and I helped create the services side of the business back in 2001.
What’s your favorite album to listen to while you’re working?
I could listen to Black Sabbath’s first self-titled album over and over again. But I live on Spotify and listen to different things every day.
What piece of outdated technology do you miss from back in the day?
I’m a “latest and greatest” kind of guy. We’re in an industry that’s constantly evolving and improving. I feel like if development is working well, we won’t miss the old products.
Do you play video games? If so, what have you been playing lately?
I recently finished Spider-man and it was pure joy.
Annie (left) and Maisie (right)
What’s a survey-writing mistake you see over and over?
Using open-ended text questions for things that should have a formatted choice response.
What’s your #1 tip for building your surveys to meet stakeholder goals?
Think about the goal of your survey, first and foremost. What are you trying to achieve? Once you have a simple, clear objective, center all of your questions around that topic.
What has participating as a tester taught you about communicating with testers?
I’ve found that an unclear schedule makes me feel lost — like the project isn’t focused. If I know when the test will end and when tasks need to be completed, I can be a better tester.
Diego Gomez
Program Manager
How long have you been at Centercode?
4 years
What’s your favorite album to listen to while you’re working?
Eric Clapton’s Unplugged album
What piece of outdated technology do you miss from back in the day?
Typewriters. Something about the sound of the keys feels very nostalgic.
Do you play video games? If so, what have you been playing lately?
God of War, Call of Duty, and recently, a lot of N64 games.
Marcus Aurelius
What’s one of the characteristics of a well-written survey?
A well-written survey should have more traceable (multiple choice, for example) questions than open-ended ones (paragraph or sentence style). This way, you won’t have to spend hours reading and coding all of your survey responses.
What’s your #1 tip for building your surveys to meet stakeholder goals?
Get buy-in from your key stakeholders before you share your survey with testers. Even if it takes more than one meeting or interview, knowing the data your stakeholders want and need will save you more time in the long run.
What has participating as a tester taught you about communicating with testers?
Remember that your testers are here because they’re excited to test your product. Testers aren’t robots. When you treat them well, they often become product evangelists!
Drew Kropp
Program Manager
How long have you been at Centercode?
3 years
What’s your favorite album to listen to while you’re working?
Caipi by Kurt Rosenwinkel
What piece of outdated technology do you miss from back in the day?
Floppy discs. I liked fidgeting with the dust cover.
Do you play video games? If so, what have you been playing lately?
I play a couple of League of Legends matches per week.
Henry spends most of his days at Centercode HQ.
What’s a survey-writing mistake you see over and over?
It’s not uncommon to see questions that could be interpreted in multiple ways. Keeping your questions simple and clear will help you make sure your results are useful.
What’s your #1 tip for building your surveys to meet stakeholder goals?
Review your questions with your stakeholders before they approve it for launch.
What’s the best way to approach unresponsive or disengaged survey respondents/testers?
Use a personal contact method, such as phone calls. Putting your voice behind a few ignored emails is usually enough to convince people that their help and feedback matters.
Brandon Leigh
Program Manager
How long have you been at Centercode?
Since April of 2017.
What’s your favorite album to listen to while you’re working?
It depends on my mood, but it’s always loud.
What piece of outdated technology do you miss from back in the day?
Mix cassette tapes
Do you play video games? If so, what have you been playing lately?
I don’t play video games much anymore. On my off-time, I like to hang out with my family.
What’s a survey-writing mistake you see over and over?
Seeing surveys with too many questions. Surveys don’t need to be super long for you to get a lot of information out of them.
What’s your #1 tip for building your surveys to meet stakeholder goals?
Don’t repeat the same question four or five different ways.
What has participating as a tester taught you about communicating with testers?
People want to feel like their input is valuable. A “thank you” can go a long way for anyone who takes time to test your product. The feedback they provide, whether it’s positive or negative, is helpful.
Doug Sorensen
Program Manager
How long have you been at Centercode?
16 and a half years. I was the first dedicated test manager at Centercode!
What’s your favorite album to listen to while you’re working?
One of my all-time favorite albums to listen to is Gorillaz by the Gorillaz. It really gets me motivated.
What piece of outdated technology do you miss from back in the day?
I liked the simplicity of dedicated MP3 players. They were very compact and weighed almost nothing.
Do you play video games? If so, what have you been playing lately?
I still play Diablo III seasonally for extra stash space, transmorgs, etc.
Toby (left) and Luna (right)
What’s a survey-writing mistake you see over and over?
One common theme I see with online surveys is that a lot of them are marathon surveys. They have pages and pages of matrix questions, some of which don’t have an obvious answer. It can take 30 minutes or more to complete.
What’s your #1 tip for building your surveys to meet stakeholder goals?
Make sure the questions you’re including aren’t leading your testers, meaning they’re written in a way that slants their opinions. It’s also important to ensure that your questions don’t demand too much tester energy. It takes much more attention and focus to answer qualitative (open text) questions, so keep them to a minimum.
What’s the best way to approach unresponsive or disengaged survey respondents/testers?
Sometimes, your testers may not understand exactly what you need them to do. If that’s the case, find out if there’s something you can do to help them with their testing.
Michelle Williams
Program Manager
How long have you been at Centercode?
4 years
What’s your favorite album to listen to while you’re working?
I just put everything into a random playlist, iPod Shuffle style.
What piece of outdated technology do you miss from back in the day?
Tactile buttons — ones that click when you press them. Current touchscreen technology falls short in mimicking the experience of a well-engineered button.
Do you play video games? If so, what have you been playing lately?
Tom Clancy’s The Division 2 and Mindustry
Ser Pounce
What’s one of the characteristics of a well-written survey?
Using consistent metrics. It makes comparing your results from week to week a lot easier.
What’s a survey-writing mistake you see over and over?
Asking about multiple topics in a single question. For example, asking testers if they’re satisfied with battery life and screen brightness makes it impossible to answer individually.
What has participating as a tester taught you about communicating with testers?
Be brief and specific with the information you share. No one wants to read a wall of text.
Ready for more tips and best practices on writing surveys? Download a copy of our ebook, Building Surveys for Customer Validation.
