We’ve been providing managed beta tests as a service to companies of all sizes for more than 10 years now. We’ve managed tests for countless product categories, from mobile apps and massively multiplayer games, to next-gen thermostats and enterprise hardware. At this point I think it’s safe to say that we’ve managed a wider variety of beta projects than any other company, ever.
Each year we listen to our customers, and further adapt our offering toward what’s most valuable to them. This year, more than any other, we’ve made radical changes to improve our Managed Beta offering, all of which we believe you’re going to love.
Free Beta Plans
Professional Beta Test Plans have been a staple of our Managed Beta offering since its inception in 2001. In 2012 we improved our plans with a complete redesign (based on our new free Hardware and Software Beta Test Planning Kits). For 2013 we’ve decided to make these plans part of the sales process, replacing our current proposals entirely.
What this means is that we’re willing to develop a free beta plan for any company (with no commitment), just for the opportunity to demonstrate our value and expertise. As an additional benefit, we’ll more clearly understand what you’re looking to accomplish during your beta test, and you’ll more clearly understand exactly what we’ll be doing — before any money changes hands.
The plan will be delivered with a mini-proposal that includes our fixed bid for running that project. You can then choose to let us run it, purchase our software and run it yourself, or go a different direction entirely (no hard feelings, we promise).
If you have a beta in your future, please take us up on our free beta plan. I’m confident that you’ll be impressed.
Simplified Pricing Model
Full disclosure: given the diversity of product types, markets, and beta objectives; pricing a managed beta service offering has always been more than a bit challenging for us. That said, we’ve been at this a long time, have a solid understanding of the variables, and have built great processes to handle them. With that in mind, we’ve developed new predictable pricing based on only two axes — duration and testers.
Duration is calculated in weeks, which are based on the number of goals you’ve set. Testers are grouped into tiers of 25, which is based on the number of target markets (or personas) you’re attempting to reach in your beta test. This allows us to not only provide quotes faster, but also provide price charts to our Beta Program customers, giving them the opportunity to more easily budget for their annual beta tests.
In many cases our new pricing model will result in reduced prices, with our fully Managed Beta Tests now starting at $6,000.
Severity & ROI Estimates
Determining the ROI of a beta test can be a difficult task. For one, an effective beta touches and improves nearly every aspect of your product; from more effective sales and marketing messaging, to direct quality improvements, to better support preparation. Altogether these improvements lead to better reviews, higher sales, higher customer retention, fewer returns, and lower support costs (i.e. “why we exist!”). That said, we feel that it’s very important that we directly illustrate the impact our Managed Betas have.
To accomplish this, we chose to focus on defects, which are central to nearly all of the betas that we run — and valuable to everyone. We’re inserting a new step in the planning phase of each beta project, in which we’ll present you with a list of defect severities (from cosmetic to critical), each with an objective definition. You’ll have the option to adjust these descriptions to better match your product, and then also assign a monetary value to each of these types of bugs. In other words, what is a unique critical bug worth to you, if it slips through?
Next, we’ve expanded our feedback filtering routine (one of our standard Managed Beta services) to include Severity Tagging. So, beyond filtering duplicates and ensuring clarity (our current process), we’ll also adjust the severity for each bug to match the definitions included in your beta plan. As a positive side effect, this will also provide additional context for each bug, helping you further prioritize your efforts.
At the end of your beta, we’ll include the result of this (your bugs [by severity] * value [by severity]) in your case study (more on that below) as a simple ROI estimate for the project. This value isn’t intended to provide a total or precise ROI indicator for your beta, but it will serve as a basis to estimate its impact, from which you can adjust to meet your own ideals.
Weekly Status Reports
Similar to ROI, another challenge in our Managed Beta business is to quantify the total time savings our Managed Betas deliver, versus doing it yourself. We believe that this is actually our own fault, by design. Our goal in delivering Managed Betas has always been to absolutely minimize the time investment and effort required by our clients. In other words, you’re hiring us to run a beta not only for our expertise, but also so someone else (“us”) is doing the heavy lifting. The problem with this model is that as a result, you don’t see that heavy lifting, as we’ve gone out of our way to hide it, so you don’t have to worry about it.
In 2013 we have two solutions for this. The first has already been mentioned — our new beta plans more clearly demonstrate just how much work we’ll be doing, that you don’t have to. Our new Weekly Status Reports are the second. Each week we’ll be delivering a standard Status Report which outlines key details about your project, including the state of the severity of all bugs. Note that this is in addition to your ability to ask for whatever data you want, in real-time.
The second goal of our Weekly Status Reports is to provide you with a consistent means to measure the progress of your beta test, which will help you make better decisions in regard to pivoting, extending, etc. — again, ensuring your betas have the biggest possible impact and ROI.
Standardized Final Tester Surveys
Surveys are a common tool used in most of our betas. In the past we’ve helped you develop these surveys, or you’ve provided them exactly as you’d like posted. While these options are both still available, we’re also going to be running a single standard survey at the end of every beta project we manage.
This is a short beta-product-focused survey that we’ve developed as a benchmark to help understand how your product compares to others at the conclusion of beta, in the eyes of its customers. It covers customer satisfaction, quality perception, and NPS (Net Promoter Score).
Mini Case Study (Internal)
In the past our Final Report has been our ultimate deliverable, offering an extended overview of your project. While you’ll likely find great value in the details, your peers and executives would probably appreciate a simple high-level synopsis. To address this request, we’ve created a simple one-page case study template, which we’ll complete and provide at the end of each beta test. This case study is for your internal use only (i.e. we won’t be sharing it with anyone), and you’re encouraged to share it with your executive team and peers.
Our goal is that this document will help you summarize and justify your investment in us, and of course if they like what they see, we’d always appreciate a reference. (:
Tester Incentives by Centercode
Our last change is an improvement to the way participation rewards are handled for our testers. Traditionally beta incentives are built into each project deal. We prefer that testers keep the products (as they often grow attached to them), but quite often other rewards are provided.
While we’re still keeping this system in place, we’re also strengthening it with an addition of our own. While your own incentives will reward the testers who’ve met the participation requirements of the project, Centercode will be providing an additional higher tier for those who went above and beyond.
There are two reasons for this change. First, we feel it’s always important to reward our testers, and given their relationship with us we want to go out of our way to clearly thank the rockstars in each project. Second, we feel this additional tier will stimulate tester engagement even more, providing you with better results — thus adding more value to every beta test we run.
The Whole Process
As part of our recent web refresh, we’ve created a new page which outlines our entire Managed Beta process, including all of these changes. This offers our customers and prospects a clear and detailed overview of exactly what they should expect in working with Centercode.
More Still to Come
This concludes my second tome on our 2013 changes. Thanks for reading! In the coming days I’ll be addressing a couple of items not detailed here, including our new Mobile Managed Beta offering, and our improved Beta Program offering. Stay tuned!
Luke @ Centercode
Image courtesy of Flickr user Thomas Hawk.
