Recently we talked about how bug reports are used in beta testing to further quality and support goals. That post looked at what makes bug reports challenging to collect and how you can ensure you’re getting the data your team needs to fix those critical bugs that seem to pop up during most beta tests.
Being one of the fundamental aspects of beta testing, bug reports end up being a part of almost every beta test run on our beta management platform. As a result, this post is going to look at how you can craft your bug reports in Centercode to get the right data from your testers into the hands of your internal team.
How Bug Reports Work in Connect
Bug reports have three different pieces to consider: the form itself, the workflow, and the integration with your existing bug tracking tools.
The Form
Submitting bug reports is one of many different activities built into the project template that comes with every license of Connect. The bug report form in the template includes the following fields:
- Title
- Category – where testers can indicate what part of the product is affected, such as the user interface, documentation, or installation.
- Detailed Description – where testers can give you details and steps to reproduce the issue.
- Importance/Severity – how important the tester considers this bug to be to their experience.
- Related Device – here the tester can indicate which of their devices they were using, so all the relevant information about that test platform is associated with the bug report.
- Attachments/Related Files – where the tester can attach screenshots or videos of the bug.
- Status – to indicate where the bug stands in your process.
- Comment Log – so you can ask for clarification or just further discuss the bug with the tester.
As with other features in Connect, this form is completely customizable and is just a starting place for our customers to think about what information they want to collect. In addition to deciding what’s on the form, you can decide which aspects of the form are visible to your testers and what only your internal team members can see, allowing you to add valuable information to the bug report without clouding the tester’s experience.
The Workflow
The workflow outlines the process a bug report goes through after it’s submitted. It’s incredibly important to plan out your workflow before your test begins. Consider which members of your team you want to handle new bug reports when they come in. You can create notifications for different members of your team based on anything in the form, so if you want your project manager to be notified every time a critical bug is reported (according to the importance field) you can set that up. Or if you want your QA to triage all bugs, they can be notified.
You’ll also want to consider what different statuses you need for your bug reports, such as “Needs More Information”, “Assigned to QA”, “Duplicate”, or “Fixed”, and what you want to happen when a bug is assigned that status. You can notify different teams or individuals, assign ownership to different people, or send the bug report to an external destination, which brings us to discussing integrations.
The Integration
Centercode is designed to integrate with most bug tracking tools, such as Jira or Bugzilla. You can decide what information is sent to your external tool and when. For example, if one of your team members is responsible for reviewing new bugs within Connect then you can set up your workflow to automatically send the reviewed bug report to your external bug tracking tool when your team members changes the status to “Verified”. You could also set up the workflow to automatically send all bugs with an importance of “Critical” straight to your bug tracking tool. It’s completely up to you.
Tips for Collecting Bug Reports
As we discussed in our previous post, bug reports can result in an overwhelming wave of data. These tips can help you use the features of Connect to build an efficient bug management process.
- Keep bug reports separate.
It can be tempting to just have one general feedback form where testers can submit all their feedback, but this is can get messy quickly as your team is trying to sort through feature requests, general impressions, and use cases to find your critical bugs. Therefore, it’s usually best to have a specific bug report form and direct your testers you submit all possible bugs through there. - Decide how triage is going to work.
- Do you want all your bugs to go straight to your bug tracking tool for your development team to sort through? Or do you want some of your internal team members to act as the first level of support — reviewing, clarifying, and verifying feedback before it’s put onto your dev team’s plate? Either way your workflow needs to reflect your plan.
- Mimic your QA process as much as possible.
Most quality assurance teams have a process for submitting and handling bugs. Finding out what data they collect and crafting your bug reports and workflows to mirror their existing processes can make things run a lot more smoothly once your test is underway. - Use notifications.
There’s a lot of information coming in during a beta test. Don’t expect your team members to know which bugs need their attention. Notifications are an easy way to make sure the right people know about a bug at the right time, so be sure to use them. - Provide a little training.
One of the biggest challenges of bug reporting is getting clear, useable data from your testers. Consider including some documentation in your project on what makes a good bug report, so your testers know what you expect and can give you a clear and complete bug report on the first try. - Attach test platforms.
In our community, testers give us detailed information about all the devices they own when they join. They can then attach this information to each bug report. This way you know everything about the laptop on which this bug occurred without the tester needing to add those details to the bug form. That sort of detailed information can be gold to your development team as they try to reproduce and fix the bug in question. - Make it a conversation.
Include a comments log at the bottom of your form so you can discuss the bug with the tester. This’ll make it easy to ask for clarification, suggest a workaround, or have the tester confirm a fix.
Bug reports are one of the most valuable aspects of beta testing and can provide the most quantifiable return on investment (ROI) for your beta program. Hopefully this post has given you an idea of how bug reports work in Connect and provided some useful tips to get the most out of them in your next test.
