Fog Creek

7 Ways to Get More Out of Beta Testing

The weird and wonderful bugs that get thrown up when real users first start using your code never ceases to amaze. There’s always some odd edge case that had been overlooked, despite you having thought about little else for several weeks. We’ve been through this many times whilst developing FogBugz, Kiln and Trello. Beta testing is an important stage in the testing process. To learn a little about them check out the following video, then read on for our key tips:

Here’s 7 things you can do to get the most out of our your beta tests.

1. Ask for a commitment to provide feedback
Response rates will be higher if you ask your beta testers upfront to commit to providing feedback. This doesn’t have to be formal, just part of an application form. But having agreed to it, people are more likely to follow through.

2. Do not release with known bugs
Most beta testers will only provide feedback once. So you don’t want to burn any tester to just hear about known issues.

3. Allow enough time
Use the following as a rough guide. For a major development effort, say about a year’s work, then you’ll want 10-12 weeks for beta testing. Decrease as necessary – so if it took a month to develop, then around a week will suffice.

4. Be feature complete
Only beta test when you’re feature complete. Adding in things as you go sets you back to the start. Otherwise, it just means the new code and its impact on existing functionality isn’t as well tested as the rest. Something you’ll regret later.

5. Make it easy to get in touch
You want to make it as easy as possible for your beta testers to provide feedback, so give them direct emails and offer to jump on a Hangout/Skype if they’d prefer.

6. Remind, but don’t annoy
Whilst your product might be front and center for you, it’s not going to be that way for your beta testers. You’ll want to remind them along the way. But don’t over do it, they’re helping you out so you don’t want to annoy them with too many emails.

7. Don’t forget to provide feedback
Make sure during and after the test that you send them updates about how you are putting their feedback to use. People like to hear that their time wasn’t wasted. And don’t be tight with the swag – a free t-shirt can do wonders!

To learn more about beta testing be sure to check out the video above.