Skip to main content
How does an A/B test work?

This article tells you how an A/B test for an email works.

Francine avatar
Written by Francine
Updated over 6 months ago

With A/B testing an email, you test what works best. Is it a blue button, or a green button? Does a clear subject work best or does a teasing subject generate more opens? You'll learn with an A/B test.

An A/B test always consists of two versions of the same e-mail: a version A and a version B. Both versions differ slightly from each other, depending on the part you want to test. At the end of the A/B test, the winning version (the best performing e-mail) is sent to the remaining group.

You can test on three metrics:

  • Open ratio

  • Click ratio

  • Unsubscribes

Tip: test one element at a time. If you want to test all three degree measures, weigh them all equally important. So each measure weighs 1/3. Are you testing two measures at the same time? Then also weigh them equally against each other by 50/50.

A/B test grade tips

We would like to give you some tips on what to test for:

  • Open ratio. To test your open ratio let the sender name, email subject or pre-header differ from each other. Always test one of the three. Otherwise, you'll never know which change was the deciding factor in picking the winner.

  • Click Ratio. We measure the number of clicks in your newsletter. Your click ratio is about the content of your newsletter. Make the content of variant A different from variant B. Change only a button, hero image or text. Which variant gets the most clicks? Test it and make your content match your target audience.

  • Unsubscribes. We measure the number of unsubscribes per newsletter. Many people unsubscribe from the newsletter because the content is not interesting. Try to put yourself in your customer's shoes: why would you unsubscribe from a newsletter? Are there no interesting offers in your newsletter? Or is it because of the articles themselves? Test why your customers unsubscribe from the newsletter.

Did this answer your question?