All Collections
Report and Stats
A/B Testing Campaigns
A/B Testing Campaigns
Vincenzo Ruggiero avatar
Written by Vincenzo Ruggiero
Updated over a week ago

What is A/B testing?

A/B testing (also known as bucket tests or split-run testing) is an experiment with two variants, A and B, all parameters -but one- being equal, used to see what works best.

The golden rule of A/B testing: only tweak ONE element at a time. It may seem obvious, but if you go ahead and modify more than one feature of–in this case–your email, it’ll be impossible to identify what caused the increase (or the drop) in opens, clicks, or any other KPI you want to improve.

Before launching a large-scale campaign, you might want to test on a sample of your target in order to see what works and maximize the effect.

Know your KPIs!

There are traditionally three ways to measure the success of an email campaign:

  1. Open rate: Do prospects open my emails?
    The math: Total emails opened/Total emails sent * 100

  2. Click rate: Do prospects click my links?
    The math: Total links clicked/Total emails sent * 100
    or Total links clicked/Total emails opened * 100

  3. Reply rate: Do prospects reply to my emails?
    The math: Total emails answered/Total emails sent * 100
    or Total emails answered/Total emails opened * 100

For the last two metrics, make sure you always use the same denominator.

Run A/B testing in Overloop

At this time, Overloop doesn't provide a built-in feature to run A/B testing. The best way to run this kind of test is to duplicate the campaign you want to test, make the change, and compare the results in the reporting section.

To see the comparison report navigate to the Reports page and switch to the Outbound tab.

Scroll down and select the needed campaigns (at least two) and click Compare Campaigns to see the report.

Did this answer your question?