How to A/B test email subject lines?
To create your A/B test, access the Engage > Email menu and click Create Email as you would do to create a regular Email Marketing campaign.
The only difference is that, when selecting an Email Marketing campaign or an Automation Template/Workflow, there will be a third option called A/B Test Email Subject Line. Select this option, enter the name of the campaign, and click Create Email:
Setting up Subject Line version A and B
When editing the email, you are going to find 2 email subject line fields instead of 1. Click either of the email subjects to edit them. Just enter which will be version A and version B:
Then, finish creating and send or schedule your email campaign as usual.
Is it possible to select which recipients will receive version A or version B?
Currently, it is not possible to do this in RD Station Marketing. Of the Leads selected to receive the campaign, 50% will be randomly chosen to receive version A and 50% will receive version B.
Thus, you will be able to use the results of the experiment to improve your email subject lines for future campaigns.
How can I see the campaign metrics for version A and version B?
To see the performance of each version, click the campaign name on the Email Marketing Homepage and check the Statistics tab.
A/B campaigns have a select button at the top where you can choose to see campaign statistics for version A, version B, or both:
Making sure your results are statistically significant
If you toss a coin twice and it lands on tails both times, you know that this did not happen because the coin was biased toward tails, since tossing a coin only twice is not enough to guarantee a reliable result. You need to have a larger sample and toss the coin more times to determine whether or not it is biased. This is called statistical significance.
It is the same with email campaigns: if you send an A/B Test to 10 Leads, you are not going to be able to determine which email is better. To ensure statistical significance and reduce the chances of obtaining random results, a few measures must be taken:
- The greater the number of Leads that interact with your email, the more likely you are to achieve a statistically significant result. Therefore, we recommend that A/B campaigns are sent to at least 1,000 engaged recipients;
- You should wait a minimum amount of time before deciding a winner. We recommend that you wait at least 4 hours after sending the campaign. However, even 4 hours may not be enough, as email campaigns require around 2 days before most Leads receive the email and check their inbox;
- Use a statistics calculator to check if your results are significant.
If the result does not show a statistically significant difference, you must wait longer or run another test with more recipients, because this may be a random result.
If the calculator determines a winner, congratulations! You have discovered the type of email subject that works best with your recipients and can start applying this insight to future email marketing campaigns.