How to Accurately A/B Test Your Marketing Content

You are probably asking yourself, what is the point of A/B testing? You know your product, your company, and your market. You don’t need a bunch of charts and graphs to tell you how to design your website, what products to offer, or how to adapt your emails, landing pages, or calls to action.

You’re right, in a way, you do know your company and product better than anyone else. However, the modern marketplace is more competitive and cutthroat than ever before. Customers are less loyal to companies and brands; customers rely on online reviews, recommendations from friends and “celebrity” YouTube stars more than they do on traditional marketing.

However, that doesn’t mean you should slash your marketing and trust your company to the Three Fates. Rather, it means it is time to modernize and embrace data-driven marketing practices. Customers need more; they need marketing to be informative but also funny, short but long enough to draw them into the product, expressive but not overwhelming. The old days of navigating these “tastes” by gut and experience are over. Each customer and market are different, and data enables you to identify and target your customers with more precision than was ever dreamt possible.

How do you A/B test content?

It is called A/B testing because you are testing one variable. You decide on one variable to examine, a color, design, image, layout, and compare it to your “control” group. The “A” team is your control group, and the “B team is your team with the one variable change.

Based upon the differences in the response rate, click rate, and other measurements – you can determine if the variable improved or detracted from your campaign. Eventually, without enough results, you can begin crafting specialized campaigns for subsets of your customers that are more efficient at converting leads.

How do you know if it worked?

Much like testing the one variable, you need to determine what you are trying to improve. Are you trying to get more subscribers to your email list or your weekly blog post updates? Are you trying to improve your conversion rate? How about shares on social media?

Decide how and where you want to improve your company. Armed with this information, you can measure the success or failure of your new campaigns.

What is a control?

The “control” is the original marketing technique. For example, if you want to change your call to action, the “control” is your original call. The variable is the new call that you are testing. You then measure the difference in the response rate between the two groups to determine if your campaign worked, failed, and how it can improve.

Test Groups

Your test group is a subset of your customer base. However, it is very easy to corrupt your results. Therefore, it's crucial that you send these two groups the same amount of random traffic. For instance, if you swamp your “A” team more than your “B team, then it is possible that the increased web traffic could influence the results of your “A” team, rather than the change in your marketing technique.

In short, you need to treat both teams as close to the same as possible, except for the one or two variable changes. It is incredibly easy to corrupt your results accidentally. Therefore, it is crucial that you carefully monitor your marketing campaigns.

Once you perfect your A/B testing procedures, you can test every aspect of your marketing campaign. You will know if your call should go last or first if your new logo is favored or not, or what colors to use. A/B testing brings the power of scientific method to your marketing campaign. Done properly, it will allow your company to take advantage of the new “data” driven economy.

About the Author: Samantha Anderson

Samantha Anderson

Sam is a co-founder and inbound strategist at 41 Orange.