A/B Testing

What is A/B Testing?


The primary principle behind A/B tests is creating two versions to see which version users respond favorably to - Version A or Version B?

Questions to Consider

Why is it important?


AB testing has proven to increase profit, revenue, and help narrow your target audience, as seen in the following two studies:


  • In a study conducted by Indow Windows, a custom window inserting company, A/B testing was utilized, stressing awareness rather than focusing on conversions to address their recent dip in revenue. Though a vague variable, Indow Windows experienced an 1800% increase in lead volume and a 94% decrease in cost-per-lead in only two weeks following the change they made according to the A/B test.


  • In a case study done by Harvard Business Review, a Microsoft employee discovered a new way to layout the headlines on Bing. Utilizing AB testing to measure if this was an improvement from the original display, the company found a 12% increase in revenue.


These studies have proven that A/B testing can help target and narrow your audience by decreasing the cost per lead enhancing your profit.


To visualize, think about fishing with a net. With the results you gather from A/B testing casting a “more efficient net.” This “net” allows you to gauge what your audience likes in your advertisement, which is why there was an increase in revenue in the Harvard Business study.


Your Pathway to Benefits

How to Maximize Effectiveness


A/B testing is quickly becoming a frequented tool in the business industry due to its low cost and high impact. It’s overwhelming benefits are a no-brainer to most online-competing companies. With this, creating an effective campaign is just as important as utilizing the tool. For A/B testing, certain aspects can make or break a successful campaign.


Step 1: Research

Research serves as the foundation of any test. By using existing site data, you can identify what a page does well, and where it’s struggling. There are plenty of resources at your disposal to scrape this information. Some include:

  • Heatmap tools

  • Google Analytics

  • Session Recording Tools


Step 2: Create Hypothesis

Formulating a hypothesis is a great way to give direction to your campaign. In essence, it will serve as the backbone of your testing. Using the data gathered in the previous step, analyze, observe, and look for patterns. The final result should be something imperative to your larger goals and easy to set up.




Step 3: Variation


Variations want to create a noticeable difference, but not a significant change. Including too many metrics can hinder the effectiveness of your testing. The key is isolating one to two variables. These variables can include:


  • Color

  • Layout

  • Image Formatting

  • Number of Images

  • Text


Step 4: Run Test

The testing method is just as important as testing itself, and should reflect the analysis and trends found during step 2. When creating it, make sure to be as detailed as possible. Factors to keep in mind include.