How To Run A/B Testing On Your WordPress Sites

Article written by:
  • Inbound Marketing
  • Updated: Sep 02, 2013

I bet you want to get the most out of your site. Sometimes we take some decisions based more in our feeling than in actual data or tests and there’s where the danger lies.

What if I tell you that you are losing sales because of the color of your button? You’d call me crazy, sure, but how do you make sure that everything in your page is in the optimal setup in order to increase results?

Let me introduce you the A/B testing method.

This simple method help you testing different versions of your site with your real visitors and measuring results, so you’ll know which version is better than the other.

Today we’ll understand more this process and learn how to set this up in your WordPress site.

The A/B Testing Method

As we said before, the main goal here is to test 2 versions of an element in order to define which one you should use.

A / B Test

A key word in this process is conversion (or conversion rate). It measures how many users have achieved the goal defined in that page. So if you have a sales page the goal is certainly click the “buy” button. But you could also have a newsletter subscription page, then your goal would be the “subscribe” button (with a valid email address would be even better :D). If you have a posts listing page you can assume that the goal is getting the user to read one post, and in the post page the goal may be amaze the user so much that he’ll share or comment (hey, multiple goals!).

You get the idea, right? So in the A/B test we measure the conversion rate for the A version, and the conversion rate for the B version so we’ll know which version we should keep after some time, in order to increase our results.

But there are some rules. In order to understand what is causing the difference in the conversion rate you should only test one element at a time. This is important, because you may have conflicting elements, if you change more than one you simply can’t ensure that the A or B is better and the goal here is to have all key elements in their better shape (if you change everything it’s harder to test and analyze results).

Why Should I Test?

One question that may pop once a while is: why should I waste my time on this?

The answer is quite compelling, increasing conversion is the easiest way to improve results. Your final result depend on traffic (how many users you have) and your conversion rate (how many users have done what they should do there in order to make you happy).

Traffic is much harder to build, it depends on marketing budget, SEO knowledge, and social media marketing skills. Conversion on the other hand in your full control. You can change pretty much anything in your site in order to increase the conversion so you’ll have more results with the same traffic.

Why Do I Test Only 2 Versions?

As the A/B name suggests the recommendation is 2 versions per test (the control / original and the variation). But why can’t we test 2, 3 or 4 variations? The truth is, you can but that will require a lot of traffic.

With 2 versions you need small traffic to mathematically ensure that your conversion result isn’t an exception, but the tendency (pretty much the same science that helps us in polls, to define that a sample will represent well the whole).

What Should You Test?

The first thing, dear Padawan, is defining your goals. Of course we want to improve everything but you should define a single element or a small set of elements to test.

Here are some suggestions:

  • Your “buy” button font size, color, background and position
  • Having a moneyback guarantee message or not
  • Adding a “Verisign-like” badge or not (those badges that “ensure” that your site is safe)
  • Labels above the fields, in the same line, as mad-libs (this is quite awesome, we will certainly talk about this in other articles)
  • Having optional fields or suppressing them (and adding somewhere else, like in the profile edit page)
  • Search box position
  • Number of posts per column (a single post, 2 posts, infinite posts?)

And, here's an infographic quoted from the

So, after you have defined your goals we can start the implementation. We can use quite a few tools to help us there, let’s see them.

WordPress Plugin For A/B Test: Max A/B

The first plugin we’ll see is MaxA/B. This is an awesome free plugin to test your versions. Install and activate it, and you’ll see a new menu item in your WP admin panel.

Before actually using it you need to create your control and test versions (A and B page). Create 2 simple pages, here what I’ve done is a simple page with a link to contact us like this:

Max A/B

In this case you can use pretty much anything you want to test, as per defined in your goals. Then let’s get our hands dirty, go to “experiments” and you’ll see this screen:


Now you can imagine how this will look like when we actually fill up our data :) Well, go ahead and add a new experiment. Add a cool name, your control version and your first variation. You can add up to 3 variations, which is good but remember that the more variations you have, more traffic you need to validate the conversion rate.

The conversion page is an important concept, it’s the goal of your conversion. For instance, if you have a “buy” button, the conversion page is the “payment” page. If you have a subscription page, the conversion is the “thanks” page and so on. This simply tells to the plugin that every time a user goes to “contact” page from our “test page” it means that he completed the goal and that should count in the conversion rate.

Then you can set when your experiment ends and you’re good to go.

Here is what I’ve done there:


After you create, you’ll let your users access your pages. After a few access your results table will look quite close to this:


Here you can see that the “Improvement” between our control and the variation is actually negative, which means that the variation page is worse than the original. You can check also the statistical significance of your results.

This is a cool approach, but sometimes what needs to be tested can be in a sidebar, homepage or any other place. Then we’ll need a different tool.

WordPress Themes For A/B Test: SES Theme Split Test

Here you can use the child theme concept in your favor. You can create a variation in a different place of the theme (sidebar, custom page template, number of posts called in the homepage) and compare the performance between the themes.

As I said before, avoid modifying a lot of things, otherwise you don’t know what exactly improved your performance.

After you have installed or created the child theme you can actually use the SES plugin. After installing and activating the plugin you can go to Settings > SES Theme Split Test where you’ll see this screen:


There you can select which themes you’ll test. Then you’ll need to use a different tool to track your results, in our case we’ll set this up via Google Analytics.

This is quite an easy step, create your account there and set up your site’s analytics. After you’ve pasted the analytics code and validated it you can go to the “Advanced Segments” section of your GA panel, in the top menu (beneath date). This is the screen that you’ll see:


Let the “User defined values” then check the “Matches Exactly” option and you’ll see that it automatically loads plugin data. Now “create a segment” and then we’re done to get our data.

Wrapping up

So, what do you think about this method? Would you use it? Share your thoughts with us!

About Rochester Oliveira

I'm a web designer and entrepreneur from Itajubá (MG), Brasil. I love writing about obscure topics and doing some cool stuff.