Conversion rate optimisation and AB testing- what are they?

As CRO marketing's popularity grows, getting lost in the vast array of tools & platforms, test options, techniques and methods is easy. It is important to remember that in most cases, the simpler the test to begin with, the better the results. More complex testing methods certainly have their place, but initial testing should be simple. Start basic and build on those initial results and learnings.

As mentioned, there are many different types of CRO tests. The most common include A/B tests, multivariate tests, split or redirection tests and funnel tests. With the rise in the importance of personalisation, the strategies within these tests are endless.

In this blog, I will focus on the bread and butter of Conversion Rate Optimisation A/B tests. I will also point out some often overlooked issues that can save you a lot of stress.

What is AB testing?

So, you’ve read the intro, and now I hear you asking, ‘What is AB Testing?’ Well, essentially, AB testing is running two pages simultaneously against each other and determining which page performs better. Traffic will be split between the current page on your site (original) and the page containing the variants (variation). Statistical analysis is then used to determine the winning page. There are many aspects of the page that you could AB test. These include, but are not limited to;

• Headlines
• Calls to Action
• Images
• Videos
• Links
• Content Layout

The conversion rate optimisation testing process

AB tests are designed to test certain aspects of an existing page, and almost all the CRO tools out there will have a WYSIWYG editor (pronounced ‘wiz-ee-wig’). This editor allows you to create variations, modify pages, and manage the test settings. The first variation will be created by default. This is a carbon copy of the original version of the page, and the editor will allow you to modify elements on the page.

Now, before you get over excited and start creating tests left, right and centre, the initial research must be done. There are a few simple steps to follow when working towards conversion rate optimisation:

• Site Analytics research – use Google Analytics, or whatever analytics tool you have, to identify pages within the conversion funnel creating issues. Look at metrics such as drop-off rates and bounce rates.
• User Behaviour – once the initial analytical data is completed and you have identified the page or pages you want to test, it is time to analyse how users use your website. This can include heatmapping, scroll maps, on-page surveys and recorded user sessions. There are many tools out there. We have used Crazy Egg and Loop11 with great success.
• Test Hypothesis – what changes, based on the previous steps, will help improve the conversion rate?
• Variation Design – Design the variation page based on your initial research.
• What we find helpful is to create a few design variations, sign up to a design survey site, such as com, and submit your designs to one of the available test options, where you can gain unbiased feedback.
• Test Implementation – Choose which CRO test suits your hypothesis and get testing

Analysing CRO test results

• Analyse results – it is important not to stop the test too early. Waiting for statistical significance is imperative to gain accurate results. All good CRO platforms will have a statistical significance calculator built into them, and there are plenty of online calculators available online.

AB testing vs. split testing

There is also common confusion about the difference between an AB test and an ‘AB’ redirection test (split test) and when to use which one. A split test, or redirection test, is used to test variations of a specific page hosted on different URLs. In other words, a split test’s variations are hosted on various URLs, while AB tests are hosted on the same URL. You’re now probably wondering, ‘Well, that’s all well and good, but when should I use an AB test, and when should I use a split test?’. At the risk of oversimplifying it, for any significant back-end design changes or complete page re-designs, a split test is preferred. If you are only looking to test simple front-end changes to the page, an AB test is preferred. Bear in mind a split test requires more effort and work hours, as you will need to build a completely new HTML, whereas, in an AB test, changes can easily be made within a WYSIWYG editor.

I can already hear you SEO marketers yelling at the computer screen about how these split tests will impact the page rankings. But fear not; Google, as always, has you covered. Google recommends using the attribute rel=”canonical” and not a no-index tag on your alternative pages. This will indicate to Google that the variations are similar to the original.

Another useful use for split tests we have come across for our smaller clients is when pages you want to test have small amounts of volume. There is nothing more demoralising than doing all the initial research and analysis on your website, then plugging in the data to an online CRO test duration calculator, like this one, only to find out that the test would need to run for 10 months to reach statistical significance. We have found a technique using split tests to be very useful with this.

We have a travel client who was looking to do a CRO test on their great deal package page. The site contained these great deal package pages for all their available destinations. The layout of the pages was exactly the same, with the only difference being the actual content, which was unique to each destination. We found that there was not enough volume going towards these individual destination pages for us to conduct a test in a reasonable amount of time.

We knew what we wanted to do, and we wanted to completely re-design the pages, with more emphasis on the benefits the client provides and the CTAs. We chose 6 of the top volume destinations, created new variation pages on the website, and re-designed their great deal pages in exactly the same way, with the only difference being the destination-specific content. We made sure the variation pages had a common element in their URL. The original pages contained ‘great-deals’, and our variations contained ‘your-deals’.

We then set up a redirection test within our CRO platform, AB Tasty, and used a regular expression within the split test section of their editor, which allowed all the original destination ‘great deal’ pages to redirect to their corresponding variation ‘your deal’ pages. This technique allowed us to combine all the data from the six original pages and six variation pages into one test, allowing us to reach statistical significance in a shorter amount of time while comparing our original design and the variation design. The result ended up with a 36% increase in conversions for our variation page design.

CRO tracking

Being in the digital marketing industry you’re probably tired of hearing it, but I’m going to say it again, accurate tracking is imperative to any CRO test success. You need to ensure all conversion points on the page are being tracked as well as any type of funnel conversion points. If you are testing a certain page within a funnel, you need to ensure the CTA on that page is tracked, as well as tracking the lead or transaction complete. You don’t only want to test if your variation succeeds in getting people to the next page of the funnel, you want to ensure that your variation is also succeeding in getting more people to complete a lead or transaction.

All quality CRO tools, such as Optimizely, AB Tasty, Google Optimize and VWO provide their own tag which goes across each page of the website, allowing you to track what you need.

Tracking becomes more complicated when you decide to get adventurous and throw live chat or call tracking into the mix. As most live chats operate through an iFrame, the implementation is quite complex, and if you don’t have experienced dev on your side, will require assistance from the tool’s technical support.

The biggest tracking issue we have come across has been with call tracking. For our clients, we use third-party call tracking, which generates dynamic numbers based on the channel a user comes through. We found that a lot of the CRO platforms do not support and integrate effectively with third-party call tracking, with the exception of a select few, including Optimizely Classic and Google Optimize (if you have the third party call tracking set up as a goal within Analytics). This creates extra work and manual data integration to include these call conversions. It is so important to prepare for these issues early on so you don’t find yourself in any awkward conversations with your client or management halfway through a test.

Another piece of advice would be to set up a report in Analytics as well, so you can cross check your data with the results in the CRO platform. If you’re testing part of a funnel, I find setting up an Analytics goal funnel a good back up.

There you have it, although AB tests are the most standard of the CRO tests, they still require a good amount of initial research and preparation. As you start to embark on the exciting journey that is CRO, keep in mind the above points to save yourself some trouble later on.