There are a few different ways to approach a conversion rate test but most of them boil down to comparing a variant (or set of variants) to the original or control. This would typically involve these steps:
- Identify a conversion action of interest
- Identify a page/element to test
- Develop a variant or series of variants of the test page/element
- Set a sample size
- Run a test splitting traffic between the control & variant(s)
- Analyse the results
- Implement the best variant/keep the control based on performance
There are a number of ways to actually implement this, but let’s look at simplified A/B tests.
EXAMPLE: We identify that users arriving on page A often leave the site before performing our desired conversion, filling out a contact form. We want to reduce the number of people leaving and increase our conversion rate on page A.
Having a look at Google Analytics, we see that page A receives 1,000 visits a month, and 10% of users take action. With our heatmapping we see that only 30% of users make it below the fold (first visible section of the page), and so don’t even see our contact form.
Charged with this information we investigate direct competitors and note that some of them have moved their contact form to the header. We develop a hypothesis that by moving the form into the header, we will get greater visibility and therefore increase the number of users who complete our form.
We redesign the page, moving the form into the header, and also update some of the text, images and icons above the fold, to give as much pertinent information to the user in the section of the page they actually view.
We decide to test 2,000 users over two months, with 50% of the traffic going to the control (A), and 50% to the variant (B).
After the test has concluded we note that page B had a conversion rate of 16% while page A had a conversion rate of 11%. This improvement is statistically significant, and we can be confident that it was due to our change, not to random chance. Note that we are comparing our uplift against conversion rate of page A in the same period at 11%, not against our historical measurement of A at 10%.
We implement page B on the website.