A Beginners Guide to A/B Testing

If you’re planning a large marketing campaign, you may find yourself feeling slightly daunted by what approach to take. Have you considered A/B testing? Perhaps not, if you haven’t, you really should!

If you don’t know, A/B testing allows you to trial a number of specific messages on a small test group to determine which is the most effective. You take these messages and you send them to the groups and monitor which has the most responses. A/B testing can be a fantastic way to decide between messages if you just can’t choose!

How to split the group

This really depends on how large your test group is. If you’re testing an email message, it’s wise not to send the test emails to everyone, just select a smaller group to trial the messages and once you’re sure you can send them. In other words, don’t just split your mailing list 50/50. You must ensure your sample group is large enough to give you accurate results though – so not too big and not too small! If you’re testing different versions of a landing page, the core idea remains the same, only you’re testing random visitors. These visitors are directed to a certain version of your page, allowing you to test elements such as copy or layout . What is key out of both of these A/B tests is the conversion rates – how many people complete your desired action, perhaps making a sale, signing-up or following a link. The format with the largest conversion rates is the winner and should be the one you choose. 

Ways to approach

There are a few different ways to approach an A/B test, firstly let’s look at email A/B testing.

Email testing

Let’s say you have an important message you want to get across, what’s the best way for you to do it? First of all you will need a mailing list to contact. These should be people who potentially have interest in your product or content. Try and be selective with your mailing list, the more specific you are, the better your results should be – there’s no point trying to persuade people who just don’t care. Once you have a mailing list you are confident in, you need to decide what to tell them. Copy could be one of the aspects you decide to A/B test. If this is the case, try and keep your key message the same. Let’s take this example…

Say we’re a company who manufactures socks and we’ve just launched a new range of socks which we want to promote. These are the two copy versions we are testing:

Version 1

Version 2

Version 1 is very formal, whereas version 2 is much more relaxed. We would measure success based on how which copy persuaded most people to follow the link. The overall message stayed the same – we’re a sock company and we’ve just released a new range of socks, come and look at them. What changed was our tone and the A/B test will show which is most effective.

How do I track success?

Good question! You could just wait for replies and assume the more responses you get the more successful the email. This is a bit of a vague method though, for thorough A/B testing, we really need to get right into the nitty-gritty. For this, you’ll need an email monitoring software. Mandrill by MailChimp is the one I have always used though there are others available. This monitoring software will show you which of your emails customers have opened and which clicked-through via links within the emails. It’s a good idea to chart these, something like this…

Yours will probably look a bit more impressive and the difference may not be as striking. In this case though, we can see version 1 is clearly the more successful.

You may also consider testing:

• Subject lines
• Graphics and pictures
• Link placement

Or anything else you can think of – the more you test the more certain you can be of your message!

But I want to test my website..

But what if it’s a web page you want to test? Maybe you want to test out the layout or something as simple as colour? A/B testing is a great way to do this. The first thing you’ll need is a tool to let you view which version is most successful, the same way you needed software to monitor your email campaign. The best free version of this is part of Google analytics (it used to be known as Google Website Optimizer). This method will ask you to select a percentage of your users who will take part in the testing. Google analytics requires you to add ‘experiment code’ to your page, so if you aren’t comfortable doing this, you should consider searching for a different tool.  Your chosen tool will monitor the testing and provide you with the results (Google analytics’ help page gives you all the information you could need and more for this).

You should be careful when testing your site not to make drastic changes to hastily, this could confuse and annoy your regular visitors, so try and make changes gradually. Our sock company, for example, may decide to change their homepage, but know to make changes slowly and test them correctly so they don’t lose visitors. Let’s say to begin with, they decide to change the image on their homepage. Version 1, their original image, will be seen by 50% of their visitors and version 2 will be seen by 50%  and success will be determined by how many users follow the link to view the catalogue.

Version 1

Version 2

The images give very different tones – version 1 seems cosy and traditional, while version 2 is a little raunchy and may attract an entirely different crowd. Other tests that could follow from this one may monitor what sorts of products customers view after seeing the images or how many subscribe to the mailing list.

The most important tips…

A/B testing will work on all manner of business elements, not just the two I have discussed, but the same key principles will apply:

• You need to set out with a clear goal – if you don’t know what you’re wanting to achieve your tests will likely be inconclusive
• Make sure your sample size is big enough or small enough – this will depend on how big your user base is, you want to get accurate results though, so make sure you get enough ‘responses’!
• Be patient – you might have to wait a little bit to get the results you need, if your testing doesn’t look successful, don’t be too hasty to give in!
• But don’t leave it too long…if something  is clearly not working, don’t keep using it, you could be losing yourself valuable customers!
• If at first you don’t succeed, try and try again – you might not get the right results first time, so don’t be afraid to re-test. Remember to leave a reasonable amount of time between test though, customers will get annoyed and bored if every time they visit your site, something else has changed. The same rule applies for email, bombarding people will only get you known as a spammer!

Photos courtesy of FreeDigitalPhotos.net

Opinions expressed in the article are those of the guest author and may not reflect the views of Webaholic.

Written by Ellen, Marketing Assistant for Blooie.