What is A/B testing?
A/B testing is an experimentation process where two or more versions of a web page/product/app (or an element of a webpage/product/app) are created. The traffic to the concerned part of the website is split and the different versions are shown to different segments of the visitors at the same time. Specific goals are tracked during the experiment to measure which version leaves the highest impact and drives business metrics.
For instance, if you want to test the copy of the call-to-action (CTA) button on your homepage, you create versions of your homepage based on the number of alternatives you want to test. Each version displays an alternative CTA copy and you can track which one leads to a higher number of clicks as compared to others. You can then release the optimal version to 100% of your audience and improve the engagement on your homepage.
Benefits of A/B testing
There are many benefits to running A/B tests on your digital properties, however, we’ve listed the top 3.
- Get better ROI from existing traffic
The cost of acquiring quality traffic on your website is huge. A/B testing lets you make the most out of your existing traffic and helps you increase conversions without having to spend additional dollars on acquiring new traffic. When you test data-backed hypotheses, you improve your chances of achieving significant business conversions even with small changes on your website.
- Improve visitor experience
You deliver a good page experience to visitors when your page is built keeping in mind their expectations. A/B testing helps you listen to your visitors and understand better what they want to accomplish through your website. Finally, A/B testing gives you the data to decide which experience delights your visitors and deploy the same to bring in conversions.
- Achieve statistically significant improvements
Since A/B testing is entirely data-driven with no room for guesswork, gut feelings, or instincts, you can quickly determine a “winner” and a “loser” based on statistically significant improvements on metrics like time spent on the page, number of demo requests, cart abandonment rate, click-through rate, and so on. To ensure that you make the right decisions from statistically significant A/B tests, VWO offers free calculators:
- How long you should run your tests
- Calculate statistical significance
- Calculate sample size for your tests
A/B testing examples
BluTV, Turkey’s leading subscription-based, over-the-top (OTT) video-on-demand service, redesigned its home page for mobile devices and A/B tested it against the original home page. With the new design as the winner, they increased their subscription trials for mobile visitors by 42%. You can read the full case study here.
Ubisoft, a French video game publisher, A/B tested its new ‘Buy Now’ page and increased lead generation by 12%. You can read the full case study here.
How to run an A/B test
Running an A/B test broadly requires the below steps:
- Research – Quantitative and qualitative analysis to determine the current performance of your website; how many users are coming onto the site, which pages drive the most traffic, the various conversion goals of different pages, etc
- Hypothesis formation – Once you have the visitor data available through research, analyze it to form an effective hypothesis and test it against parameters such as how much confidence you have of it winning, its impact on macro goals, and how easy it is to set up.
- Creating variations – Create a version of the page you want to test based on your hypothesis. You can test multiple variations against the control to see which one works best.
- Run test – Pick a testing method that works best for your hypothesis and determine the goals you want to measure and impact with this test. Once this is done, you are ready to start the experiment with your A/B testing tool.
A/B testing with VWO
VWO is a leading A/B testing platform trusted by thousands of brands around the world including Domino’s, HBO, eBay, and Disney. It offers everything you need to create and run tests to increase revenue, combat cart abandonment, and build stellar digital experiences that convert, with minimum dependency on your developers. You can conduct qualitative and quantitative visitor research, build an experimentation roadmap and run continuous experiments to optimize web experience across desktop, mobile, and other devices, to deliver a unique experience based on visitor preferences. Start a 30-day all-inclusive free trial today to explore the features and capabilities of VWO.
If you are unsure where to start, we’ve got you covered. Download this eBook for some inspiring ideas that assure quick wins on your A/B testing journey.
