Bag Servant A/B Tests To Increase Its Conversions

The company’s business model was based on marketing products of affiliates. Thus, being a part of the conversion funnel, a primary need was to ensure that the Bag Servant website increased click-throughs to product sites.

To earn revenue; it was vital that the site is effective not only in persuading visitors to complete the purchase successfully but also increasing their average order value (AOV).

Around the time when the Bag Servant team was grappling with this challenge, Tim Ash’s book Landing Page Optimization came out. After reading it, the Bag Servant team wondered about the credibility and trust of their website—the absence of which was identified in Tim’s book as one of the “seven deadly sins of landing page design.”

Until then, the team had focused on testing website elements such as the copy of headlines and CTA buttons, button size, color and so on. The lack of credibility and trust was seen as a major gap and gave Bag Servant’s A/B testing endeavors in an entirely new direction.

The team started to build its website credibility by leveraging the power of social proof. The company’s Twitter account boasted of 4,000+ followers, and they used this to convince visitors to the website. This change improved conversion rates, assuring the team that they were headed in the right direction.

Given below is the control page on which the above change was made:

The team then decided to A/B test the original (which displayed the 4,000+ Twitter count badge) against a version that replaced the number of Twitter followers with a WOW badge—a relatively hard-to-get badge presented by a renowned businesswoman.

The hypothesis was that the WOW badge would convey a higher level of trust and credibility than just the count of followers on Twitter.

This is what the variation page looked like:

The A/B testing was done using VWO platform. The test was set up to track the following 3 goals:

Goal #1: Visits to Affiliate Partner Sites
Goal #2: Site Engagement
Goal #3: Product Exploration

The results proved the hypothesis that the WOW badge did increase engagement and conversions.

• Visits to affiliate partners’ websites marked a 72.05% of improvement with the results having 98% statistical confidence.
• Site engagement saw a boost of 10.27%, with 98% chance to beat the original
• Product exploration on the home page saw an increase of 60.42%; however, the statistical confidence for this goal was only 93%.

This is the comparison image that shows the control and variation versions of the test:

In hindsight, implementing “best practices” seems a no-brainer. But this decision isn’t as simple as it may sound. This is because the context of the page makes a huge difference to test results.

Indeed, if the change is not made to the right page or in the right context of buyer psychology, the change may actually not deliver expected results. The effectiveness of a “proven practice” may well be invalidated.

Bag Servant sought to increase trust with customers by adding the Versign Seal and the Twitter Follow badge in two different variation pages of its search page. But both variation pages got them negative results for all the 3 goals mentioned above.

It is important to formulate robust hypotheses and test these adequately before making any change to the website permanent.