A/B Testing

What is A/B Testing?

What is A/B testing?

Also known as split/bucket testing, it is a method for understanding which version of a website (or app) is better by comparing two versions of websites (or apps) against each other.

How does it help?

AB testing using data and statistics to validate new design changes. By comparing an existing version of a website (or app) against new variation one can ask focused questions about the changes made and then see what impact it has made on the website. It also eliminates confusions that exist in the business by changing their conversations from “we think” to “we know”. This way by analyzing the impact of the changes one can corporate changes that produce positive results as well as helps in website optimization(or app).

How it works:

You must take a webpage or an app screen and modify and create a second version. The change could be as simple as changing the color of a button or a font size or as complex as redesigning an entire page.

Now we shall measure the experience in an analytics dashboard and analyze it using the statistical engine. We then determine if the change brought a positive change or negative or had a neutral effect on visitor behavior.




A/B testing process

Collect Data

Include analytics in your web pages and gain insight as to where there are low visitor counts, low conversion rates or pages that can be improved.

Identify goals

Identify if you would like to change the color of a button, or change the header height and so on.

Generate hypothesis

Once you have a set of goals, now generate a hypothesis, prioritize the goals and identify how these goals will have an impact on your webpage (or app).

Create Variations

No make the changes to the website or mobile app experience like changing the color of a button or swap order of elements and so on.

Run Experiment

Kick off your experiment and wait for results. Visitors will be allowed to engage on your website. Their experience, interaction is then measured and counted.



Analyze Results

Now analyze the results. A/B testing will present data from the experiment and show the difference on how your page performed for the two variants and whether there was any significant improvement.

Now if you get a positive result, you can go ahead implement it in your website and see changes if not don’t worry you can just use it as a learning experience and generate better hypothesis next time.



Why A/B testing?

Product owners can now make careful changes to their app or web pages while they collect data on the results. They can construct the hypothesis, experiment using the same, understand what has had a positive or negative effect and then incorporate the same into their product.

A/B testing and SEO

How to prevent Googlebot from getting confused with multiple versions of the same webpage?

Use rel=”canonical” attribute to point the variations back to the original page.

Use 302(temporary) Redirects Instead of 301s (permanent).

Remember to run experiments only as long as it’s necessary.


So how do you go ahead and start your first A/B test?

1. Include the Visual Website Optimizer code snippet in your website



This means that we can now run the tests we create for the website.

2. Create variations using simple what you see is what you get editor.

Create website design for the variations using HTML, CSS code.



3. Choose your goals

Your goal could be to determine number of visitors to your webpage.



4. Start and track your test

You can track the data in real-time.


Using A/B testing one can test one change at a particular point and see the effect it had on the visitor’s behavior. You can then combine all the changes that had a positive impact to improve the overall experience for the user.


At Recro we combine Design Thinking, Lean and Agile to help you achieve market-leading  performance.

We partner with our clients with the aim to transform their business challenges into growth opportunities.