What is A/B testing?

Also known as split / bucket testing, it is a method for understanding which version of a website (or app) is better by comparing two versions of websites (or apps) against each other.

How it helps?

AB testing using data and statistics to validate new design changes. By comparing an existing version of a website (or app) against new variation one can ask focused questions about the changes made and then see what impact it has made on the website. It also eliminates confusions that exist in the business by changing their conversations from “we think” to “we know”. This way by analyzing the impact of the changes one can corporate changes that produce positive results as well as helps in website optimization(or app).

How it works:

You must take a webpage or an app screen and modify and create a second version. The change could be as simple as changing the color of a button or a font size or as complex as redesigning an entire page.

Now we shall measure the experience in an analytics dashboard and analyze it using statistical engine. We then determine if the change brought a positive change or negative or had a neutral effect on visitor behavior.




A/B testing process

Collect Data

Include analytics in your webpages and gain insight as to where there are low visitor counts, low conversion rates or pages that can be improved.

Identify goals

Identify if you would like to change the color of a button, or change the header height and so on.

Generate hypothesis

Once you have a set of goals, now generate hypothesis, prioritize the goals and identify how these goals will have an impact on your webpage (or app).

Create Variations

No make the changes to the website or mobile app experience like changing color of a button or swap order of elements and so on.

Run Experiment

Kick off your experiment and wait for results. Visitors will be allowed to engage on your website. Their experience, interaction is then measured and counted.



Analyze Results

Now analyze the results. A/B testing will present data from the experiment and show the difference on how your page performed for the two variants and whether there was any significant improvement.

Now if you get a positive result, you can go ahead implement it in your website and see changes if not don’t worry you can just use it as a learning experience and generate better hypothesis next time.



Why A/B testing?

Product owners can now make careful changes to their app or webpages while they collect data on the results. They can construct hypothesis, experiment using the same, understand what has had a positive or negative effect and then incorporate the same in their product.

A/B testing and SEO

How to prevent Googlebot from getting confused with multiple versions of same webpage?

Use rel=”canonical” attribute to point the variations back to the original page.

Use 302(temporary) Redirects Instead of 301s (permanent).

Remember to run experiments only as long as it’s necessary.


So how do you go ahead and start your first A/B test?

1. Include the Visual Website Optimizer code snippet in your website



This means that we can now run the tests we create for the website.

2. Create variations using simple what you see is what you get editor.

Create website design for the variations using HTML, CSS code.



3. Choose your goals

Your goal could be to determine number of visitors to your webpage.



4. Start and track your test

You can track the data in real-time.


Using A/B testing one can test one change at a particular point and see the effect it had on the visitor’s behavior. You can then combine all the changes that had a positive impact to improve the overall experience for the user.

Related Post

How to make Minimum Viable...

Based on Eric Ries description MVP is defined as: A minimum viable product is that...

Lean Startups Methodology

Why do startups fail? Often startups begin with the idea of perfecting a product, then...

Growth hacking

Sean Ellis coined the phrase “growth hacker” in 2010. What do you mean by growth...

Why startups fail?

Startups often fail when they live in a vacuum and then perfect and release a product to...

Referral Marketing

  What is Referral Marketing? Referral marketing, put simply, is a strategy adopted...

Why should you Outsource to...

Understanding the term- In the wake of globalization resulting in heightened...

Leave a Comments