Highly Advanced A/B (Split) Testing Techniques

A/B or split testing refers to the technique in which two or more versions of web pages are tested to analyze conversion and choose the version producing maximum conversion. This is as simple as that. However, in this article, we will discover some highly advanced A/B testing techniques.

Understanding the Fundamental Concept

Typically an A/B test is implemented to analyze and measure a single goal conversion. You can put a URL that you would like to set as a conversion goal and measure the conversion level. This is pretty straightforward and when you do this successfully, you are surely operating much ahead than that of your competitors.

Multivariate Split Testing Concept

While the above and most commonly used technique is univariate by nature, multivariate split testing concept is believed to be more complex and advanced technique. As the name suggests, multivariate split testing depends on a number of factors, and not just a single goal.

Before we analyze what multivariate split testing is and how to implement it, we need to understand why we need it.

A website may not have only one goal to convert. For example, you may have a squeeze page that contains an opt-in form with a free offer available to subscribers to download. This action is quite clear-cut and you can easily set a goal to the download URL, and measure how many users actually convert following this action-path.

But at the same time, if you are offering a service and if you have included a bonus offer inside the free gift and if you actually wish to measure how many subscribers actually use this bonus offer and make a purchase, it needs multivariate analysis.

So, by now it is clear why we need multivariate split testing. Now it’s our turn to find out how to implement it.

Basically, if you consider the above example, you can easily understand that not all subscribers are interesting in purchasing your service / product, but they are still interested in your free product (and maybe a good cluster is a group of prospective customers in near future).

On the other hand, there are subscribers who are converted into specific buying action and take interest in the bonus offer you are giving along with the free offer. So, there could be endless possibilities of variable integration applied on the action-path directed to different goals.

In order to analyze this situation (or even more complex), you need to employ multivariate split testing measurement tool.

Analytical Surface Defined

Analyzing the above situation, you can easily define 3 types of visitors in your site – (1) new visitors (not converted yet to subscribers), (2) subscribers (researching on products, interested in free offers, not converted into buyers) and (3) subscribers and buyers (subscribers converted into buyer; at least once).

Therefore, quite obviously not a single goal would be a sufficient solution to the problem. Isn’t it? Basically, you can see the following diagram to get an idea -

So, it’s quite obvious that you need to set a multivariate analytic in order to identify how different types of visitors convert. While there are still different factors which act on the same surface such as returning visitors turned to subscribers, or subscribers turned to buyers. Additionally, you may need to take volume of unsubscribers into account, if you actually want to see how your offer runs.

So, a multivariate split testing option includes a wide range of variables that you need to take under consideration. Definitely, if you are new to business, you should seek professional help, because it is only a professional who can help you out with multivariate A/B (split) test. Good luck!