A/B Testing

A/B testing, at its simplest, is randomly showing a visitor one version of a page- (A) version or (B) version- and tracking the changes in behavior based on which version they saw. (A) version is normally your existing design (“control” in statistics lingo), and (B) version is the “challenger” with one copy or design element changed. In a “50/50 A/B split test,” you’re flipping a coin to decide which version of a page to show.

How long should you run an A/B Test?

For you to get a representative sample and for your data to be accurate, experts recommend that you run your test for a minimum of one to two weeks. By doing so, you would have covered all the different days in which visitors interact with your website.

What is A/B multivariate testing?

Multivariate testing is a technique for testing a hypothesis in which multiple variables are modified. The goal of multivariate testing is to determine which combination of variations performs the best out of all of the possible combinations. Websites and mobile apps are made of combinations of changeable elements.

What is A/B testing in statistics?

Like any type of scientific testing, A/B testing is basically statistical hypothesis testing, or, in other words, statistical inference. It is an analytical method for making decisions that estimate population parameters based on sample statistics.

What kind of A/B testing questions we should expect from a data scientist?

To really understand A/B testing, you should learn about experimental design and statistical inference. See the full article for more details.