Posts List

A/B Testing Overview

Overview and Definitions The purpose of A/B testing is to determine through the use of statistical methods whether an experiment generates enough of a practically significant effect to support implementation. This is not as simple as seeing if the rates of two different groups are different, because of the inherent randomness in sampling from a population. Consider this toy example: library(scales) set.seed(1234) pop_1 <- rnorm(100, 0, 1) pop_2 <- rnorm(100, 0, 1) paste("The mean of pop_1 is: ", comma(mean(pop_1))) ## [1] "The mean of pop_1 is: -0.