AB Testing Procedure:
-
Experiment Definition
-
Data Collection
-
Data Processing
-
Experiment: Frequentist Approach
-
Experiment: Bayesian Approach¶
-
Interpret Results - In which direction is the metric significant statistically and practically?
-
Launch Decision - Based on the results and trade-offs, was the change succesful?
References: [1] Good blog to understand Beta Distribution: http://varianceexplained.org/statistics/beta_distribution_and_baseball/