Back to Statistics & AB Testing
Statistics & AB Testing

Statistics & AB Testing

36 of 68 Completed

Introduction to Statistics and A/B Testing
Hypothesis Testing
A/B Testing & Experiment Design
Confidence Intervals
A/B Testing Common Scenarios
A/B Testing Tradeoffs
Generalized Linear Models and Regression

General Framework

Let’s start with a general framework for A/B testing. In practice, an A/B test can be summarized into four steps.

  1. Choose and characterize metrics to evaluate your experiments. What do you care about? How do you want to measure the effect?

  2. Choose the significance level, power, the length of the test, and calculate the required sample size.

  3. Implement the A/B test with control/treatment groups and run the test.

  4. Analyze the results and draw valid conclusions

In the next few sections, we’ll dive into each step in full detail.

To frame our understanding, let’s say that we’re looking to design an experiment around different features of Interview Query.

Good job, keep it up!



You have 32 sections remaining on this learning path.

There's so much more to Interview Query! Sign up to access hundreds of interview questions, expert coaching and a flourishing data science community.