# Hypothesis Testing aka AB Testing 2022-08-20 >[!Abstract] **AB testing**, a.k.a. randomized experiments. We randomly split a set of subjects (patients, users, customers, …) into a treatment and a control group and give the treatment to the treatment group. This procedure ensures that ex-ante, the only expected difference between the two groups is caused by the treatment. [[self.stats/Statistics]] >[!Abstract] The process of testing whether or not a sample of data supports a particular hypothesis is called hypothesis testing. The steps of hypothesis testing are as follows: >1. State a null hypothesis $H_0$ and an alternative hypothesis > - Either the null hypothesis will be rejected, or it will fail to be rejected (not sufficient evedience to reject it) >2. Evaluate the null hypothesis using a test statistic and calculate the [[P-value]] >3. Compare the p-value to a certain degree of significance $\alpha$ >[!Important] When thinking about A/B Testing, always think about the relevant test statistic and the cause of its validity (Usually the [[Central Limit Theorem]]) ## Types of Hypothesis Tests - One-tail Hypothesis test: $H_0 : \mu = \mu_0 \text{ versus } H_1 : \mu < \mu_0 \text{ or } H_1 : \mu > \mu_1$ - Two-tailed hypothesis test: $H_0 : \mu = \mu_0 \text{ versus } H_1 : \mu \neq \mu_0$ ## Test Statistic In order to evaluate difference in the point estimate, we need to conduct a test statistc. For which we have several options: >[!Abstract] Here are some test statistics >- [[Z-test]]: assumes the test statistic follows a normal distribution under the null hypothesis >- [[t-test]]: uses student's t-distribution rather than a normal distribution >- [[Chi-squared Test]]: used to assess goodness of fit and to check whether two categorical variables are independent ## Examples - *Which email campaign increases product's conversion rates?* - *Is there a difference in height between population A and population B?* >[!Important] Sometimes you are not able to conduct a randomised trial or an AB Test. In that case then, what do you do? For that you need to do [[Causal Inference]]