# Power Analysis >[!Abstract] Power analysis is a statistical technique used to determine the sample size needed for a study. It helps researchers to determine the number of subjects they need to include in their study in order to have sufficient statistical power to detect an effect of a certain size, or to determine the required sample size needed to achieve a desired level of power. Power analysis is often used in the planning stages of a study, before the data is collected, to ensure that the study is adequately powered to detect the effects of interest. > A power analysis determines what sample size will ensure a high probabilty that we correctly reject the **Null Hypothesis** that there is no difference between the groups. > - If we use the sample size recommended by a *power analysis* then we know, regardless of the p-value, we used enough data to make a decision. >[!Important] Statistical power, or the power of a hypothesis test is the probability that the test correctly rejects the null hypothesis. >That is, the probability of a true positive result. It is only useful when the null hypothesis is rejected. ![[stat_power_curves.png]] Before you do an experiment, you should perform a power analysis to estimate the number of observations you need to have a good chance of detecting the effect you're looking for. A statistical hypothesis test makes an assumption about the outcome, called the null hypothesis. For example, the null hypothesis for the Pearson’s correlation test is that there is no relationship between two variables. The null hypothesis for the Student’s t test is that there is no difference between the means of two populations. The test is often interpreted using a p-value, which is the probability of observing the result given that the null hypothesis is true, not the reverse, as is often the case with misinterpretations. - **p-value (p)**: Probability of obtaining a result equal to or more extreme than was observed in the data. The p-value is determined with respect to a significance level usually denoted with $\alpha$, usually 5%. - **p $\leq$ alpha**: reject H0, different distribution. - **p gt;$ alpha**: fail to reject H0, same distribution. We can see that the p-value is just a probability and that in actuality the result may be different. The test could be wrong. Given the p-value, we could make an error in our interpretation. There are two types of errors; they are: - **Type I Error**. Reject the null hypothesis when there is in fact no significant effect (false positive). The p-value is optimistically small. - **Type II Error**. Not reject the null hypothesis when there is a significant effect (false negative). The p-value is pessimistically large. In this context, we can think of the significance level as the probability of rejecting the null hypothesis if it were true. That is the probability of making a Type I Error or a false positive. Statistical power, or the power of a hypothesis test is the probability that the test correctly rejects the null hypothesis. The higher the statistical power for a given experiment, the lower the probability of making a Type II (false negative) error. That is the higher the probability of detecting an effect when there is an effect. In fact, the power is precisely the inverse of the probability of a Type II error. More intuitively, the statistical power can be thought of as the probability of accepting an alternative hypothesis, when the alternative hypothesis is true. When interpreting statistical power, we seek experiential setups that have high statistical power. - **Low Statistical Power**: Large risk of committing Type II errors, e.g. a false negative. - **High Statistical Power**: Small risk of committing Type II errors. Experimental results with too low statistical power will lead to invalid conclusions about the meaning of the results. Therefore a minimum level of statistical power must be sought. It is common to design experiments with a statistical power of 80% or better, e.g. 0.80. This means a 20% probability of encountering a Type II area. This different to the 5% likelihood of encountering a Type I error for the standard value for the significance level. ## Sources - [A Gentle Introduction to Statistical Power and Power Analysis in Python](https://machinelearningmastery.com/statistical-power-and-power-analysis-in-python/) - [[P-value| P Values]] ![[power_analysis.webp]]