# Module 5 - Sampling Distributions and the Central Limit Theorem >[!Abstract] In this module, you will learn about the Law of Large Numbers and the Central Limit Theorem. You will also learn how to differentiate between the different types of histograms present in statistical analysis. ```toc ``` ## Parameter and Statistic What is the average height of adult men in the US? This can be *estimated* quite well with a relatively small sample The **parameter** is a quantity of interest about the population: the population average $\mu$, or the population standard deviation $\sigma$ A **statistic (estimate)** is the quantity of interest as measured in the sample: the sample average $\bar{x}$, or the sample standard deviation $s$. ## The Expected Value If we sample an adult male at random then we *expect* his height to be around the average population average $\mu$, give or take about one standard deviation $\sigma$. The **expected value** of one random draw is the population average $\mu$ The **expected value of the sample average, $\mathbb{E}(\bar{x}_n)$**, is the population average $\mu$ $\mathbb{E}(\bar{x}_n) = \mu$ ![[The Standard Error#The Standard Error]] ## Expected Value and Standard Error ### Standard Error for the Sum What if we are interested in the sum of the $n$ draws, $S_n$, rather than the average $\bar{x}_n$? Since the sum and the average are related: $S_n = n\bar{x}_n$ $\mathbb{E}(S_n) = n\mu$ $SE(S_n) = \sqrt{n}\sigma$ Thus the variability of the sum of $n$ draws increases at the rate of $\sqrt{n}$ Note that the standard error for the sum increases at a rate of $\sqrt{n}$ while the standard error for the average goes down. ### Standard Error for Percentages What percentage of likely voters approve of the way the US President is handling his job? - Note that *percentage of likely voters* is an average (percentage of 1's among the 1 and 0 labels) - The percentage of voters approving is the percentage of 1s, which s $\frac{S_n}{n} \times 100\% = \bar{x}_n \times 100\%$ Therefore, $\mathbb{E}(\text{percentage of 1s}) = \mu\times100\%$ $SE(\text{percentage of 1s}) = \frac{\sigma}{\sqrt{n}}\times100\%$ ## The Law of Large Numbers The Square Root Law says that the SE of the sample mean goes to zero as the sample size increases $\lim_{n \rightarrow \inf} SE(\bar{x}_n) = 0$ The [[Law of Large Numbers]] states that the sample mean $\bar{x}_n$ will likely be close to the expected value $\mu$ if the sample size is large Keep in mind that the law of large numbers applies - For averages and therefore also for percentages, but not for sums as their SE increases by a rate of $\sqrt{n}$ - For sampling with replacement from a population, or for simulating data from a probability histogram ## The Central Limit Theorem As n gets large, the probability histogram looks more and more similar to the normal curve. This is an example of the [[Central Limit Theorem]] The key point of the CLE theorem is that the sampling distribution of the statistic is normal no matter what the population histogram is. If we sample $n$ incomes at random, then the sample average $\bar{x}_n$ follows the normal curve centred at $\mathbb{E}(\bar{x}_n)$ and the spread is given by $SE = \sigma/\sqrt{n}$. In other words, as long as you sample randomly, the sample mean of your process will always follow the [[Normal Distribution]] regardless of what the original data follows. In order for the CLE to apply - We sample with replacement, or we simulate independent random variables from the same distribution - The statistic of interest is a sum (averages and percentages are sums in disguise) - The sample size is large enough: the more skewed the population histogram is, the large the required sample size $n$