# Module 3 - Probability --- >[!Abstract] In this module, you will learn about the definition of probability and the essential rules of probability that you will need for solving both simple and complex challenges. You will also learn about examples of how simple rules of probability are used to create solutions for real-life complex situations. > >- Understanding what probability is and why we use it >- Learning about examples of probability applications in real-life >- Knowing essential rules of probability that help with solving both simple and complex challenges ```toc ``` ## Four Basic Rules of Probability 1. **Compliment Rule** $P(\neg A) = 1 - P(A)$ 2. **Rule for equally likely outcomes** $P(A) = \frac{\text{number of outcomes in }A}{n}$ 3. **Addition Rule** Given that A and B are *mutually exclusive* $P(A \text{ or } B) = P(A) + P(B)$ 4. **Multiplication Rule** Given that A and B are *independent* $P(A \text{ and }B) = P(A) P(B)$ ## Conditional Probability The *conditional probability* of B given A is: $P(B|A) = \frac{P (A \text{ and } B)}{P(A)}$ Rearranging the conditional probability given above results in the general multiplication rule (rule 4 of [[Module 3 - Probability#Four Basic Rules of Probability|the four rules of probability]]): $P(A \text{ and }B) = P(A) P(B|A)$ In the special case where A and B are *independent*: $P(A \text{ and }B) = P(A) P(B)$ ## Bayes' Rule Suppose you are building a spam filter. How would you compute the following probability? $P(\text{email is spam | money appears in email})$ This is essentially asking P(spam|money) i.e. $ $P(A|B)$. We can use the rule of conditional probability as follows: $P(B|A) = \frac{P(A \text{ and } B)}{P(A)} = \frac{P(B \text{ and } A)}{P(A)} = \frac{P(A|B)P(B)}{P(A)}$ This rule is called Bayes' rule. The denominator, $P(A)$ sometimes needs to be computed using the following expanded version of Bayes' rule $P(B|A)= \frac{P(A|B)P(B)}{P(A|B)P(B) + P(A|\neg B)P(\neg B)}$ ## Bayesian Analysis The spam filter given above is an example of a *Bayesian analysis*. - Before examining the email, there is *a prior probability of 20%* that it is a spam *The idea is to look at the evidence inside the email, such as certain key words like money, and then to improve on that prior probability using that evidence we find inside the email.* After examining the email for certain keywords such as 'money', the filter updates this *prior probability* using Bayes' rule to arrive at the *posterior probability* that the email is a spam. Example: - 1% of the population has a certain disease. If an infected person is tested, then there is a 95% chance that the test is positive. If the person is not infected, then there is a 2% chance that the test gives an erroneous positive result (‘false positive’). - Given that a person tests positive, what are the chances that he has the disease? P(D) = 1% P(+|D) = 95% P(+|no D) = 2% Question: P(D|+) -> ? $P(D|+) = \frac{P(+|D)P(D)}{P(+)}$ What is $P(+)$? For that we can use the expanded version of Bayes Rule $P(D|+)= \frac{P(+|D)P(D)}{P(+|D)P(D) + P(+|\neg D)P(\neg D)} = 32.4\%$ See also [[Warner’s randomized response model]]