Bayes' Rule | Deep learnings

Bayes' Rule:

A fundamental concept in probability theory. It allows us to update our belief about an event based on new evidence.

probability of an event A, given B is probability of event B given a * probability of Event A / Probability of event

With a clear example:

Let's see how can we can use this baye's theorem practically.

Remember:

Baye's theorem allows us to update our belief about an event based on new evidence.

Events:

A: The couple breaks up.

B: The couple has a public argument.

Now, let's explain P(A|B) and P(B|A):

1. P(A|B) reads as probability of A given B:
2. probability of couple breaks up (A), given that we observed they having a public argument (B)
3. P(B|A) reads as probability of B given A:
4. probability of couple has public arguments (B), given that the couple breaks up.

Let's use a simpler example to illustrate:

Imagine we're talking about just one couple, let's call them Amy & Sam.

P(A) : Probability that Amy & Sam breaks up, P(A) : 30%

P(B|A): Probability they had a public argument given they are breaking up P(B | A): 80%

P(B): Probability that Amy & Sam having a public argument = 31%

Now, let's say we actually see Amy & Sam. having a public argument. We want to update our belief about whether they'll break up.

We use Bayes' Rule:

P(A|B) = P(B|A) * P(A) / P(B)

P(A|B) = 0.80 * 0.30 / 0.31 ≈ 0.77 or 77%

Before we saw them argue, we though Alex and Sam had 30% chance of breaking up. After seeing them argue in public we now think they have 77% chance of breaking up.

Baye's theorem helped us update our belief about likelihood of break up after obeserving new evidence (the public argument).

Baye's theorm

P(A|B) = P(B|A) * P(A) / P(B)

Where:

• P(A|B) is the posterior probability
• P(B|A) is the likelihood
• P(A) is the prior probability
• P(B) is the evidence

1. What is posterior probability, P(A|B) ?

1. Probability of event occurring after taking into account the new evidence or information.It's an updated belief about something after considering new data.

2. What is likelihood, P(B|A)?

A: The couple breaks up.

B: The couple has a public argument.

"What is the probability of the couple having a public argument, given that they are going to break up?"

The likelihood, denoted as P(B|A), is the probability of observing the evidence B, given that the hypothesis A is true.

P(B|A) is the likelihood of observing a public argument among couples who are destined to break up. Among couples who are going to break up, 80% of them will have a public argument.

3. What is the prior probability, P(A)?

 P(A) is the probability that a couple will break up, before we observe any arguments or other evidence. For example, if P(A) = 0.3, it means we believe that 30% of couples in general will break up.

4. What is the evidence, P(B)?

P(B) is the overall probability of observing a public argument, regardless of whether the couple breaks up or not. It's called the evidence because it's what we actually observe. P(B) considers all scenarios: arguments from couples who will break up and from those who won't.

For example, if P(B) = 0.4, it means that 40% of all couples have public arguments, regardless of their eventual outcome.

To illustrate:

• P(A) = 0.3: We think 30% of couples break up (prior)
• P(B|A) = 0.8: Among couples who break up, 80% have public arguments (likelihood)
• P(B) = 0.4: 40% of all couples have public arguments (evidence)

P(B) is crucial because it normalizes our calculation in Bayes' Rule, ensuring our updated probability (posterior) is properly scaled.

Most important probability distributions in statistics

Probability distributions are mathematical function that describe the likelihood of different outcomes in an experiment. They are fundamental to statistics & probability theory, providing a framework for understanding & analyzing random phenomena.

Types of Probability Distributions: 

Discrete Distribution:

1. Bernoulli Distribution: Models a single trial with two possible outcomes (success or failure).
2. P(X=1) = p, P(X=0) = 1-p, where p is the probability of success
3. Mean (expected value): E[X] = p
4. Variance: Var(X) = p(1-p)
5. Example: Flipping a coin once (heads = success, tails = failure)
6. Binomial Distribution: Models the number of successes in n independent Bernoulli trials
7. Example: Number of heads in 10 coin flips
8. Poisson Distribution:
9. Models the number of events occurring in a fixed interval of time or space
10. Example: Number of customers arriving at a store per hour

Continuous Distributions:

 It's characterized by its bell-shaped curve and is defined by two parameters: the mean (μ) and the standard deviation (σ). symmetric around the mean.

68-95-99.7 Rule:

• About 68% of the data falls within 1σ of the mean
• About 95% falls within 2σ of the mean
• About 99.7% falls within 3σ of the mean

Parameters: Characterized by two parameters:

• μ (mu): the mean
• σ (sigma): the standard deviation

Importance:

• Many natural phenomena approximately follow this distribution
• Central to many statistical methods due to the Central Limit Theorem
• Often used as a starting assumption in data analysis and modeling

1. Normal (Gaussian) Distribution: Bell-shaped curve, symmetric around the mean
2. It's characterized by its bell-shaped curve and is defined by two parameters: the mean (μ) and the standard deviation (σ).
3. Example: Heights of people in a population, most people are of average height which is middle.
4. When data is distributed in this manner:
5. the mean = median = mode
6. with the bell curve having exactly one mode.
7. Bell curve is also symmetric, if you draw a vertical line at the middle, you will two mirror images on either side.
8. If you know the mean, you will know the top of the bell is.
9. if you know Standard deviation, you will know how spread out the bell curve is.
10. larger the sd, more spread out the bell curve will be.

Key concepts:

• Probability distributions
• Joint probabilities
• Conditional probabilities
• Chain rule of probability

Has been around 500 years. Pascal invented the triangle.

People has not come to the conclusion of definition of probability.

1. For all probabilities 0 <= P(a) <= 1 for any proposition or event "a"
2. P(true) = 1 & P(false) = 0
3. P (a union b) = p(a) + (b) - p(a intersect b)

Gaussian distribution:

what is anti-derivative?

1/(2 * pie) ^ 1/2 (e ^(-1/2 * x^ 2))

S e^ (-x)^2 * dx does not have an anti derivative.

true mean = expectation of probability density function.

statistics that estimates the true mean.

mass probability.