Probability and Mathematic Stastics Lecture #2 Conditional Probability

Author： Bi_Gu
发布时间：September 29, 2022
831 views
No comments
11475 words
Categories：概率论与数理统计

<button class="btn m-b-xs btn-light btn-roundedbtn-addon" onclick='window.open("https://cdn.dabigu.xyz/101532138_p0.png","_blank")'><i class="fa fa-camera"></i>$\quad$ **フィッシュル by Ninelie** </button>

# Lecture 2 Conditional Probability

## Definition & Intuition

### Thinking Conditionally

* New data and information may affect our uncertainties
* Conditional probability : how to **update** our belief?
* All probabilities are **conditional**! (works as explicit/implicit background info or assumption)
* Conditioning is a powerful problem-solving strategy, which can reduce a complicated probability problem to a bunch of simpler conditional probability problems
* **First-step analysis** (首步分析法) : obtain recursive solution to multi-stage problems
  
  > 适用于具有多个步骤且往往拥有某种递归结构的问题
* "Conditioning is the soul of statistics"

### Definition of Conditional Probability

If $A$ and $B$ are events with $P(B)>0$, then the *conditional probability* of $A$ **given** $B$, denoted by $P(A\mid B)$, is defined as

$$
P(A\mid B)=\frac{P(A\cap B)}{P(B)}
$$

where $P(A)$ is the **prior probability** of $A$ (先验概率), $P(A\mid B)$ is the **posterior probability** (后验概率) of $A$.

> 因晚于$A$发生的事件$B$而改变了我们对于$A$的印象

## Bayes' Rule & The Law of Total Probability

### Chain Rule (链式法则)

For any events $A_1,A_2,\cdots,A_n$ with positive probabilities,

$$
P(A_1,A_2,\cdots,A_n)=P(A_1)P(A_2\mid A_1)P(A_3\mid A_1,A_2)\cdots P(A_n\mid A_1,\cdots,A_{n-1})
$$

Here $P(A_1,A_2,\cdots A_n)\triangleq P(A_1\cap A_2\cap\cdots\cap A_n),\mathrm{etc.}$

### Bayes' Rule

For any events $A$ and $B$ with positive probabilities,

$$
P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)}
$$

**Proof**

$$
\begin{aligned}
P(B\mid A)&=\frac{P(B\cap A)}{P(A)}\Rightarrow P(B\cap A)=P(A)P(B\mid A)\\
\Rightarrow P(A\mid B)&=\frac{P(A\cap B)}{P(B)}=\frac{P(A)P(B\mid A)}{P(B)}.
\end{aligned}
$$

### The Law of Total Probability (LOTP)

Let $A_1,A_2,\cdots, A_n$ be a *partition* of the sample space $S$ ($\mathrm{i.e.}$ the $A_i$ are disjoint events and their union is $S$), with $P(A_i)>0$ for all $i$. Then

$$
P(B)=\sum_{i=1}^n P(B\mid A_i)P(A_i)
$$

**Proof**

Since that $B=\bigcup_{i=1}^n(B\cap A_i)$, where $B\cap A_i$ are disjoint sets, according to the axiom for probability we have

$$
P(B)=P\left(\bigcup_{i=1}^n(B\cap A_i)\right)=\sum_{i=1}^nP(B\cap A_i)=\sum_{i=1}^nP(B\mid A_i)P(A_i).
$$

### Inference & Bayes' Rule

Let $A_1,\cdots, A_n$ be a *partition* of the sample space $S$ ($\mathrm{i.e.}$ the $A_i$ are disjoint events and their union is $S$), with $P(A_i)>0$ for all $i$. Then for any event $B$ such that $P(B)>0$, we have

$$
P(A_i\mid B)=\frac{P(A_i)P(B\mid A_i)}{P(A_1)P(B\mid A_1)+\cdots+P(A_n)P(B\mid A_n)}.
$$

**Proof**

$$
P(A_i\mid B)\xlongequal{\text{Bayes' Rule}}\frac{P(A_i)P(B\mid A_i)}{P(B)}\xlongequal{\text{LOTP}}\frac{P(A_i)P(B\mid A_i)}{\sum_{i=1}^nP(A_i)P(B\mid A_i)}.
$$

**Example : Bayes Spam Filter**

A spam filter is designed by looking at commonly occurring phrases in spam. Suppose that 80% of the email is spam. In 10% of the spam emails, the phrase "free money" is used, whereas this phrase is only used in 1% of non-spam emails. A new email has just arrived, which does mention "free money". What is the probability that it is spam?

Event $S$ = "An email is a spam"; $F$ = "An email has the phrase 'free money'"

$P(S)=0.8, P(S^c)=0.2$;

$$
\begin{aligned}
  P(S\mid F)&=\frac{P(F\mid S)P(S)}{P(F)}\\&=\frac{P(F\mid S)P(S)}{P(F\mid S)P(S)+P(F\mid S^c)P(S^c)}\\&=\frac{0.1\cdot0.8}{0.1\cdot0.8+0.01\cdot0.2}\\&=\frac{80}{82}\\&\approx0.9756
  \end{aligned}
$$

**Example : Random Coin**

You have one fair coin, and one biased coin which lands Heads with probability $\displaystyle\frac{3}{4}$. You pick one of the coins at random and flip it three times. It lands Heads all three times. Given this information, what is the probability that the coin you picked is the fair one?

Event $A$ = "The chosen coin lands Heads three times"; $F$ = "We picked the fair coin".

$$
\begin{aligned}
P(F\mid A)&=\frac{P(A\mid F)P(F)}{P(A\mid F)P(F)+P(A\mid F^c)P(F^c)}
\\&=\frac{\displaystyle(\frac{1}{2})^3\cdot\frac{1}{2}}{(\displaystyle\frac{1}{2})^3\cdot\frac{1}{2}+(\frac{3}{4})^3\cdot\frac{1}{2}}\\&=\frac{8}{35}.
\end{aligned}
$$

**Example : Communication Channel**

Suppose that Alice sends only one bit (a $0$ or $1$) to Bob, with equal probabilities. If she sends a $0$, there is a $5\%$ chance of an error occurring, resulting in Bob receiving a $1$; If she sends a $1$, there is a $5\%$ chance of an error occurring, resulting in Bob receiving a $0$. Given that Bob receives a $1$, what is the probability that Alice actually sent a $1$?

Event $A_1$ = "Alice sent a $1$"; $B_1$ = "Bob received a $1$"

$$
\begin{aligned}
P(A_1\mid B_1)&=\frac{P(B_1\mid A_1)P(A_1)}{P(B_1\mid A_1)P(A_1)+P(B_1\mid A_1^c)P(A_1^c)}\\
&=\frac{0.95\cdot0.5}{0.95\cdot0.5+0.05\cdot0.5}\\&=0.95.
\end{aligned}
$$

## Conditional Probabilities are Probabilities

### Conditional Probability

Condition on an event $E$, we update our beliefs to be consistent with this knowledge.

$P(\cdot\mid E)$ is also a probability function with sample space $S$:

* $0\leq P(\cdot\mid E)\leq 1$ with $P(S\mid E)=1$ and $P(\varnothing\mid E)=0$.
* If events $A_1,A_2,\cdots$ are disjoint, then
  
  $$
  P\left(\bigcup_{j=1}^{\infty}A_j\mid E\right)=\sum_{j=1}^{\infty}P(A_j\mid E).
  $$
* $P(A^c\mid E)=1-P(A\mid E)$.
* Inclusion - Exclusion : $P(A\cup B\mid E)=P(A\mid E)+P(B\mid E)-P(A\cap B\mid E)$.

### Bayes' Rule with Extra Conditioning

Provided that $P(A\cap E)>0$ and $P(B\cap E)>0$, we have

$$
P(A\mid B,E)=\frac{P(B\mid A,E)P(A\mid E)}{P(B\mid E)}
$$

**Proof**

* First we have Bayes' Law $\displaystyle P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)}$;
* Let $\hat{P}(\cdot)\triangleq P(\cdot\mid E)$;
* Bayes' Law also applies to $\hat{P}(\cdot)$ that
  
  $$
  \hat{P}(A\mid B)=\frac{\hat{P}(B\mid A)\hat{P}(A)}{\hat{P}(B)}=\frac{P(B\mid A,E)P(A\mid E)}{P(B\mid E)}.
  $$

### LOTP with Extra Conditioning

Let $A_1,\cdots, A_n$ be a *partition* of the sample space $S$ ($\mathrm{i.e.}$ the $A_i$ are disjoint events and their union is $S$), with $P(A_i\cap E)>0$ for all $i$. Then

$$
P(B\mid E)=\sum_{i=1}^nP(B\mid A_i,E)P(A_i\mid E).
$$

**Proof**

* First we have LOTP $P(B)=\displaystyle\sum_{i=1}^nP(B\mid A_i)P(A_i)$;
* Let $\hat{P}(\cdot)=P(\cdot\mid E)$;
* LOTP also applies to $\hat{P}(\cdot)$ that
  
  $$
  \hat{P}(B)=\sum_{i=1}^n\hat{P}(B\mid A_i)\hat{P}(A_i)=\sum_{i=1}^nP(B\mid A_i,E)P(A_i\mid E).
  $$

**Example : Random Coin**

You have one fair coin, and one biased coin which lands Heads with probability $\displaystyle\frac{3}{4}$. You pick one of the coins at random and flip it three times. It lands heads all three times. If we toss the coin a fourth time, what is the probability that it will land Heads once more?

Let Event $A$ : "the chosen coin lands heads all three times", $F$ : "we picked the fair coin", $H$ : "the chosen coin lands Heads on the fourth time", then

$$
\begin{aligned}
P(H\mid A)&=P(H\mid F,A)P(F\mid A)+P(H\mid F^c,A)P(F^c\mid A)\\
&=\frac{1}{2}\times\frac{8}{35}+\frac{3}{4}\times\left(1-\frac{8}{35}\right)\\&=\frac{97}{140}\left(>\frac{1}{2}\right).
\end{aligned}
$$

($P(F\mid A)$ *has been calculated eariler*)

### Approaches for Finding $P(A\mid B,C)$

* Think of $B,C$ as the single event $B\cap C$
* Use Bayes' rule with *extra conditioning* on $C$.
* Use Bayes' rule with *extra conditioning* on $B$.

## Independence of Events

### Independence of Two Events

**Definition**

Events $A$ and $B$ are *independent* if

$$
P(A\cap B)=P(A)P(B)
$$

If $P(A)>0$ and $P(B)>0$, then this is equivalent to

$$
P(A\mid B)=P(A);\quad P(B\mid A)=P(B).
$$

> 即$A,B$两事件间不提供任何信息

### Independence of Complementary Set

**Theorem**

If $A$ and $B$ are independent, then $A$ and $B^c$ are independent, $A^c$ and $B$ are independent, and $A^c$ and $B^c$ are independent.

**Proof**

$P(A\mid B)=P(A),P(B\mid A)=P(B)\Rightarrow P(B^c\mid A)=1-P(B\mid A)=1-P(B)=P(B^c)$

### Independence of Three Events

**Definition**

Events $A,B$ and $C$ are independent if all of the following equations hold:

$$
\begin{aligned}
P(A\cap B)&=P(A)P(B)\\
P(A\cap C)&=P(A)P(C)\\
P(B\cap C)&=P(B)P(C)\\
P(A\cap B\cap C)&=P(A)P(B)P(C).
\end{aligned}
$$

> 当四条性质都满足时，我们称事件$A,B,C$是**相互**独立的；当仅满足前三条时，我们称它们是**两两**独立的(*pairwise independent*)。两者虽然中文名称近似，但并不等价。

### Pairwise Independence and Independence

Consider two fair, independent coin tosses.

* Event $A$ : the event that the first is Heads.
* Event $B$ : the event that the second is Heads.
* Event $C$ : the event that both tosses have the same result.

First we have that

$$
\begin{aligned}
P(A)&=P(B)=\frac{1}{2};\\P(C)&=P[(A\cap B)\cup(A^c\cap B^c)]
\\&=P(A\cap B)+P(A^c \cap B^c)
\\&=P(A)\cdot P(B)+P(A^c)\cdot P(B^c)\\&=\frac{1}{4}+\frac{1}{4}=\frac{1}{2}.
\end{aligned}
$$

We verify that the first three equations hold, meaning that $A,B,C$ are **pairwise independent**;

However, $P(A\cap B\cap C)=P(A\cap B)=\displaystyle\frac{1}{4}\neq\frac{1}{8}=P(A)P(B)P(C)$.

> Because $C$ happens naturally when both $A$ and $B$ happens : $C$ is independent with $A$,$B$ respectively, but is not independent with $A$ and $B$ together.

### Conditional Independence

**Definition**

Events $A$ and $B$ are said to be *conditionally independent* given $E$ if:

$$
P(A\cap B\mid E)=P(A\mid E)P(B\mid E).
$$

**Example : Conditional Independence $\nRightarrow$ Independence**

We choose either a fair coin or a biased coin ($\mathrm{w.p.}$ $3/4$ of landing Heads).

But we do not know which one we have chosen and we flip it twice.

* Event $F$ : "chosen the fair coin"
* Event $A_1$ : "the first coin tosses landing Heads"
* Event $A_2$ : "the second coin tosses landing Heads"

Events $A_1$ and $A_2$ are conditionally independent given $F$ since that

$$
P(A_1\cap A_2\mid F)=P(A_1\mid F)P(A_2\mid F)\left(\frac{1}{4}=\frac{1}{2}\times\frac{1}{2}\right).
$$

However,

$$
\begin{aligned}
P(A_i)_{i=1,2}&\xlongequal{\text{LOTP}}P(A_i\mid F)P(F)+P(A_i\mid F^c)P(F^c)\\
&=\frac{1}{2}\times\frac{1}{2}+\frac{3}{4}\times\frac{1}{2}\\&=\frac{5}{8}
\end{aligned}
$$

And

$$
\begin{aligned}
P(A_1\cap A_2)&\xlongequal{\text{LOTP}}P(A_1\cap A_2\mid F)P(F)+P(A_1\cap A_2\mid F^c)P(F^c)\\
&=P(A_1\mid F)P(A_2\mid F)P(F)+P(A_1\mid F^c)P(A_2\mid F^c)P(F^c)\\
&=\frac{1}{2}\times\frac{1}{2}\times\frac{1}{2}+\frac{3}{4}\times\frac{3}{4}\times\frac{1}{2}\\&=\frac{12}{32}
\end{aligned}
$$

So that $P(A_1)P(A_2)=\displaystyle\frac{25}{64}\neq P(A_1\cap A_2)$, which means that $A_1$ and $A_2$ are not independent.

## Conditioning As A Problem-Solving Tool

### Example : Laplace's Rule of Succession

There are $k+1$ coins in a box. When flipped, the $i^{th}$ coin will turn up heads with probability $i/k$, $i=0,1,\cdots,k$. A coin is randomly selected from the box and is then repeatedly flipped. If the first $n$ flips all result in Heads, what is the conditional probability that the $(n+1)^{st}$ flip will do likewise?

## Pitfalls & Paradoxes

### Example of Simpson's Paradox

Let Event $A$ : "a successful surgery", $B$ : "Nick is the surgeon", $C$ : "the surgery is a heart surgery", then we have

$$
\begin{aligned}
P(A\mid B,C)=\frac{1}{5}&<P(A\mid B^c,C)=\frac{7}{9};\\
P(A\mid B,C^c)=\frac{9}{10}&<P(A\mid B^c,C^c)=1;\\
\text{But }P(A\mid B)=0.83&>P(A\mid B^c)=0.8.
\end{aligned}
$$

### Math Behind Simpson's Paradox

If $P(A\mid B,C)<P(A\mid B^c,C)$ and $P(A\mid B,C^c)<P(A\mid B^c,C^c)$,
it is possible that $P(A\mid B)>P(A\mid B^c)$.

### Solution of Monty Hall Problem

* Label three doors 1,2,3; We assume the contestant picks door 1
* By conditioning on the location of the car, Event $C_i$ = "the car is behind the door $i$", $i=1,2,3$; $P(C_i)=\displaystyle\frac{1}{3}$.
* By LOTP, we have
  
  $$
  \begin{aligned}
  P(\text{get a car})&=P(\text{win})\\&=P(\text{win}\mid C_1)P(C_1)+P(\text{win}\mid C_2)P(C_2)+P(\text{win}\mid C_3)P(C_3)
  \end{aligned}
  $$
  
  From which we have
  
  * Stay(Not switching) : $P^{\text{stay}}(\text{win})=\displaystyle 1\times\frac{1}{3}+0\times\frac{1}{3}+0\times\frac{1}{3}=\frac{1}{3}$;
  * Switching : $P^{\text{switch}}(\text{win})=\displaystyle0\times\frac{1}{3}+1\times\frac{1}{3}+1\times\frac{1}{3}=\frac{2}{3}$.
  
  Which means that we have a better chance winning if we choose to switch.

Last modification：September 29, 2022

Support if you like my work! qwq

版权声明：若文中无特殊说明，则本文为原创文章，版权归 Bi_Gu 所有。

本文链接：https://dabigu.xyz/index.php/archives/620/

所有原创文章采用知识共享署名-非商业性使用 4.0 国际许可协议进行许可。
您可以自由的转载和修改，但请务必注明文章来源并且不可用于商业目的。

猜你想看 Probability and Mathematic Stastics Homework #1 Probability and Mathematic Stastics Lecture #1 Probability and Counting

随机文章 CS100 Lecture #5 20220228 Maths Analysis II - Homework #23 20220519 "world.execute(me);"手抄pv源代码

Probability and Mathematic Stastics Lecture #2 Conditional Probability

Bi_Gu • 2022 年 09 月 29 日

# Lecture 2 Conditional Probability

## Definition & Intuition

### Thinking Conditionally

### Definition of Conditional Probability

If $A$ and $B$ are events with $P(B)>0$, then the *conditional probability* of $A$ **given** $B$, denoted by $P(A\mid B)$, is defined as

$$
P(A\mid B)=\frac{P(A\cap B)}{P(B)}
$$

where $P(A)$ is the **prior probability** of $A$ (先验概率), $P(A\mid B)$ is the **posterior probability** (后验概率) of $A$.

> 因晚于$A$发生的事件$B$而改变了我们对于$A$的印象

## Bayes' Rule & The Law of Total Probability

### Chain Rule (链式法则)

For any events $A_1,A_2,\cdots,A_n$ with positive probabilities,

$$
P(A_1,A_2,\cdots,A_n)=P(A_1)P(A_2\mid A_1)P(A_3\mid A_1,A_2)\cdots P(A_n\mid A_1,\cdots,A_{n-1})
$$

Here $P(A_1,A_2,\cdots A_n)\triangleq P(A_1\cap A_2\cap\cdots\cap A_n),\mathrm{etc.}$

### Bayes' Rule

For any events $A$ and $B$ with positive probabilities,

$$
P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)}
$$

**Proof**

$$
\begin{aligned}
P(B\mid A)&=\frac{P(B\cap A)}{P(A)}\Rightarrow P(B\cap A)=P(A)P(B\mid A)\\
\Rightarrow P(A\mid B)&=\frac{P(A\cap B)}{P(B)}=\frac{P(A)P(B\mid A)}{P(B)}.
\end{aligned}
$$

### The Law of Total Probability (LOTP)

Let $A_1,A_2,\cdots, A_n$ be a *partition* of the sample space $S$ ($\mathrm{i.e.}$ the $A_i$ are disjoint events and their union is $S$), with $P(A_i)>0$ for all $i$. Then

$$
P(B)=\sum_{i=1}^n P(B\mid A_i)P(A_i)
$$

**Proof**

Since that $B=\bigcup_{i=1}^n(B\cap A_i)$, where $B\cap A_i$ are disjoint sets, according to the axiom for probability we have

$$
P(B)=P\left(\bigcup_{i=1}^n(B\cap A_i)\right)=\sum_{i=1}^nP(B\cap A_i)=\sum_{i=1}^nP(B\mid A_i)P(A_i).
$$

### Inference & Bayes' Rule

$$
P(A_i\mid B)=\frac{P(A_i)P(B\mid A_i)}{P(A_1)P(B\mid A_1)+\cdots+P(A_n)P(B\mid A_n)}.
$$

**Proof**

$$
P(A_i\mid B)\xlongequal{\text{Bayes' Rule}}\frac{P(A_i)P(B\mid A_i)}{P(B)}\xlongequal{\text{LOTP}}\frac{P(A_i)P(B\mid A_i)}{\sum_{i=1}^nP(A_i)P(B\mid A_i)}.
$$

**Example : Bayes Spam Filter**

Event $S$ = "An email is a spam"; $F$ = "An email has the phrase 'free money'"

$P(S)=0.8, P(S^c)=0.2$;

**Example : Random Coin**

Event $A$ = "The chosen coin lands Heads three times"; $F$ = "We picked the fair coin".

**Example : Communication Channel**

Event $A_1$ = "Alice sent a $1$"; $B_1$ = "Bob received a $1$"

$$
\begin{aligned}
P(A_1\mid B_1)&=\frac{P(B_1\mid A_1)P(A_1)}{P(B_1\mid A_1)P(A_1)+P(B_1\mid A_1^c)P(A_1^c)}\\
&=\frac{0.95\cdot0.5}{0.95\cdot0.5+0.05\cdot0.5}\\&=0.95.
\end{aligned}
$$

## Conditional Probabilities are Probabilities

### Conditional Probability

Condition on an event $E$, we update our beliefs to be consistent with this knowledge.

$P(\cdot\mid E)$ is also a probability function with sample space $S$:

### Bayes' Rule with Extra Conditioning

Provided that $P(A\cap E)>0$ and $P(B\cap E)>0$, we have

$$
P(A\mid B,E)=\frac{P(B\mid A,E)P(A\mid E)}{P(B\mid E)}
$$

**Proof**

### LOTP with Extra Conditioning

Let $A_1,\cdots, A_n$ be a *partition* of the sample space $S$ ($\mathrm{i.e.}$ the $A_i$ are disjoint events and their union is $S$), with $P(A_i\cap E)>0$ for all $i$. Then

$$
P(B\mid E)=\sum_{i=1}^nP(B\mid A_i,E)P(A_i\mid E).
$$

**Proof**

**Example : Random Coin**

Let Event $A$ : "the chosen coin lands heads all three times", $F$ : "we picked the fair coin", $H$ : "the chosen coin lands Heads on the fourth time", then

($P(F\mid A)$ *has been calculated eariler*)

### Approaches for Finding $P(A\mid B,C)$

* Think of $B,C$ as the single event $B\cap C$
* Use Bayes' rule with *extra conditioning* on $C$.
* Use Bayes' rule with *extra conditioning* on $B$.

## Independence of Events

### Independence of Two Events

**Definition**

Events $A$ and $B$ are *independent* if

$$
P(A\cap B)=P(A)P(B)
$$

If $P(A)>0$ and $P(B)>0$, then this is equivalent to

$$
P(A\mid B)=P(A);\quad P(B\mid A)=P(B).
$$

> 即$A,B$两事件间不提供任何信息

### Independence of Complementary Set

**Theorem**

If $A$ and $B$ are independent, then $A$ and $B^c$ are independent, $A^c$ and $B$ are independent, and $A^c$ and $B^c$ are independent.

**Proof**

$P(A\mid B)=P(A),P(B\mid A)=P(B)\Rightarrow P(B^c\mid A)=1-P(B\mid A)=1-P(B)=P(B^c)$

### Independence of Three Events

**Definition**

Events $A,B$ and $C$ are independent if all of the following equations hold:

$$
\begin{aligned}
P(A\cap B)&=P(A)P(B)\\
P(A\cap C)&=P(A)P(C)\\
P(B\cap C)&=P(B)P(C)\\
P(A\cap B\cap C)&=P(A)P(B)P(C).
\end{aligned}
$$

### Pairwise Independence and Independence

Consider two fair, independent coin tosses.

* Event $A$ : the event that the first is Heads.
* Event $B$ : the event that the second is Heads.
* Event $C$ : the event that both tosses have the same result.

First we have that

We verify that the first three equations hold, meaning that $A,B,C$ are **pairwise independent**;

However, $P(A\cap B\cap C)=P(A\cap B)=\displaystyle\frac{1}{4}\neq\frac{1}{8}=P(A)P(B)P(C)$.

> Because $C$ happens naturally when both $A$ and $B$ happens : $C$ is independent with $A$,$B$ respectively, but is not independent with $A$ and $B$ together.

### Conditional Independence

**Definition**

Events $A$ and $B$ are said to be *conditionally independent* given $E$ if:

$$
P(A\cap B\mid E)=P(A\mid E)P(B\mid E).
$$

**Example : Conditional Independence $\nRightarrow$ Independence**

We choose either a fair coin or a biased coin ($\mathrm{w.p.}$ $3/4$ of landing Heads).

But we do not know which one we have chosen and we flip it twice.

* Event $F$ : "chosen the fair coin"
* Event $A_1$ : "the first coin tosses landing Heads"
* Event $A_2$ : "the second coin tosses landing Heads"

Events $A_1$ and $A_2$ are conditionally independent given $F$ since that

$$
P(A_1\cap A_2\mid F)=P(A_1\mid F)P(A_2\mid F)\left(\frac{1}{4}=\frac{1}{2}\times\frac{1}{2}\right).
$$

However,

$$
\begin{aligned}
P(A_i)_{i=1,2}&\xlongequal{\text{LOTP}}P(A_i\mid F)P(F)+P(A_i\mid F^c)P(F^c)\\
&=\frac{1}{2}\times\frac{1}{2}+\frac{3}{4}\times\frac{1}{2}\\&=\frac{5}{8}
\end{aligned}
$$

And

So that $P(A_1)P(A_2)=\displaystyle\frac{25}{64}\neq P(A_1\cap A_2)$, which means that $A_1$ and $A_2$ are not independent.

## Conditioning As A Problem-Solving Tool

### Example : Laplace's Rule of Succession

## Pitfalls & Paradoxes

### Example of Simpson's Paradox

Let Event $A$ : "a successful surgery", $B$ : "Nick is the surgeon", $C$ : "the surgery is a heart surgery", then we have

$$
\begin{aligned}
P(A\mid B,C)=\frac{1}{5}&<P(A\mid B^c,C)=\frac{7}{9};\\
P(A\mid B,C^c)=\frac{9}{10}&<P(A\mid B^c,C^c)=1;\\
\text{But }P(A\mid B)=0.83&>P(A\mid B^c)=0.8.
\end{aligned}
$$

### Math Behind Simpson's Paradox

If $P(A\mid B,C)<P(A\mid B^c,C)$ and $P(A\mid B,C^c)<P(A\mid B^c,C^c)$,
it is possible that $P(A\mid B)>P(A\mid B^c)$.

### Solution of Monty Hall Problem

Probability and Mathematic Stastics Lecture #2 Conditional Probability

Leave a Comment Cancel reply

《Phigros》游戏记录

使用RD Client实现Windows远程桌面

基于opencv和pyautogui的腾讯会议自动入会

Probability and Mathematic Stastics Homework #1

Introduction to Machine Learning Homework #2

Diary Set #11 20220613-20220626

Maths Analysis II - Homework #3 20220222

20220514 祝自己生日快乐

《月圆之夜》游戏记录

给你的博客引入fontawesome图标

Probability and Mathematic Stastics Lecture #2 Conditional Probability