Bayes’ Theorem

A partition of Ω is a sequence of disjoint sets A1, A2, … such that 

partition

The Law of Total Probability

Let A1…Ak be a partition of Ω. Then, for any event B,
law_of_total_probability

Example: There probability of accident of an accident-prone person is 0.4 and that of a not accident-prone person is 0.2. If there are 30% accident-prone person in society, what is the probability of accident of a random person?
Solution: Note that accident-prone and not-accident prone is the partition of types of people.
P(A): (probability of a person to be accident-prone): 0.3
P(Ac): 0.7
P(X|A): (probability of accident if the person is accident prone): 0.4
P(X|Ac): 0.2
Therefore, P(X) = P(X|A)P(A) + P(X|Ac)P(Ac) = 0.4×0.3 + 0.2×0.7 = 0.26


Bayes’ Theorem

Let A1, …, Ak be a partition of Ω such that P(Ai)>0 for each i.
If P(B)>0 then, for each i=1,…, k,
bayes

We call P(Ai) the prior probability of A and P(Ai|B) the posterior probability of A. The Bayes’ theorem is commonly seen in this form,

{displaystyle P(Amid B)={frac {P(Bmid A)P(A)}{P(B)}}}

Notice that this is nothing but a consolidated form of the law of total probability in the denominator. Try deriving both the forms of Bayes’ theorem. Hint: Use the conditional probability formula.

Example: I divide my email into three categories: A1 = “spam,” A2 = “low priority” and A3 = “high priority.” From previous experience, I find that
P(A1) = 0.7
P(A2) = 0.2
P(A3) = 0.1
Of course, 0.7 + 0.2 + 0.1 = 1
Let B be the event that the email contains the word “free.” From previous experience, P(B|A1) = 0.9, P(B|A2) = 0.01, P(B|A1) = 0.01. (Note: .9 + .01 + .01 (= 1.) I receive an email with the word “free.” What is the probability that it is spam? Bayes’ theorem yields,
bayes theorem exercise

Leave A Comment