Math 19b: Linear Algebra with Probability Oliver Knill, Spring 2011Lecture 3: Conditional probabilityThe conditional probability of an event A under the condition that the event Btakes place is denoted with P[A|B] and defined to b e P[A ∩ B]/P[B].1 We throw a coin 3 times. The first 2 times, we have seen head. What is the chance t ha t weget tail the 3th time?Answer: The probability space Ω consists of all words in the alphabet H, T of length 3.These are Ω = {HHH, HHT, HT H, HTT, T HH, T HT, T T H, T T T }. The event B is theevent that the first 2 cases were head. The event A is the event that the third dice is head.2 Problem: Dave has two kids, one of them is a girl. What is the chance that the other is agirl?Intuitively one would here give the answer 1/2 because the second event looks independentof the first. However, this initial intuition is misleading and the probability only 1/3.Solution. We need to introduce the pr obability space of all p ossible eventsΩ = {BG, GB, BB, GG}with P[{BG }] = P[{GB }] = P[{BB }] = P[{GG }] = 1/4. Let B = {BG, GB, GG } bethe event that there is at least one girl and A = {GG} the event that both kids are girls.We haveP[A|B] =P[A ∩ B]P[B]=(1/4)(3/4)=13.3 You are in the Monty-Hall game show and need to chose from three doors. Behind one dooris a car and behind t he others are goa t s. The host knows what is behind the doors. Afteryou open the first door, he opens an other door with a goat. He asks you whether you wantto switch. Do you want to?Answer: Yes, you definitely should switch. You double your chances to win a car:No switching: The probability space is the set of all possible experiments Ω = {1, 2, 3 }.You choose a door and win with probability 1/3. The opening o f the host does no t affectany more your choice.Switching: We have the same probability space. If you pick the car, then you lose becausethe switch will turn this into a goat. If you choose a door with a goat, the host opens theother door with the goat and you win. Since you win in two cases, the probability is 2/3 .Also here, intuition can lead to conditional probability traps and suggest to have awin probability 1/3 in general. Lets use the notion of conditional probability to givean other correct arg ument: the intervention of the host has narrowed the laborato ry toΩ = {12, 13, 21, 23, 31, 32 } where 21 for example means choosing first door 2 then door 1.Assume the car is behind door 1 (the other cases are similar). The host, who we assumealways picks door 2 if you pick 1 with the car (the other case is similar) gives us the conditionB = {13, 21, 31 } because the cases 23 and 32 are not possible. The winning event is A ={21, 31 }. The answer to the problem is the conditional probability P[A|B] = P[A∩ B]/P[B]= 2/3.If A, B be events in the probability space (Ω, P ), then Bayes rule holds:P[A|B] =P[B|A] · P[A]P[B|A] + P[B|Ac].It is a formula for the conditional probability P[A|B] when we know the unconditionalprobability of A and P[B|A], the likelihood. The formula immediately follows from thefact that P[B|A] + P[B|Ac] = P[B].While the formula followed directly from the definition of conditional probability, it is veryuseful since it allows us to compute the conditional probability P[A|B] from the likelihoodsP[B|A], P[B|Ac]. Here is an example:4 Problem. The probability to die in a car accident in a 24 hour period is one in a million.The probability to die in a car accident at night is one in two millions. At night there is30% traffic. You hear that a relative of yours died in a car accident. What is the chancethat the accident too k place at night?Solution. Let B be the event to die in a car accident and A the event to drive at night.We apply the Bayes formula. We know P[A ∩ B] = P[B|A] · P[A] = (1/200 0000) · (3/10) =3/20000000.P[A|B] =P[A ∩ B]P[B]= (3/20000000)/(1/1000000) = 3/20 .The accident took place at night with a 15 % chance.A more general version of Bayes rule deals with more than just two possibilities. Thesepossibilities are called A1, A2, ..., An.Bayes rule: If the disjoint event s A1, A2, . . . Ancover the entire laboratory Ω, thenP[Ai|B] =P[B|Ai] · P[Ai]Pnj=1P[B|Aj] · P[Aj].Pro of: Because the denominator is P[B] =Pnj=1P[B|Aj]P[Aj], the Bayes rule just saysP[Ai|B]P[B] = P[B|Ai]P[Ai]. But these are by definition both P[Ai∩ B].5 Problem: A fair dice is rolled first. It gives a random number k from {1, 2, 3, 4, 5, 6 }.Next, a fair coin is tossed k times. Assume, we know that all coins show heads, what is theprobability that the score of the dice was equal to 5?Solution. Let B be the event that all coins are heads and let Ajbe the event t hat the diceshowed the number j. The problem is to find P[A5|B]. We know P[B|Aj] = 2−j. Because theevents Aj, j = 1, . . . , 6 are disjoint sets in Ω which cover it, we have P[B] =P6j=1P[B ∩Aj] =P6j=1P[B|Aj]P[Aj] =P6j=12−j/6 = (1/2 + 1/4 + 1/8 + 1/16 + 1/32 + 1/64)(1/6) = 21/128.By Bayes rule,P[A5|B] =P[B|A5]P[A5](P6j=1P[B|Aj]P[Aj])=(1/32)(1/6)21/128=263,1234560.10.20.30.40.5Figure: The conditional probabilities P [Aj|B] in the previous problem. The knowledgethat all coins show head makes it more likely to have thrown less coins.Homework due February 2, 20111 Problem 2) in Chapter 3: if the probability that a student is sick at a given day is 1 percentand the probability that a student has an exam at a given day is 5 percent. Suppo se that6 percent o f the students with exams go to the infirmary. What is the probability that astudent in the infirmary has an exam on a given day?2 Problem 5) in chapter 3: Suppose that A, B are subsets of a sample space with a probabilityfunction P. We know tha t P[A] = 4/5 and P[B] = 3/5. Explain why P[B|A] is at least1/2.3 Solve the Monty Hall problem with 4 doors. There are 4 doors with 3 go ats and 1 car. Whatare the winning probabilities in the switching and no-switching cases? You can assume thatthe host always opens the still closed goat closest to the