Economics 520, Fall 2005Lecture Note 2: Conditional Probability and Independence (CB 1.3)A fundamental topic of probability theory is how to update or modify probabilities to reflectthe arrival of new information. This plays a major role in certain parts of statistics, andalso in many economic models involving choice under uncertainty. We will start with a verysimple example to try to develop some intuition.Example 1: A randomly chosen child is either male or female, and either right-handed orleft-handed. The probabilities of each possible combination are:girl boyRH .45 .4275LH .05 .0725Note that the 4 probabilities sum to 1. Let E1denote the event of drawing a girl. Thus E1={(girl,RH), (girl,LH)}, and since (girl,RH) and (girl,LH) are disjoint, P r(E1) = .45+.05 = .5We call this a marginal or unconditional probability.Next, we want to know what is the probability of left-handedness among girls? To do this,we need to define “probabilities given certain events.”Definition 1 The conditional probability of an event E2given an event E1isP r(E2|E1) =P r(E1∩ E2)P r(E1),provided P r(E1) > 0.Example 1 continued: Let E2be the event (girl,LH). Note that P r (E2) = .05. Also,since E2⊂ E1,P r(E1∩ E2) = P r(E2) = .05.So the conditional probability of left-handedness given the child is a girl, is:P r(E2|E1) =.05.45 + .05= .10.This says that 10% of girls are left-handed. By similar calculation, we can see that 13.5%of boys are left-handed. So in a certain sense, the knowledge of the gender of the childmatters for how likely the child is left-handed.Notice that by dividing by P r(E1), we are essentially renormalizing by the marginal prob-ability. In essence, we are restricting attention to the left column of probabilities in theTable, and renormalizing them so that they sum to 1.Example 2: You draw a two cards out of a deck of 52. What is the probability that exactlyone of them is an ace? This probability is:P r(exactly one ace) =# ways to draw exactly one ace# ways to draw 2 cards out of 521=# ways to draw one of 48 non-ace cards × # ways to draw one of 4 ace cards# ways to draw 2 cards out of 52=48141582=32221.Again, we call this a marginal or unconditional probability, to emphasize that it does notreflect any additional information.Now, someone tells you that at least one of the two cards is an ace. We would like to beable to s ay, given that at least one of the two cards is an ace, what is the probability thatexactly one is an ac e?Let E1= {At least one ace} and E2= {Exactly one ace}. Since E2⊂ E1, the numer-ator probability is equal to P (E2) = 32/221, as we calculated previously. The proba-bility of the conditioning event E1(at least one ace) is the sum of the probabilities ofone ace and two aces. The latter is 1/221, so the probability of the conditioning eventis 33/221. (Alternatively the probability of at least one ace is 1 minus the probabilityof no aces which is 1 − (48/52) · (47/51)). Then the conditional probability is the ratio(32/221)/(33/221)=32/33. 2Example 3: A simple example is that of two coin tosses. What is the probability of twoheads given that you have at least one head in the two tosses. E1= {HH, T H, HT }, withprobability 3/4, E2= {HH}, so E1∩ E2= {HH} with probability 1/4, soP r(E2|E1) =P r(E1∩ E2)P r(E1)=1/43/4=13,not 1/2 as many people think at first.Example 4: It is easy to ge t m ixed up when thinking about conditional probabilities. Afamous example is the Monty Hall problem. You are a contestant in a game show and haveto choose one of three doors. Behind one of the doors is a prize; the other doors are empty.After you choose a door, the game show host (Monty Hall) opens one of the other two doorsand shows you there is no prize behind that door. The host then offers you the opportunityto switch from the door you chose to the third door. Should you switch?To solve this problem we first have to remove some of the ambiguities in the descriptionabove. We assume that the door behind which the prize is located is choosen randomly,with probability 1/3 for each door. More importantly, we assume that the host will alwaysopen one of the doors not chosen by the contestant and not containing the prize. If there isa choice for the host, for example if you choose door a and the prize is in fact behind doora, the host will choose one of the eligible doors (b or c) randomly. We also establish somenotation. Let P be the door with the prize, Y the door you choose, and H the door thehost opens, with P, Y, H ∈ {a, b, c}. The information then can be formulated asP r(P = a|Y ) = P r(P = b|Y ) = P r(P = c|Y ) = 1/3,P r(H = Y ) = P r(H = P ) = 0,P r(H = h|P, Y ) = 1/2, for all h ∈ {a, b, c|P 6= h, Y 6= h}, if Y = P,2P r(H = h|P, Y ) = 1, for all h ∈ {a, b, c|P 6= h, Y 6= h}, if Y 6= P.The question is, given the door you choose, say door a, and given that the host reveals thatdoor b is empty whether the probability that the prize is in c is higher or lower than theprobability that the prize is in a:P r(P = a|Y = a, H = b) >< P r(P = c|Y = a, H = b).By symmetry this relation is obviously the same asP r(P = a|Y = a, H = c) >< P r(P = b|Y = a, H = c).Another way of asking the question is whether the information that the host opens b isrelevant. If it is not relevant (implied by no benefit from switching) then the followingequality should hold:P r(P = a|Y = a, H = b) = P r(P = a|Y = a).Let us calculate the two probabilities for the last relation. First, P r(P = a|Y = a) is clearlyequal to 1/3. The probabilityP r(P = a|Y = a, H = b) =P r(P = a, H = b|Y = a)P r(H = b|Y = a)=P r(H = b|Y = a, P = a) · P r(P = a|Y = a)P r(H = b, P = a|Y = a) + P r(H = b, P = b|Y = a) + P r (H = h, P = c|Y = a)The numerator is equal to (1/2) · (1/3) = 1/6. The first term in the denominator is also1/6, the second is zero because the host never opens the door with the prize, and the thirdisP r(H = b, P = c|Y = a)= P r(H = b|P = c, Y = a) · P r(P = c|Y = a) = 1 · (1/3) = 1/3.Hence the denominator is 3/6, and the conditional probability is 1/3. Hence the probabilitythat you win if you don’t switch is 1/3, and the probability that you win if you switch is2/3: you should always switch.For a nice graphical explanation, seehttp://math.ucr.edu/~jdp/Monty_Hall/Monty_Hall.htmlIf you found that your original intuition for this problem was wrong, you …