Discrete Mathematics for ECE 14:332:202 Spring 2004 Recitation 9: Conditional Probability, Bayes’ Theorem, and Independence Introduction: In dealing with random events, it sometimes can help us out a little to have more information about the event. Finding out the results of another event that is related to our event can help us to better predict the outcome of the event. For example, say that we have been locked in the basement of the EE building for a long time to where we are completely out of touch with the weather. We may want to perform an experiment to test whether it is sunny or cloudy outside, so we send someone to check. Let us say that in general it is sunny half of the time. But before we perform our experiment we see someone walk into the building with sunglasses on – knowing this added information will increase the conditional probability that it is sunny outside. For our event regarding the weather, W, we may say that P[W=’sunny’] = 0.5, however P[W=’sunny’ | ’sunglasses’] = 0.98. If information about one event B does not change the probability of event A, we can write P[A|B] = P[A], or P[B|A] = P[B]. P[A] describes our a priori knowledge of event A before we have information about event B. If added knowledge does not change our a priori probability, then we say that the events are independent. Let’s look at another example. We draw at random a card from a deck of 52 cards: what is the probability that this card is a 7 (event A)? What is the probability it is a club (event B)? There are 4 7’s in the deck, so P[A] = 4/52 = 1/13. There are 13 clubs in the deck, so P[B] = 13/52 = 1/4. Now, say that we know that Suit=’clubs’, now what is the probability that this randomly drawn card is a 7? There are 13 clubs, and one is a 7, so we know P[A|B] = 1/13 = P[A] – A and B are independent events in that knowing B has no impact on the probability of A, and vice versa. Another way of stating independence is: -- Two events (A and B) are independent if and only if P[AB] = P[A]P[B]. The probability of the intersection must be equal to the product of the a priori probabilities of these events. In the previous example, the two events are independent also because P[AB] = P[Value = 7 and Suit = ‘clubs’] = 1/52 = P[A]P[B] = (1/4)(1/13). Two terms that are often confused are mutually exclusive and independent, but in the Venn Diagram representing the previous example we see that two events can be independent without being mutually exclusive Bayes’ Theorem gives us a convenient way to use our knowledge of events, so we will state it explicitly. P[B|A] = P[A|B]P[B]/P[A] – memorize it! This means we may have information about an event B, but wish to know how much a knowledge of event A will tell us about B. It is useful in probabilistic inference, when we can observe an effect, but we wish to know more about its cause that we cannot observe directly. Exercise: We wish to play a simple version the game of blackjack (one player, no dealer). You are dealt two cards, and the goal is to keep requesting more cards until you are as close to without going over 21. The cards are valued as follows: • Non-face cards (2-10) have the value on the card • J,Q,K each have value 10 points • Ace can be either 1 or 11, whichever puts you closest to 21 without going over. A blackjack is achieved when the first two cards dealt add up to 21 (ie Ace and {10,J,Q,K}). B A S BA ∩Answer the following questions on paper: 1. What is the total number of two-card combinations that can be drawn from a 52 card deck? 2. How many of these combinations (outcomes of our experiment) add up to 21? 3. What is the probability of the event B, having a blackjack from two cards drawn at random? 4. Given the event C, that one of the cards drawn is a Jack, what is the probability that the hand will result in a blackjack, P[B|C]? 5. What does this tell us about the independence of events B and C? 6. Using Bayes’ Theorem, calculate P[C|B], that given you got blackjack, what is the probability that one of the cards was a jack. Let’s implement our simple game of Blackjack in MATLAB using the functions newdeck.m (which will give us a shuffled deck) and draw.m (which will draw cards off the deck). Create a function blackjack.m with no input or output arguments. It will be a one handed game of blackjack using our deck made from the structured data-type. Our goal is to see what information will increase our chances of winning. The probability calculation with which we are concerned is what the probability is that the next card drawn will cause us to go bust (over 21). This probability will change with our knowledge of various things. Have your program: 1. Before we have been dealt any cards, we wish to play blind. Based on no knowledge, what is the probability of going bust (event F), P[F] with 3 cards being dealt. (The prob. that the values of 3 cards are greater than 21 … hint: this is a matter of counting the number of combinations that add to greater than 21 divided by total number of 3 card combinations. Assume an Ace is counted as a 1). Display to the screen. (see below) 2. Now, say you know one of the card’s values (value A). What is P[F|A=a]. Deal one card and have MATLAB count the card combinations. 3. Knowing both card values A and B, what is P[F|A,B=(a,b)]? Display this to the screen. 4. Allow an input after your cards and probabilities have been displayed: 1. HIT 2. STAY, and see if you can “beat the odds” If this seems hard, don’t worry – do the best you can. I would like to really challenge you in these recitations. Have fun with this, and I hope you learn along the way! Additional Exercises: A Fibonacci sequence f(n), n=1,2,3… has a recurrence relation such that the nth term is the sum of the previous two, or f(n) = f(n-1) + f(n-2). Typically it begins with 0,1, such that the first few terms of the sequence are 0 1 1 2 3 5 8 … Say we want to use a modified Fibonacci sequence that can begin with two randomly selected numbers. We select the first two numbers A and B according to the following probability distribution: Any number in the set {1 , 2 , 3 } is chosen with Probability 1/3. We will describe the sequence as A, B, C, D, … 1. Calculate P[C=4], and P[C=2] 2. Now, say we know B=2, calculate P[C=4|B=2] and P[C=2|B=2]. Does this added knowledge change our probabilities? 3. If we