ISyE8843A, Brani Vidakovic Handout 11 Probability, Conditional Probability and Bayes FormulaThe intuition of chance and probability develops at very early ages.1However, a formal, precise definitionof the probability is elusive.If the experiment can be repeated potentially infinitely many times, then the probability of an event canbe defined through relative frequencies. For instance, if we rolled a die repeatedly, we could construct afrequency distribution table showing how many times each face came up. These frequencies (ni) can beexpressed as proportions or relative frequencies by dividing them by the total number of tosses n : fi=ni/n. If we saw six dots showing on 107 out of 600 tosses, that face’s proportion or relative frequency isf6= 107/600 = 0.178 As more tosses are made, we “expect” the proportion of sixes to stabilize around16.Famous Coin Tosses: Buffon tossed a coin 4040 times. Heads appeared 2048 times. K. Pearson tossed acoin 12000 times and 24000 times. The heads appeared 6019 times and 12012, respectively. For these threetosses the relative frequencies of heads are 0.5049, 0.5016,and 0.5005.What if the experiments can not be repeated? For example what is probability that guinea pig Squikisurvives its first treatment by a particular drug. Or “the experiment” of you taking ISyE8843 course inFall 2004. It is legitimate to ask for a probability of getting a grade of an A. In such cases we can defineprobability subjectively as a measure of strength of belief.Figure 1: A gem proof condition 1913 Liberty Head nickel, one of only five known and the finest of the five.Collector Jay Parrino of Kansas City bought the elusive nickel for a record $1,485,000, the first and onlytime an American coin has sold for over $1 million.Tutubalin’s Problem. In a desk drawer in the house of Mr Jay Parrino of Kansas City there is a coin, 1913Liberty Head nickel. What is the probability that the coin is heads up?The symmetry properties of the experiment lead to the classical definition of probability. An ideal die issymmetric. All sides are “equiprobable”. The probability of 6, in our example is a ratio of the number offavorable outcomes (in our example only one favorable outcome, namely, 6 itself) and the number of allpossible outcomes, 1/6.21Piaget, J. and Inhelder B. The Origin of the Idea of Chance in Children, W. W. Norton & Comp., N.Y.2This definition is attacked by philosophers because of the fallacy called circulus vitiosus. One defines the notion of probabilitysupposing that outcomes are equiprobable.1(Frequentist) An event’s probability is the proportion of times that we expect the event tooccur, if the experiment were repeated a large number of times.(Subjectivist) A subjective probability is an individual’s degree of belief in the occurrenceof an event.(Classical) An event’s probability is the ratio of the number of favorable outcomes andpossible outcomes in a (symmetric) experiment.Term Description ExampleExperimentPhenomenon where outcomes are un-certainSingle throws of a six-sided dieSample spaceSet of all outcomes of the experimentS ={1, 2, 3, 4, 5, 6}, (1, 2, 3, 4, 5, or 6dots show)EventA collection of outcomes; a subset of SA = {3} (3 dots show), B ={3, 4, 5, or 6} (3, 4, 5, or 6 dots show)or ’at least three dots show’ProbabilityA number between 0 and 1 assigned toan event.P (A) =16. P (B) =46.Sure event occurs every time an experiment is repeated and has the probability 1. Sure event is in factthe sample space S.An event that never occurs when an experiment is performed is called impossible event. The probabilityof an impossible event, denoted usually by ∅ is 0.For any event A, the probability that A will occur is a number between 0 and 1,inclusive:0 ≤ P (A) ≤ 1,P (∅) = 0, P (S) = 1.The intersection (product) A · B of two events A and B is an event that occurs if both events A and Boccur. The key word in the definition of the intersection is and.In the case when the events A and B are independent the probability of the intersection is the product ofprobabilities: P (A · B) = P (A)P (B).Example: The outcomes of two consecutive flips of a fair coin are independent events.Events are said to be mutually exclusive if they have no outcomes in common. In other words, it isimpossible that both could occur in a single trial of the experiment. For mutually exclusive events holdsP (A · B) = P (∅) = 0.2In the die-toss example, events A = {3} and B = {3, 4, 5, 6} are not mutually exclusive, since theoutcome {3} belongs to both of them. On the other hand, the events A = {3} and C = {1, 2} are mutuallyexclusive.The union A∪B of two events A and B is an event that occurs if at least one of the events A or B occur.The key word in the definition of the union is or.For mutually exclusive events, the probability that at least one of them occurs isP (A ∪ C) = P (A) + P (C)For example, if the probability of event A = {3} is 1/6, and the probability of the event C = {1, 2} is1/3, then the probability of A or C isP (A ∪ C) = P (A) + P (C) = 1/6 + 1/3 = 1/2.The additivity property is valid for any number of mutually exclusive events A1, A2, A3, . . . :P (A1∪ A2∪ A3∪ . . . ) = P (A1) + P (A2) + P (A3) + . . .What is P (A ∪ B) if the events A and B are not mutually exclusive.For any two events A and B, the probability that either A or B will occur is given bythe inclusion-exclusion ruleP (A ∪ B) = P (A) + P (B) − P (A · B)If the events A abd B are exclusive, then P (A · B) = 0, and we get the familiar formula P (A ∪ B) =P (A) + P (B).The inclusion-exclusion rule can be generalized to unions of arbitrary number of events. For example,for three events A, Ba and C, the rule is:P (A ∪ B ∪ C) = P (A) + P (B) + P (C) − P (A · B) − P (A · C) − P (B · C) + P (A · B · C).For every event defined on S, we can define a counterpart-event called its complement. The complementAcof an event A consists of all outcomes that are in S, but are not in A. The key word in the definition ofan complement is not. In our example, Acconsists of the outcomes: {1, 2, 3, 4, 5}.The events A and Acare mutually exclusive by definition. Consequently,P (A ∪ Ac) = P (A) + P (Ac)Since we also know from the definition of Acthat it includes all the events in the sample space, S, thatare not in A, soP (A) + P (Ac) = P (S) = 1For any complementary events A and Ac,P (A) + P (Ac) = 1, P (A) = 1 − P (Ac), P (Ac) = 1 − P (A)3These equations simplify solutions
View Full Document
Unlocking...