MIT OpenCourseWare http://ocw.mit.edu 21L.017 The Art of the Probable: Literature and Probability Spring 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.Bayes “An Essay towards solving a Problem in the Doctrine of Chances” and Notes on Bayes’ Essay Shankar Raman Since Bayes’ groundbreaking contribution to probability is not easy to read, I have adopted the tactic of excerpting the sections of relevance to us, and interspersing a commen-tary that explicates some of the key ideas. The sections you are expected to read are the following: 1. Price’s letter introducing Bayes’ essay. 2. Section I, consisting of the Definitions and Proposition 1 - 7: Proposition 4 is very difficult to reconstruct, so just focus on the result. 3. In Section 2 focus on the laying out of the problem, the results of Postulate 1 and 2, the result of Proposition 8, Proposition 9 and the Scholium. Don’t worry about the demonstra-tions in this section, since my commentary reconstructs the argument being made in a way that is easier to follow. Before turning to the essay itself, however, it may be useful to set out some basic ideas of probability theory in its modern form to aid the understanding of Bayes’ thought experiment. Basic Ideas: Conditional Probability: Probabilities are always relative to a universe, which we have called Ω. If we think, however, of a set of events in that universe in terms of what it shares with a different subset of that universe, the latter in effect defines a new universe. And this is what conditional probability is about: looking at events as conditioned upon or given a subset of Ω as domain of concern. The conditional probability of a set A given a set B (both subsets of Ω) is written as P (A|B). This conditional probability can be expressed in terms of the unconditional (that is, relative to Ω) probabilities of A and B. Bayes will attempt below to “prove” that relationship. But modern probability theory simply defines unconditional probability as this relationship (thereby creating a new conditional space for which the basic axioms of probability are valid). Let me sum up the definitions: P (A ∩ B)P (A|B)= P (B) ⇒ P (B) ∗ P (A|B)= P (A ∩ B) (1) 1and likewise P (A ∩ B)P (B|A)= P (A) ⇒ P (A) ∗ P (B|A)= P (A ∩ B) (2) Intuitively, what these equations express is that a set of events shared by two subsets A and B in the universe – in other words, the events comprising A∩ B – can be described from different perspectives: (1) We can look at them unconditionally as a subset of the universe as a whole (in which case their probability is written as P [A∩ B]). Or (2) we can view those events as events in B conditioned upon A’s happening and ask about this subset of B type events relative to A (i.e P [B|A]). Or (3) we can view these as events in A conditioned upon B’s happening (i.e., P [A|B]). Independence: Two sets of events A and B are independent if the occurrence of one gives us no information regarding the occurrence of the other. Or, A’s happening has no effect on B’s happening (and vice versa). This relationship can be stated as a relationship of conditional and unconditional probabilities. Viz. P (A|B)= P (A) and P (B|A)= P (B). Alternatively, using the equation 1 for P (A|B) above and rearranging terms: if A and B are independent, P (A ∩ B)= P (A) ∗ P (B). NOTE: Independence is NOT THE SAME as disjunction. Two events are disjunct if A ∩ B = ∅, that is, A and B share no events in common. But independence requires that they share events in common. Why? Well, if A and B are disjunct, this means that if A happens, B does not happen and vice versa. Therefore the happening of A gives us a great deal of information about B because it tells us that we can be certain that B did not happen. But independence requires that the happening of A has no bearing on the happening of B. Let’s see this via an example. If I toss a coin three times in succession, each toss is clearly independent of the other – getting a head on the first tells me nothing about what I may get in on the second or the third. For each toss, the outcome space is the same, that is, Ω = {H, T }, and, for instance, P(toss 1 = head, given that toss 2 = head) = P (toss 1 = head). Now consider, the set of outcomes of 3 tosses, That is, Ω= {HHH,HHT,HTH,HTT,TTT,TTH,THT,THH}. These outcomes are mutually exclusive, that is, they are disjunct, so the happening of any one tells us that the other cannot happen. Consequently, they are not independent. The Multiplication Rule: This rule allows us to write the joint probability of a series of events in terms of their conditional probabilities. Say we have 3 events A, B and C, 2then the probability that all three will occur (that is, P [A ∩ B ∩ C]) can be calculated by multiplying the unconditional probability of A happening, by the conditional probability that B happens given that A happened, by the conditional probability that C happens given that both A and B happened. That is, P (A ∩ B ∩ C)= P (A)P (B|A)P (C|A ∩ B) This can be simply verified by plugging in the appropriate terms for the conditional prob-abilities using equation 1 above. And if you draw this out as a tree or think about the Venn diagram, you will get an intuitive sense of this result. And the multiplication rule can be extended to any number of events. A, B and C need not be independent. If they are independent, the equation gets simpler, since all the conditionals disappear, leaving P (A ∩ B ∩ C)= P (A)P (B)P (C) In other words, if events are independent the likelihood of all of them happening is simply the product of the (unconditional) individual probabilities of their occurrence. 3LII. An Essay towards solving a Problem in the Doctrine of Chances. By the late Rev. Mr. Bayes, communicated by Mr. Price, in a letter to John Canton, M. A. and F. R. S. Dear Sir, Read Dec. 23, 1763. I now send you an essay which I have found among the papers of our deceased friend Mr. Bayes, and which, in my opinion, has great merit, and well deserves to be preserved. Experimental philosophy, you will find, is nearly interested in the subject of it; and on this account there