Introduction to Bayesian Inference Introduction to Bayesian Inference What are we trying to do Make inferences about hypotheses based on all information at our disposal See how new data affects our inferences Need to identify all hypotheses states of Nature that may be true Need to know what each hypothesis predicts that we will observe Need to know how to compute the consequences Our approach is Bayesian it reflects how scientists actually think Bayesian Inference 8 28 02 1 The mathematics we will be using is actually very simple There is nothing hard or deep or difficult no measure theory nothing complex The most difficult part of Bayesian inference is learning to think in a Bayesian manner Once you get it things become very easy People who have already learned some classical statistics experience may some difficulty in shifting to a Bayesian mode of thinking Some things may have to be unlearned However once you get it you ll find it much easier to understand standard statistical ideas as well as the Bayesian ones So it is an advantage for classical statistics to understand Bayesian statistics Bayesian Inference Introduction to Bayesian Inference Theory Hypothesis Model Creativity Deduction Predictions Inference Verification Falsification Data What do hypotheses predict about potential data How does data support or undermine hypotheses Bayesian Inference 8 28 02 2 Introduction to Bayesian Inference Induction Observation 8 28 02 3 Deduction Deduce outcomes from hypotheses B A B C A A Therefore B D Induction Infer hypotheses from outcomes If A then we are likely to observe B and C B and C are observed Therefore A is supported A B C E D Bayesian Inference 8 28 02 4 Introduction to Bayesian Inference Introduction to Bayesian Inference How new data support hypotheses H1 D1 H1 D1 H2 D2 H2 D2 H3 D3 H3 D3 What do we infer if we observe D1 D2 D3 Bayesian Inference How new data support hypotheses 8 28 02 5 Observing D1 refutes H1 supports H2 a little and H3 strongly Observing D2 supports H1 and H3 a little and H2 moderately Bayesian Inference Introduction to Bayesian Inference 8 28 02 6 Introduction to Bayesian Inference Statistical inference is not magic Cannot get information that isn t present in the data Statistical inference is easily misused Watch out for Garbage in Garbage out Often the right solution is a better experiment or better observations not slick statistical procedures Bayesian Inference 8 28 02 The first principle is that you must not fool yourself and you are the easiest person to fool Richard Feynman 7 Bayesian Inference 8 28 02 8 Introduction to Bayesian Inference Herman Rubin s Five Commandments All models are wrong but some models are useful G E P Box Bayesian Inference 8 28 02 9 For the client 1 Thou shalt know that thou must make assumptions 2 Thou shalt not believe thy assumptions For the consultant 3 Thou shalt not make thy client s assumptions for him 4 Thou shalt inform thy client of the consequences of his assumptions For the person who is both e g a biostatistician or psychometrician 5 Thou shalt keep thy roles distinct lest thou violate some of the other commandments Bayesian Inference Probability 8 28 02 10 Probability What is the probability that I will roll a 1 on a fair die Before rolling the die Bayesian Inference 8 28 02 11 What is the probability that I will roll a 1 on a fair die Before rolling the die After rolling but before looking Bayesian Inference 8 28 02 12 Probability Probability What is the probability that I will roll a 1 on a fair die Before rolling the die After rolling but before looking After the professor looks but before the professor says what he saw Bayesian Inference 8 28 02 13 What is the probability that I will roll a 1 on a fair die Before rolling the die After rolling but before looking After the professor looks but before the professor says what he saw After the professor says what he saw Bayesian Inference Probability 8 28 02 14 Probability What is the probability that I will roll a 1 on a fair die Before rolling the die After rolling but before looking After the professor looks but before the professor says what he saw After the professor says what he saw After a student looks and reports what was seen Bayesian Inference 8 28 02 15 What is the probability that I will roll a 1 on a fair die Before rolling the die After rolling but before looking After the professor looks but before the professor says what he saw After the professor says what he saw After a student looks and reports what was seen After you personally look Bayesian Inference 8 28 02 16 Probability Probability What is the probability that I will roll a 1 on a fair die Before rolling the die After rolling but before looking After the professor looks but before the professor says what he saw After the professor says what he saw After a student looks and reports what was seen After you personally look For a Bayesian probability is conditioned on what each individual knows It can vary from individual to individual Probability is not out there It is in your head Probability does not exist Phil Dawid Probability is about epistemology not ontology Bayesian Inference 8 28 02 17 In the Bayesian view probability describes your degree of belief given what you know Betting odds allow us to elicit probabilities Can apply to hypotheses not just events Probability is sometimes viewed as the frequency of occurrence after a finite number of repeated trials We will call this frequency not probability Probability is sometimes viewed as the frequency after a hypothetical infinite set of trials This is the basis of classical or frequentist statistics Bayesian statistics does not use this notion but we will discuss it where appropriate Bayesian Inference Probability 8 28 02 18 Probability R T Cox and independently I J Good proposed the following reasonable assumptions about plausibilities or degrees of belief Plausibility should be transitive i e if A is more plausible than B and B more plausible than C then A is more plausible than C This means that it should be possible to attach a real number P A to each proposition and rank plausibilities by those numbers The plausibility of A not A negation of A A should be some function of the plausibility of A P A f P A The plausibility of A and B should be some function of the plausibility of A given that B is true and the plausibility of B P A B g P A B P B Bayesian Inference 8 28 02 19 From these assumptions Cox and Good were able to