Evidence Fisher Neyman Pearson Likelihood Ratios The Nature of Evidence Bret Larget larget stat wisc edu Departments of Botany and of Statistics University of Wisconsin Madison Botany 940 January 31 2006 Bayesian Evidence Fisher Neyman Pearson Likelihood Ratios Bayesian Definitions of Evidence What is evidence According to The Merriam Webster Dictionary Evidence is P ROOF or T ESTIMONY matter submitted in court to determine the truth of alleged facts A statistical definition according to Goodman and Royall 1988 Evidence is a property of data that makes us alter our beliefs about how the world around us is working Evidence Fisher Neyman Pearson Likelihood Ratios Bayesian Definitions of Evidence What is evidence According to The Merriam Webster Dictionary Evidence is P ROOF or T ESTIMONY matter submitted in court to determine the truth of alleged facts A statistical definition according to Goodman and Royall 1988 Evidence is a property of data that makes us alter our beliefs about how the world around us is working Evidence Fisher Neyman Pearson Likelihood Ratios Bayesian Definitions of Evidence What is evidence According to The Merriam Webster Dictionary Evidence is P ROOF or T ESTIMONY matter submitted in court to determine the truth of alleged facts A statistical definition according to Goodman and Royall 1988 Evidence is a property of data that makes us alter our beliefs about how the world around us is working Evidence Fisher Neyman Pearson Likelihood Ratios The Debate in Statistics over Evidence The debate over what statistical inference methods ought to be used in science extends back to the 1920s The debate continues today However almost all introductory courses in statistics for scientists do not discuss the debate within the statistics community about the related philosophical issues You may be surprised that the primary inferential procedures taught in most statistics courses is a combination of two schools of thought and was disliked greatly by the founders of each school Bayesian Evidence Fisher Neyman Pearson Likelihood Ratios The Debate in Statistics over Evidence The debate over what statistical inference methods ought to be used in science extends back to the 1920s The debate continues today However almost all introductory courses in statistics for scientists do not discuss the debate within the statistics community about the related philosophical issues You may be surprised that the primary inferential procedures taught in most statistics courses is a combination of two schools of thought and was disliked greatly by the founders of each school Bayesian Evidence Fisher Neyman Pearson Likelihood Ratios The Debate in Statistics over Evidence The debate over what statistical inference methods ought to be used in science extends back to the 1920s The debate continues today However almost all introductory courses in statistics for scientists do not discuss the debate within the statistics community about the related philosophical issues You may be surprised that the primary inferential procedures taught in most statistics courses is a combination of two schools of thought and was disliked greatly by the founders of each school Bayesian Evidence Fisher Neyman Pearson Likelihood Ratios Testing Now you have the chance to tell me how to carry out a statistical test Bayesian Evidence Fisher Neyman Pearson Likelihood Ratios Schools of Thought I will describe four separate schools of thought on how to do statistical inference 1 Fisher and Significance tests 2 Neyman and Pearson and Hypothesis tests 3 Likelihood Ratios 4 Bayesian Inference Bayesian Evidence Fisher Neyman Pearson Likelihood Ratios The Fisher School Every experiment may be said to exist only in order to give the facts a chance of disproving the null hypothesis R A F ISHER 1937 According to Fisher the necessary elements of a significance test were a null hypothesis and a test statistic with a null distribution This results in a p value The p value is interpreted as the probability of obtaining data at least as extreme as the observed data assuming that the null hypothesis is true Data for which the p value is less than an arbitrary threshold such as 0 05 is called significant Bayesian Evidence Fisher Neyman Pearson Likelihood Ratios Bayesian The Neyman Pearson School According to Neyman and Pearson the necessary elements of a hypothesis test were null and alternative hypotheses a test statistic with a null distribution and a rejection region The decision to accept or reject the null hypothesis in favor of the alternative hypothesis is based on whether or not the test statistic falls into the rejection region Evidence Fisher Neyman Pearson Likelihood Ratios A Revealing Quote no test based upon a theory of probability can by itself provide any valuable evidence of the truth or falsehood of a hypothesis But we may look at the purpose of tests from another viewpoint Without hoping to know whether each separate hypothesis is true or false we may search for rules to govern our behavior with regard to them in following which we insure that in the long run of experience we shall not often be wrong J N EYMAN AND E P EARSON 1933 Bayesian Evidence Fisher Neyman Pearson Likelihood Ratios Bayesian Discussion of Quote Neyman and Pearson regarded hypothesis testing as a process which guaranteed a long run error rate of rejecting false null hypotheses The price of this objective method to make decisions is that we abandon our ability to measure evidence or judge truth in individual experiments As the inability to make judgments in individual experiments is clearly undesirable standard practice evolved to fix the Neyman Pearson procedure Fisher s p value was added as a measure of the strength of evidence against the null hypothesis Evidence Fisher Neyman Pearson Likelihood Ratios Bayesian The Combined Approach The combined approach became standard practice despite the vehement arguments against each other s methods from the founders of each school The combined method is lauded by many as being scientific because of its objectivity However the combined method is an automatic procedure for drawing inferences that does not allow for the inclusion of judgment or knowledge of the underlying scientific processes There is no mechanism to include any prior evidence Evidence Fisher Neyman Pearson Likelihood Ratios P values as Measures of Evidence Over the decades many authors have criticized p values and hypothesis testing procedures for
View Full Document
Unlocking...