Ockham Efficiency Theorem for EmpiricalMethods Conceived as Empirically-Driven,Countable-State Stochastic ProcessesKevin KellyConor Mayo-WilsonNovember 29, 2008AbstractOckham’s razor is the principle that, all other things being equal, itis rational to prefer simpler scientific theories to more complex ones. In aseries of a pap ers, Kelly, Glymour, and Schulte argue that scientists whoheed Ockham’s razor make fewer errors and retract their opinions lessoften than do their complexity preferring counterparts. The centerpieceof their argument is the Ockham Efficiency Theorem, which provides aprecise explanation of errors, retractions, and Ockham’s razor within amo del of scientific inquiry developed by formal-learning theorists. Kelly,Glymour, and Schulte’s previous arguments, however, were restricted intwo important ways: (1) they applied only to deterministic (rather thanrandomized) methods for choosing scientific theories from data and (2)they failed to successfully model inference from statistical data with er-ror. In this paper, we full address the first issue by extending the OckhamEfficiency Theorem to prove that, amongst any set of randomized strate-gies, a systematic preference for simpler theories minimizes the number oferrors and retractions one commits before converging to the true theory.By incorporating probabilistic elements into the model employed by for-mal learning theorists, moreover, we take a large step towards addressingthe second issue as well.1 IntroductionFrom the Copernican revolution to Einstein’s jettisoning of absolute space frommechanics, some of the most celebrated advances in the history of science weremotivated in part by a desire to simplify existing theories. The systematic pref-erence for simpler theories, moreover, still dominates scientific practice today.Faced with multiple competing theories that are all compatible with existingexperimental and observational evidence, scientists eschew complexity in favorof theories with fewer laws, fewer free parameters, fewer postulated causes, fewerfundamental entities (e.g. particles), and so on. Moreover, this systematic bias1for simpler theories is often tacitly built into computer-statistical packages thathave become the modern-day toolbox for working scientists. But why shouldscientists favor simpler scientific theories when the world might, in fact, be ex-tremely complicated? In particular, is there any reason to believe that simplertheories are more likely to be true?To answer these questions, many philosophers have argued that simpler the-ories possess other theoretical virtues. Simpler theories, they claim, are moreunified (Friedman), more easily falsified or tested (Popper, Mayo), more ex-planatory (Harman, Nolan, and Baker), and more concise, in that simpler theo-ries minimize description length (Risannen, Vitanyi, Li, Simon,). However, thescientific theory that truly describes the world might lack unity or be “dappled”(Cartwright); it might be difficult to test and/or falsify, and its explanation ofobserved phenomena might be long and convoluted. In short, unless one hasindependent reason to think that true scientific theories possess these othervirtues (unifying power, falsifiability, explanatory power, etc.), the above argu-ments provide no reason to think that simpler theories are more likely to betrue.Other philosophers and statisticians have argued that scientists who favorsimpler hypotheses will eventually endorse the true theories in the long run(Sklar, Friedman, Rozenkrantz). Yet as Reichenbach first noted, and was sub-sequently endorsed by Hempel and Salmon, almost any arbitrary bias is com-patible with finding true scientific theories eventually. [Finish - describe priorwashing out in long run]So-called Bayesians and confirmation theorists, argue that simpler theoriesare better confirmed, and hence, there is reason to think that simpler theories aremore likely to be true. Such arguments, however, either explicitly (Jeffreys) orimplicitly (Rosenkrantz) assume that simpler theories are assigned higher priorprobabilities.1But then simpler theories are more probable precisely becauseone assumes them to be more probable. Clearly, such circular arguments areunacceptable.Most recently, a number of philosophers have harnessed mathematical theo-rems from statistics and machine learning to argue simpler theories make betterpredictions (Harman and Kulkarni, Vapnik, Forster, Sob er, and Hitchcock). Butthe theorems prove too much: simpler theories, according to the theorems, makebetter predictions regardless of whether they are true or not. For this reason,Vladamir Vapnik, the inventor of statistical learning theory, argues that oneshould use simpler statistical models in many practical applications even whenit is known that the simpler model is false. Astute philosophers have noticed thisfeature of the statistical theorems they employ. For example, Forster, Sober,and Hitchcock argue that simplicity is merely instrumental in helping one makebetter predictions, but it is no indication of the truth of a scientific theory.In a series of papers, Kevin Kelly, Clark Glymour, and Oliver Schulte haveprovided a more nuanced and promising defense of Ockham’s razor. They ar-gue that scientists who systematically favor simpler hypotheses will make fewer1Explain why uniform prior on simpler and complex theories begs the question.2errors (i.e. they will endorse false theories less often), and they will retractpreviously endorsed theories fewer times before ultimately settling on the truetheory in the long run. This thesis is captured by the slogan, “Ockham’s razorequals efficient convergence to the truth.” Importantly, the model of scientificinquiry developed by these authors provides a successful explanation of whysimpler theories ought to be preferred in a number of scientific problems in-cluding curve-fitting, causal inference, and estimating conserved quantities inparticle physics.Kelly, Glymour, and Schulte’s arguments, however, only consider determinis-tic methods for choosing scientific theories from observed data. In game theory,it is familiar that the use of randomized strategies often allows one to minimizecosts (or maximize gains) in a way in which deterministic strategies cannot.Thus, an important question is the following:“Does a scientist who employs Ockham’s razor minimize errors and retractionsin converging to truth, when compared, as well, to scientists capable of