1Kansas S tate UniversityDepartm ent of Computing and I nform a tion S ciencesCIS 830: Advanced Topics in Artificial IntelligenceFriday, March 3, 2000Pallavi ParanjapeDepartment of Computing and Information Sciences, KSUReadings:“In Defense of Probability”Peter CheesemanReasoning under Uncertainty(1 of 4)Lecture 20Lecture 20Ka nsas S tat e Un i versi tyDe partm en t of Com pu ti n g and In f o rm atio n Sci e ncesCIS 830: Advanced Topics in Artificial IntelligencePresentation OverviewPresentation Overview• Paper– “In Defense of Probability”– Author: Peter Cheeseman, SRI International• Overview– Concepts in Probability– Common misconceptions about probability– Probability and Logic– Subjective Probability• Goal– Probability is all that is needed for representing and reasoning about uncertainty• References– Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, byJudea PearlKa ns as S ta te Uni ver si tyDepartment of Computing and Information SciencesCIS 830: Advanced Topics in Artificial IntelligenceProbability (Concepts)Probability (Concepts)• Probability– provides a quantitative description of the likelihood of occurrence of aparticular event– Measured on a scale from 0 to 1• Conditional Probability– Denoted by P(A|B)– Probability that event A will occur given the knowledge that event B hasalready occurred• Baye’s Theorem– a result that allows new information to be used to update the conditionalprobability of an event– P(Hi|E) = P(E|Hi) . P (Hi) Σkn=1P(E|Hn).P (Hn)Ka n sas St a te Uni ver si t yDepartment of Computing and Informa tion SciencesCIS 830: Advanced Topics in Artificial IntelligenceProbability Concepts (Contd.)Probability Concepts (Contd.)• Maximum Entropy– Entropy is a measure of the information contained in the data– Maximum entropy => maximum uncertainty => uniform priors• Conditional Independence– When two events occurring are not related to each other in any way– The occurrence of an event is not dependent upon the occurrence of anotherevent.Ka n sas S ta te Uni ver sit yDe partment of Computing and Information S ciencesCIS 830: Advanced Topics in Artificial IntelligencePresentation OutlinePresentation Outline• Issues– Are the various schemes put forward for representing and reasoning aboutuncertainty really necessary– Key strengths - ‘Measure of belief’ definition of probability– Key weaknesses - highly biased opinion in favor of probability defensive approach• Outline– Different approaches to probabilistic reasoning– Common Fallacies about probability• Probability as a Frequency Ratio• Need for large quantities of Data• Prior Probabilities• Need for numbers• Numbers required to represent Uncertainty– Logic and Probability– Subjective ProbabilityKa n sas St a te Uni ver si tyDe partment of Computing and Information SciencesCIS 830: Advanced Topics in Artificial IntelligenceApproaches to Probabilistic ReasoningApproaches to Probabilistic Reasoning• Statistical Approach– Frequency Ratio definition• P(A) = number of observed occurrences of event A total observed occurrences• good definition provided we have enough sample data• Law of large numbers• Logicist Approach– deals with uncertainty using nonnumeric techniques– nonmonotonic logic• a set of beliefs is assumed to be complete• upon uncovering of evidence to the contrary, the set of beliefs arerevised to add new beliefs or to drop inconsistent beliefs.2Ka n sas St a te Uni ver si tyDe partment of Computing and Information SciencesCIS 830: Advanced Topics in Artificial IntelligenceApproaches to Probabilistic ReasoningApproaches to Probabilistic Reasoning(Contd.)(Contd.)• Subjective Approach– Describes an individual’s personal judgement about the likelihood of theoccurrence of a particular event– Biased by the most relevant examples observed• Bayesian Approach– reasoning about beliefs under uncertainty– P(A|K) = belief that event A occurs given certain knowledge K– similar to the subjective approachKa nsas S tat e Un i versi tyDepartment of Computing and Information SciencesCIS 830: Advanced Topics in Artificial IntelligenceOverviewOverview• Logical approach to inference and knowledge representation is most popular– knowledge representations -> first order predicate logic– inference procedures ->logical deduction• Human reasoning is intrinsically probabilistic– attempt to enforce it into a logical mould– alternative approaches such as default logic, nonmonotonic logic, uncertainty factorSchafer/Dempster theory, Fuzzy logic, etc. attempt to overcome some perceiveddifficulty of the probability theory• Goal– probability is a measure of belief– sufficient for all uncertain inference in AIKa n sas S ta t e Uni ver si t yDe partment of Computing and Information S ciencesCIS 830: Advanced Topics in Artificial IntelligenceCommon Fallacies about ProbabilityCommon Fallacies about Probability• Probability, a frequency ratio (most common definition)– Definition - ratio of the number of occurrences (n) in which the event A is true to thetotal number of observed occurrences (m)P(A) = m/n– restricted to domains where repeated sampling is possible– law of large numbers– no such concept as ‘the’ probability• Probability, a measure of belief (advocated definition)– measure of an entity’s belief in that proposition, given the evidence– probability can be revised when new evidence comes to light– probability will depend on the observer’s knowledge– probability is subjectiveKa n sas S ta t e Uni ver si t yDe partment of Computing and Information S ciencesCIS 830: Advanced Topics in Artificial IntelligenceCommon Fallacies about Probability (Contd.)Common Fallacies about Probability (Contd.)• Bayesian Analysis requires large amounts of data– follows from the frequency ratio definition– lack of available knowledge can be countered by making assumptions likeconditional independence and maximum entropy– using maximum entropy makes stronger predictions than what the availableinformation permits– detect new information• Prior probabilities assume more information than given– assume uniform prior probabilities and maximum likelihood– there is no unique probability associated with a proposition, it is revised as moreinformation is gainedKa n sas S ta t e Uni ver si t yDe partment of Computing