TEMPLE CIS 595 - A Tutorial On Learning With Bayesian Networks

Unformatted text preview:

A Tutorial On Learning With Bayesian NetworksOutlineIntroductionWhat do Bayesian Networks and Bayesian Methods have to offer ?The Bayesian Approach to Probability and StatisticsSome Criticisms of Bayesian ProbabilitySome Answers ……Scaling ProblemProbability assessmentLearning with DataProblem ………Two ApproachesThe other approachSome basic probability formulasLikelihood functionSufficient statisticsFinally ……….To remember…Maximum Likelihood EstimationWhat is a Bayesian Network ?DefinitionSome conventions……….An ExampleTasksA technique of constructing a Bayesian NetworkProblemsBayesian inferenceLearning Probabilities in a Bayesian NetworkAssumptions to compute the posterior probabilityBut……Obvious concerns….Incomplete data (contd)The various methods of approximations for Incomplete DataGibb’s SamplingProblem in Monte Carlo methodCriteria for Model SelectionRelative posterior probabilityLocal CriteriaPriorsPriors on network parametersIllustration of independent equivalencePriors on structuresBenefits of learning structuresSearch MethodsBayesian Networks for Supervised and Unsupervised learningWhat is all this good for anyway????????Limitations Of Bayesian NetworksConclusionSome CommentsHaimonti Dutta , Department Of Computer And Information Science1 David HeckerMannA Tutorial On Learning WithBayesian NetworksHaimonti Dutta , Department Of Computer And Information Science2Outline•Introduction•Bayesian Interpretation of probability and review methods•Bayesian Networks and Construction from prior knowledge•Algorithms for probabilistic inference•Learning probabilities and structure in a bayesian network•Relationships between Bayesian Network techniques and methods for supervised and unsupervised learning•ConclusionHaimonti Dutta , Department Of Computer And Information Science3Introduction A bayesian network is a graphical model for probabilistic relationships among a set of variablesHaimonti Dutta , Department Of Computer And Information Science4What do Bayesian Networks and Bayesian Methods have to offer ?•Handling of Incomplete Data Sets•Learning about Causal Networks•Facilitating the combination of domain knowledge and data•Efficient and principled approach for avoiding the over fitting of dataHaimonti Dutta , Department Of Computer And Information Science5The Bayesian Approach to Probability and StatisticsBayesian Probability : the degree of belief in that eventClassical Probability : true or physical probability of an eventHaimonti Dutta , Department Of Computer And Information Science6Some Criticisms of Bayesian Probability•Why degrees of belief satisfy the rules of probability•On what scale should probabilities be measured?•What probabilites are to be assigned to beliefs that are not in extremes?Haimonti Dutta , Department Of Computer And Information Science7 Some Answers ……•Researchers have suggested different sets of properties that are satisfied by the degrees of beliefHaimonti Dutta , Department Of Computer And Information Science8 Scaling Problem The probability wheel : a tool for assessing probabilities What is the probability that the fortune wheel stops in the shaded region?Haimonti Dutta , Department Of Computer And Information Science9Probability assessmentAn evident problem : SENSITIVITY How can we say that the probability of an event is 0.601 and not .599 ? Another problem : ACCURACY Methods for improving accuracy are available in decision analysis techniquesHaimonti Dutta , Department Of Computer And Information Science10Learning with Data Thumbtack problemWhen tossed it can rest on either heads or tailsHeads TailsHaimonti Dutta , Department Of Computer And Information Science11Problem ………From N observations we want to determine the probability of heads on the N+1 th toss.Haimonti Dutta , Department Of Computer And Information Science12Two Approaches Classical Approach :• assert some physical probability of heads (unknown)•Estimate this physical probability from N observations •Use this estimate as probability for the heads on the N+1 th toss.Haimonti Dutta , Department Of Computer And Information Science13The other approachBayesian Approach•Assert some physical probability•Encode the uncertainty about this physical probability using the Bayesian probailities•Use the rules of probability to compute the required probabilityHaimonti Dutta , Department Of Computer And Information Science14Some basic probability formulas•Bayes theorem : the posterior probability for  given D and a background knowledge  : p(/D, ) = p( /  ) p (D/  ,  )P(D / )Where p(D/ )= p(D/ , ) p( / ) d  Note :  is an uncertain variable whose value corresponds to the possible true values of the physical probabilityHaimonti Dutta , Department Of Computer And Information Science15Likelihood function How good is a particular value of  ? It depends on how likely it is capable of generating the observed data L ( :D ) = P( D/  )Hence the likelihood of the sequence H, T,H,T ,T may be L ( :D ) =  . (1- ). . (1- ). (1- ).Haimonti Dutta , Department Of Computer And Information Science16Sufficient statisticsTo compute the likelihood in the thumb tack problem we only require h and t (the number of heads and the number of tails) h and t are called sufficient statistics for the binomial distributionA sufficient statistic is a function that summarizes from the data , the relevant information for the likelihoodHaimonti Dutta , Department Of Computer And Information Science17Finally ……….We average over the possible values of  to determine the probability that the N+1 th toss of the thumb tack will come up heads P(X =heads / D,) = p(/D, ) dn+1The above value is also referred to as the Expectation of  with respect to the distribution p(/D, )Haimonti Dutta , Department Of Computer And Information Science18To remember…We need a method to assess the prior distribution for  .A common approach usually adopted is assume that the distribution is a beta distribution.Haimonti Dutta , Department Of Computer And Information Science19Maximum Likelihood EstimationMLE principle : We try to learn the parameters that maximize the likelihood functionIt is one of the most commonly used estimators in statistics and is intuitively appealingHaimonti Dutta , Department Of Computer And Information Science20 A


View Full Document

TEMPLE CIS 595 - A Tutorial On Learning With Bayesian Networks

Download A Tutorial On Learning With Bayesian Networks
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view A Tutorial On Learning With Bayesian Networks and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view A Tutorial On Learning With Bayesian Networks 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?