Bayes Nets Representation: joint distribution and conditional independenceYi Zhang10-701, Machine Learning, Spring 2011February 9th, 2011Parts of the slides are from previous 10-701 lectures1Outline Conditional independence (C. I.) Bayes nets: overview Local Markov assumption of BNs Factored joint distribution of BNs Infer C. I. from factored joint distributions D-separation (motivation)2Conditional independence X is conditionally independent of Y given Z◦ In short:◦ Equivalent to:3Bayes nets Bayes nets: directed acyclic graphs express sets of conditional independence via graph structure◦ All about the joint distribution of variables !◦ Conditional independence assumptions are useful ◦ Naïve Bayes model is an extreme example4Three key questions for BNs Representation: ◦ What joint distribution does a BN represent? Inference◦ How to answer questions about the joint distribution? Conditional independence Marginal distribution Most likely assignment Learning◦ How to learn the graph structure and parameters of a BN from data? 5Local Markov assumptions of BNs A variable X is independent of its non-descendants given (only) its parents◦ Intuition: “flu” and “allergy” causes “headache” only through “sinus”6Local Markov assumptions of BNs A variable X is independent of its non-descendants given (only) its parents7Local Markov assumptions of BNs Local Markov assumptions only express a subsetof C.I.s on a BN◦ Is XMconditionally independent of X1given X2? But they are sufficient to infer all others 8Factored joint distribution of a BN A BN can represent the joint distributions of the following form:9Factored joint distribution of a BN A BN can represent the joint distributions of the following form:10Factored joint distribution of a BN Local Markov assumptions imply the factored joint distribution11Factored joint distribution of a BN Naïve Bayes◦ Local Markov assumptions: ◦ Factored joint distribution:12Infer C.I. from the factored joint distribution We already see: local Markov assumptions factored joint distribution Also, factored joint distribution all C.I. in the BN13Infer C.I. from the factored joint distribution Factored Joint distribution Show that14Infer C.I. from the factored joint distribution Factored Joint distribution Do we have ? In general, no.◦ Cannot be written into two separate terms of a and b15D-separation: motivation Is XMconditionally independent of X1given X2?◦ Intuitively yes: X1affects XMonly through X2.◦ Method 1: using factored joint distribution to derive◦ Method II: D-separation --- not
View Full Document
Unlocking...