1 Introduction Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University September 8th, 2008 Readings: Review: K&F: *2.1*, 2.2, 2.3 K&F: 3.1 10-708 – ©Carlos Guestrin 2008 2 One of the most exciting developments in machine learning (knowledge representation, AI, EE, Stats,…) in the last two (or three, or more) decades… My expectations are already high… 2 10-708 – ©Carlos Guestrin 2008 3 Speech recognition Hidden Markov models and their generalizations 10-708 – ©Carlos Guestrin 2008 4 Tracking and robot localization [Fox et al.] [Funiak et al.] Kalman Filters3 10-708 – ©Carlos Guestrin 2008 5 Evolutionary biology [Friedman et al.] Bayesian networks 10-708 – ©Carlos Guestrin 2008 6 Modeling sensor data Undirected graphical models [Guestrin et al.]4 10-708 – ©Carlos Guestrin 2008 7 Planning under uncertainty F’ E’ G’ P’ Peasant Footman Enemy Gold R t t+1 Time APeasant ABuild AFootman P(F’|F,G,AB,AF) [Guestrin et al.] Dynamic Bayesian networks Factored Markov decision problems 10-708 – ©Carlos Guestrin 2008 8 Images and text data [Barnard et al.] Hierarchical Bayesian models5 10-708 – ©Carlos Guestrin 2008 9 Structured data (text, webpages,…) [Koller et al.] Probabilistic relational models 10-708 – ©Carlos Guestrin 2008 10 And many many many many many more…6 10-708 – ©Carlos Guestrin 2008 11 Syllabus Covers a wide range of Probabilistic Graphical Models topics – from basic to state-of-the-art You will learn about the methods you heard about: Bayesian networks, Markov networks, factor graphs, conditional random fields, decomposable models, junction trees, parameter learning, structure learning, semantics, exact inference, variable elimination, context-specific independence, approximate inference, sampling, importance sampling, MCMC, Gibbs, variational inference, loopy belief propagation, generalized belief propagation, Kikuchi, Bayesian learning, missing data, EM, Chow-Liu, structure search, IPF for tabular MRFs, Gaussian and hybrid models, discrete and continuous variables, temporal and template models, hidden Markov Models, Forwards-Backwards, Viterbi, Baum-Welch, Kalman filter, linearization, switching Kalman filter, assumed density filtering, DBNs, BK, Relational probabilistic models, Causality,… Covers algorithms, theory and applications It’s going to be fun and hard work 10-708 – ©Carlos Guestrin 2008 12 Prerequisites 10-701 – Machine Learning, especially: Probabilities Distributions, densities, marginalization… Basic statistics Moments, typical distributions, regression… Algorithms Dynamic programming, basic data structures, complexity… Programming Matlab will be very useful We provide some background, but the class will be fast paced Ability to deal with “abstract mathematical concepts”7 10-708 – ©Carlos Guestrin 2008 13 Review Sessions Very useful! Review material Present background Answer questions Thursdays, 5:00-6:20 in Wean Hall 5409 First recitation is this Thursday Review of probabilities & statistics Sometimes this semester: Especial recitations most likely on Mondays 5:30-7pm Cover special topics that we can’t cover in class These are optional, but you are here to learn… Do we need a Matlab review session? 10-708 – ©Carlos Guestrin 2008 14 Staff Two Great TAs: Great resource for learning, interact with them! Amr Ahmed <[email protected]>, Dhruv Batra <[email protected]> Administrative Assistant Michelle Martin <[email protected]>, Wean 4619, x8-55278 10-708 – ©Carlos Guestrin 2008 15 First Point of Contact for HWs To facilitate interaction, a TA will be assigned to each homework question – This will be your “first point of contact” for this question But, you can always ask any of us For e-mailing instructors, always use: [email protected] For announcements, subscribe to: 10708-announce@cs https://mailman.srv.cs.cmu.edu/mailman/listinfo/10708-announce We will also use a discussion group: Post your questions, discuss projects, etc Be nice… Don’t give away any answers… http://groups.google.com/group/10708-f08 10-708 – ©Carlos Guestrin 2008 16 Text Books Primary: Daphne Koller and Nir Friedman, Structured Probabilistic Models, in preparation. These chapters are part of the course reader. You can purchase one from Michelle Martin Secondary: M. I. Jordan, An Introduction to Probabilistic Graphical Models, in preparation. Copies of selected chapters will be made available.9 10-708 – ©Carlos Guestrin 2008 17 Grading 5 homeworks (50%) First one goes out next Wednesday! Homeworks are long and hard please, please, please, please, please, please start early!!! Final project (30%) Done individually or in pairs Details out soon Proposals due October 6th Final (20%) Take home, out Dec. 3rd Due Dec. 10th at NOON (hard deadline) 10-708 – ©Carlos Guestrin 2008 18 Homeworks Homeworks are hard, start early Due in the beginning of class 3 late days for the semester After late days are used up: Half credit within 48 hours Zero credit after 48 hours All homeworks must be handed in, even for zero credit Late homeworks handed in to Michelle Martin, WEH 4619 Collaboration You may discuss the questions Each student writes their own answers Write on your homework anyone with whom you collaborate IMPORTANT: We may use some material from previous years or from papers for the homeworks. Unless otherwise specified, please only look at the readings when doing your homework ! You are taking this advanced graduate class because you want to learn, so this rule is self-enforced 10 10-708 – ©Carlos Guestrin 2008 19 Enjoy! NO CLASS THIS WEDNESDAY 9/10 Probabilistic graphical models are having significant impact in science, engineering and beyond This class should give you the basic foundation for applying GMs and developing new methods The fun begins… 10-708 – ©Carlos Guestrin
View Full Document