IntroductionSpeech recognitionTracking and robot localizationEvolutionary biologyModeling sensor dataPlanning under uncertaintyImages and text dataStructured data (text, webpages,…)SyllabusPrerequisitesReview SessionsStaffFirst Point of Contact for HWsText BooksGradingHomeworksEnjoy!What are the fundamental questions of graphical models?More details???Where do we start?TodayEvent spacesProbability distribution P over (,S)Interpretations of probability – A can of worms!Conditional probabilitiesTwo of the most important rules of the semester: 1. The chain ruleTwo of the most important rules of the semester: 2. Bayes ruleMost important concept: a) IndependenceMost important concept: b) Conditional independenceRandom variableMarginal distributionJoint distribution, MarginalizationMarginalization – The general caseBasic concepts for random variablesConditionally independent random variablesProperties of independenceBayesian networksHandwriting recognitionWebpage classificationHandwriting recognition 2Webpage classification 2Let’s start on BNs…What if variables are independent?Conditional parameterization – two nodesConditional parameterization – three nodesThe naïve Bayes model – Your first real Bayes NetWhat you need to knowNext classIntroductionGraphical Models – 10708Carlos GuestrinCarnegie Mellon UniversitySeptember 13th, 2006Readings:Review: K&F: *2.1*, 2.5, 2.6K&F: 3.110-708 –©Carlos Guestrin 20062One of the most exciting developments in machine learning (knowledge representation, AI, EE, Stats,…) in the last two (or three, or more) decades…My expectations are already high… ☺10-708 –©Carlos Guestrin 20063Speech recognitionHidden Markov models and their generalizations10-708 –©Carlos Guestrin 20064Tracking and robot localizationKalman Filters[Fox et al.][Funiak et al.]10-708 –©Carlos Guestrin 20065Evolutionary biologyBayesian networks[Friedman et al.]10-708 –©Carlos Guestrin 20066Modeling sensor dataUndirected graphical models[Guestrin et al.]10-708 –©Carlos Guestrin 20067Planning under uncertaintyDynamic Bayesian networksFactored Markov decision problemsF’E’G’P’PeasantFootman Enemy GoldRtt+1TimeAPeasantABuildAFootmanP(F’|F,G,AB,AF)[Guestrin et al.]10-708 –©Carlos Guestrin 20068Images and text dataHierarchical Bayesian models[Barnard et al.]10-708 –©Carlos Guestrin 20069Structured data (text, webpages,…)Probabilistic relational models[Koller et al.]10-708 –©Carlos Guestrin 200610And manymanymanymanymanymore…10-708 –©Carlos Guestrin 200611Syllabus Covers a wide range of Probabilistic Graphical Models topics – from basic to state-of-the-art You will learn about the methods you heard about: Bayesian networks, Markov networks, factor graphs, decomposable models, junction trees, parameter learning, structure learning, semantics, exact inference, variable elimination, context-specific independence, approximate inference, sampling, importance sampling, MCMC, Gibbs, variational inference, loopy belief propagation, generalized belief propagation, Kikuchi, Bayesian learning, missing data, EM, Chow-Liu, structure search, IPF for tabular MRFs, Gaussian and hybrid models, discrete and continuous variables, temporal and template models, hidden Markov Models, Forwards-Backwards, Viterbi, Baum-Welch, Kalman filter, linearization, switching Kalman filter, assumed density filtering, DBNs, BK, Relational probabilistic models, Causality,… Covers algorithms, theory and applications It’s going to be fun and hard work ☺10-708 –©Carlos Guestrin 200612Prerequisites 10-701 – Machine Learning, especially: Probabilities Distributions, densities, marginalization… Basic statistics Moments, typical distributions, regression… Algorithms Dynamic programming, basic data structures, complexity… Programming Matlab will be very useful We provide some background, but the class will be fast paced Ability to deal with “abstract mathematical concepts”10-708 –©Carlos Guestrin 200613Review Sessions Very useful! Review material Present background Answer questions Thursdays, 5:00-6:30 in Wean Hall 4615A First recitation is tomorrow Review of probabilities & statistics Sometimes this semester: Especial recitations on Mondays 5:30-7pm in Wean Hall 4615A Cover special topics that we can’t cover in class These are optional, but you are here to learn… ☺ Do we need a Matlab review session?10-708 –©Carlos Guestrin 200614Staff Two Great TAs: Great resource for learning, interact with them! Khalid El-Arini <[email protected]> Ajit Paul Singh <[email protected]> Administrative Assistant Monica Hopes, Wean 4619, x8-5527, [email protected] –©Carlos Guestrin 200615First Point of Contact for HWs To facilitate interaction, a TA will be assigned to each homework question – This will be your “first point of contact” for this question But, you can always ask any of us (Due to logistic reasons, we will only start this policy for HW2) For e-mailing instructors, always use: [email protected] For announcements, subscribe to: 10708-announce@cs https://mailman.srv.cs.cmu.edu/mailman/listinfo/10708-announce10-708 –©Carlos Guestrin 200616Text Books Primary: Daphne Koller and Nir Friedman, Bayesian Networks and Beyond, in preparation. These chapters are part of the course reader. You can purchase one from Monica Hopes. Secondary: M. I. Jordan, An Introduction to Probabilistic Graphical Models, in preparation. Copies of selected chapters will be made available.10-708 –©Carlos Guestrin 200617Grading 5 homeworks (50%) First one goes today! Homeworks are long and hard ☺ please, please, please, please, please, please start early!!! Final project (30%) Done individually or in pairs Details out October 4th Final (20%) Take home, out Dec. 1st, due Dec. 15th10-708 –©Carlos Guestrin 200618Homeworks Homeworks are hard, start early ☺ Due in the beginning of class 3 late days for the semester After late days are used up: Half credit within 48 hours Zero credit after 48 hours All homeworks must be handed in, even for zero credit Late homeworks handed in to Monica Hopes, WEH 4619 Collaboration You may discuss the questions Each student writes their own answers Write on your homework anyone with whom you collaborate IMPORTANT: We
View Full Document
Unlocking...