Midterm Review Machine Learning 10 701 Tom M Mitchell Machine Learning Department Carnegie Mellon University March 1 2011 See practice exams on our website Attend recitation tomorrow Midterm is open book open notes NO computers Covers all material presented up through today s class Some Topics We ve Covered Decision trees entropy mutual info overfitting Linear Regression minimizing sum sq error why regularization MAP Probability basics rv s manipulating probabilities Bayes rule MLE MAP conditional indep Sources of Error Na ve Bayes Bayesian Networks conditional independence of parameters to estimate decision surface Logistic regression form of P Y X generative vs discriminative unavoidable error bias variance Overfitting and Avoiding it factored representation of joint distribution conditional independence assumptions D separation inference in Bayes nets learning from fully partly observed data Clustering mixture of Gaussians EM 1 Understanding Comparing Learning Methods Form of learned model Inputs Outputs Optimization Objective Algorithm Assumptions Guarantees Decision boundary Generative Discriminative Na ve Bayes Logistic Regression Form of learned model Inputs Outputs Optimization Objective Algorithm Assumptions Guarantees Decision boundary Generative Discriminative 2 Four Fundamentals for ML 1 Learning is an optimization problem many algorithms are best understood as optimization algs what objective do they optimize and how Four Fundamentals for ML 1 Learning is an optimization problem many algorithms are best understood as optimization algs what objective do they optimize and how 2 Learning is a parameter estimation problem the more training data the more accurate the estimates MLE MAP M Conditional LE to measure accuracy of learned model we must use test not train data 3 Four Fundamentals for ML 1 Learning is an optimization problem many algorithms are best understood as optimization algs what objective do they optimize and how 2 Learning is a parameter estimation problem the more training data the more accurate the estimates MLE MAP M Conditional LE to measure accuracy of learned model we must use test not train data 3 Error arises from three sources unavoidable error bias variance Bias and Variance given some estimator Y for some parameter we note Y is a random variable why the bias of estimator Y the variance of estimator Y consider when is the probability of heads for my coin Y proportion of heads observed from 3 flips consider when is the vector of correct parameters for learner Y parameters output by learning algorithm 4 Four Fundamentals for ML 1 Learning is an optimization problem many algorithms are best understood as optimization algs what objective do they optimize and how 2 Learning is a parameter estimation problem the more training data the more accurate the estimates MLE MAP M Conditional LE to measure accuracy of learned model we must use test not train data 3 Error arises from three sources unavoidable error bias variance 4 Practical learning requires making assumptions Why form of the f X Y or P Y X to be learned priors on parameters MAP regularization Conditional independence Naive Bayes Bayes nets HMM s 5
View Full Document
Unlocking...