Course OutlineECE271A – Statistical Learning IDepartment of Electrical and Computer EngineeringUniversity of California, San DiegoNuno VasconcelosYour responsibilities in this class fall into three main categories:1. Take-home quizzes 20%2. Mid-term 35%3. Final 45%The HW assignments are composed of two types of problems.pen-and-paper: Each assignment includes a number of “pen and paper” problems that are notgraded. The solutions are handed out with the assignment and they are mostly for you to practice theconcepts that we discuss in class. These are the types of problems that appear in the midterm andfinal.take-home quizzes: there will be roughly one take-home quiz per HW set. These are computerproblems, where you get to implement and test some of the ideas we discuss in class. These problemsare graded and must be solved individually, without consultation of other students or resources likesites on the Internet. They are part of your evaluation for the class. You can talk to the TAs orinstructor for clarification questions. Quizzes are typically due one week after they are assigned, seethe course site for details.The official language for solving the computer problems is Matlab. I assume that students have access to itsince it is free for UCSD students. You can use other languages, like Python, but at your own risk. Dependingon the libraries you use, there could be differences between certain functions (e.g. matrix inversion, especiallywhen matrices are nearly singular) between Matlab and other languages. This may create discrepancies withthe solutions. Given the large size of the class, it is impossible for the TAs to take this into account. Thesolutions are produced with Matlab and these issues should not rise if you use Matlab.Note, that there are always issues that depend on your particular implementation, say how you handleimage borders. There are many ways to do this, zero-padding, copying pixels, etc., and there is “no rightway” to do it. However, these variations do not affect the results significantly. Hence, the grading alreadyaccounts and accepts these type of variations.InstructorNuno Vasconcelos,EBU1 5602, 4-5550, e-mail: [email protected] hours: see course siteTASee course site.Exam dates:Mid-term - In class, lecture 11 (assuming 1.5h lectures)Final - finals weekText: We will follow closelyRichard O. Duda, Peter E. Hart and David G. Stork Pattern Classification. New York, NY: JohnWiley&Sons, 2001.Supplementary hand-outs will be distributed when appropriate. There are various other books of interest.These are not required but can be used for alternative explanations of the material.1. C. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.2. T. Hastie, R. Tibshirani, J. H. Friedman, The Elements of Statistical Learning. Springer Verlag, 2001.3. Luc Devroye, Laszlo Gyorfi, Gabor Lugosi, A Probabilistic Theory of Pattern Recognition. SpringerVerlag, 1998.4. Andrew Gelman, Donald B. Rubin, Hal S. Stern, Bayesian Data Analysis, Second Edition, CRC Press;2nd edition, 2003.5. Tom Mitchell, Machine Learning, McGraw-Hill, 1997.6. Christopher Bishop, Neural Networks for Pattern Recognition. Oxford University Press, 1996.7. Vladimir Vapnik, The Nature of Statistical Learning Theory. Springer Verlag, 1999.There is a web page for the course,http://www.svcl.ucsd.edu/courses/ece271A/ece271A.htm(also accessible from http://www.svcl.ucsd.edu/~nuno)LECTURE SUBJECT Number of classesIntroduction 1Bayesian decision theory 2The Gaussian classifier 1Maximum likelihood estimation 1Bias and variance 2Bayesian parameter estimation 2Conjugate and non-informative priors 1Dimensionality and dimensionality reduction 2The nearest neighbor classifier 1Kernel-based density estimation 1Mixture models and EM 3Applications