General InformationWhat is Machine Learning?Applications of Machine LearningPrerequisitesTextbooksGrading Spring 2011Topics Covered in 2011 (Based on Alpaydin)Course ProjectsTentative ML Spring 2009 ScheduleCourse ElementsDates to RememberExamsOther UH-CS Courses with Overlapping ContentsGeneral InformationGeneral InformationCourse Id: COSC6342 Machine LearningTime: TU/TH 1-2:30pInstructor: Christoph F. Eick Classroom: AH301E-mail: [email protected]: http://www2.cs.uh.edu/~ceick/22What is Machine Learning?What is Machine Learning?Machine Learning Machine Learning is theis the•study of algorithms thatstudy of algorithms that•improve their performanceimprove their performance•at some taskat some task•with experiencewith experienceRole of Statistics: Inference from a sampleRole of Statistics: Inference from a sampleRole of Computer science: Efficient algorithms toRole of Computer science: Efficient algorithms to•Solve optimization problemSolve optimization problemss•Representing and evaluating the model for Representing and evaluating the model for inferenceinference33ApplicationsApplications of Machine Learning of Machine LearningSupervised LearningSupervised Learning•ClassificationClassification•Prediction Prediction Unsupervised LearningUnsupervised Learning•Association Analysis Association Analysis •Clustering Clustering Preprocessing and Summarization of DataPreprocessing and Summarization of DataReinforcement LearningReinforcement LearningActivities Related to Models Activities Related to Models •Learning parameters of modelsLearning parameters of models•Choosing/Comparing modelsChoosing/Comparing models•Evaluating Models (e.g. predicting their accuracy)Evaluating Models (e.g. predicting their accuracy)Prerequisites Prerequisites BackgroundBackgroundProbabilitiesProbabilities•Distributions, densities, marginalization…Distributions, densities, marginalization…Basic statisticsBasic statistics•Moments, typical distributions, regression Moments, typical distributions, regression Basic knowledge of optimization techniquesBasic knowledge of optimization techniquesAlgorithmsAlgorithms•basic data structures, complexity…basic data structures, complexity…Programming skillsProgramming skillsWe provide some background, but the class will be fast We provide some background, but the class will be fast pacedpacedAbility to deal with “abstract mathematical concepts”Ability to deal with “abstract mathematical concepts”TextbooksTextbooksTextbook:Textbook: Ethem Alpaydin, Introduction to Machine Learning, MIT Press, 2010. Mildly Recommended Textbooks:Mildly Recommended Textbooks:1.Christopher M. Bishop, Pattern Recognition and Machine Learning, 2006. 2.Tom Mitchell, Machine Learning, McGraw-Hill, 1997.Grading Spring 2011Grading Spring 20112 Exams 63-69%3 Projects 29-35%Attendance 1-2% NOTE: PLAGIARISM IS NOT TOLERATED. Remark: Weights are subject to changeTopics Covered in 2011 (Based on Alpaydin)Topics Covered in 2011 (Based on Alpaydin)Topic 1: Introduction Topic 2: Supervised Learning Topic 3: Bayesian Decision Theory (excluding Belief Networks) Topic 5: Parametric Model Selection Topic 6: Dimensionality Reduction Centering on PCA Topic 7: Clustering1: Mixture Models, K-Means and EM Topic 8: Non-Parametric Methods Centering on kNN and Density Estimation Topic 9: Clustering2: Density-based ApproachesTopic 10: Decision Trees Topic 11: Comparing Classifiers Topic 12: Combining Multiple Learners Topic 13: Linear Discrimination Topic 14: More on Kernel Methods Topic 15: Naive Bayes' and Belief Networks Topic 16: Hidden Markov Models Topic 17: Sampling Topic 18: Reinforcement Learning Topic 19: Neural Networks Topic 20: Computational Learning Theory Remark: Topics 16, 17, 19, and 20 likely will be only briefly covered or skipped---due to the lack of time.Course ProjectsCourse Projects1.1.February 2011: Individual Project; Classification February 2011: Individual Project; Classification and Prediction; learn how obtain, use, and evaluate and Prediction; learn how obtain, use, and evaluate models. models. 2.2.March 2011: Group Project, giving a survey about March 2011: Group Project, giving a survey about a subfield of Machine Learninga subfield of Machine Learning3.3.April 2011: Individual Project; Reinforcement April 2011: Individual Project; Reinforcement Learning and Adaptation; Learn how to act Learning and Adaptation; Learn how to act intelligently in an unknown/changing environmentintelligently in an unknown/changing environmentTentative Tentative ML Spring 2009 ScheduleML Spring 2009 ScheduleWeek TopicJan 20IntroductionJan 27Supervised Learning/Bayesian Decision TheoryFeb. 3Curve Fitting/Model Estimation---Parametric ApproachesFeb. 10Model Estimation---Parametric ApproachesFeb. 17Parametric Approaches/Clustering1Feb. 24Clustering1/Non-param MethodsMarch 3Non-Param Methods/Exam1March 10Clustering2/Dim. Reduction,Decision Trees March 24Dim. Reduction; DecisionTrees /Exam2March 31SVMs/Kernel Methods; Ensemble MethodsApril 7Comparing Classifiers/Group1 PresentationsApril 14Group2 Presentations/TBDLApril 21Reinforcement Learning/possibly Belief NetworksApril 28Review/Exam3March 31, 2009Course ElementsCourse ElementsTotal: 25-26 classes Total: 25-26 classes • 18-19 lectures18-19 lectures• 3-4 classes for review and discussing course projects 3-4 classes for review and discussing course projects • 2 classes will be allocated for student presentations2 classes will be allocated for student presentations• 2 exams2 exams• Ungraded homeworksUngraded homeworks• Several Problems will be given and solutions will be Several Problems will be given and solutions will be presented and discussed a week laterpresented and discussed a week laterDates to RememberDates to RememberDates to remember EventsMarch 10 + May ?? ExamsApril 5+7 Project2 Student Project PresentationsMarch 15 /17 No class (Spring Break)Feb. 28, March 31, April 28Submit Project Report /Software/…ExamsExams Will be open notes/textbookWill be open notes/textbook Will get a review list before the examWill get a review list before the exam Exams will center (80% or more) on material that was Exams will center (80% or more) on material that was covered in the lecturecovered in the lecture There will be a review prior to the second and third exam; There will be a review