Learning• Webster– To gain knowledge or understanding of or skill in by study,instruction or experience– To memorize– Synonym: discover∗ To obtain knowledge of for the first time∗ May imply acquiring knowledge with little effort or con-scious intention (as by simply being told) or it may implystudy and practice– Knowledge∗ Knowing something with familiarity gained through ex-perience or association∗ Facts or ideas acquired by study, investigation, observa-tion, or experience• Deduction? (6!)• Knowledge representation?• Performance measure?1Machine Learning• Simon– Any process by which a system improves its performance• Expert systems– Acquisition of explicit knowledge• Psychologists– Skill acquisition• Scientists– Theory formation, hypothesis formation and inductive infer-ence2Machine Learning: Motivations• Automated knowledge engineering– Expertise is scarce– Codification of expertise is difficult– Expertise frequently consists of a set of test cases– Data from measurements, but no information or knowledge• Only one computer has to learn, then copy• Understand human learning3Machine Learning: Applications• Speech recognition• Object recognition• Language learning• Autonomous navigation• Data mining• Intelligent agents• Cognitive modeling4History of Machine LearningExploration (1950s and 1960s)• Neurophysiological– Rosenblatt’s perceptron• Biological– Simulated evolution• Psychological– Symbol processing systems• Statistical– Control and pattern recognition– Samuel’s checkers program• Theoretical– Gold’s identification in the limit– Minsky and Papert’s criticism of the perceptron5History of Machine LearningDevelopment of practical algorithms (1970s)• Winston’s ARCH– Learned concept of a blocks-world arch• Buchanan and Mitchell’s Meta-Dendral– Learned mass-spectrometry prediction rules• Michalski’s AQ11– Learned soybean disease diagnosis rules• Quinlan’s ID3– Learned chess end-game rules• Fikes, Hart and Nilsson’s MACROPS– Learned macro-operators in blocks-world planning• Lenat’s AM– Discovered interesting mathematical concepts6History of Machine LearningExplosion of research directions (1980s)• Learning theory• Symbolic learning algorithms• Connectionist (neural network) learning algorithms• Clustering and discovery• Explanation-based learning• Knowledge-guided inductive learning• Analogical and case-based reasoning• Genetic algorithms7History of Machine LearningMaturity of the field (1990s)• Statistical comparisons of algorithms• Theoretical analyses of algorithms• Machine learning = Data mining (?)• Successful applications• Multi-relational learning• Ensemble and Kernel Methods8Mitchell’s Book• Practical approach to study of machine learning• Methodology snapshot (good one for