1CS 536: Machine LearningFall 2003: Michael L. LittmanTA: Yihua WuWelcome!My first class here and first time teaching ML.• Good news: I’m still energetic• Bad news: You’re my guinea pigs.2TextbookMachine Learning by Tom M. Mitchellhttp://www-2.cs.cmu.edu/afs/cs.cmu.edu/user/mitchell/ftp/mlbook.html• Chapters 1, 3-10, 13• Supplementary material on SVMs,clustering• Feedback from you.Course Goals• Introduce the background and “lore” ofmachine learning.• Prepare you to be able to use ML tools,contribute to the field.• Introduce you to the research process.Not: Find out details of the state of the art…3Course Mechanics• Problem sets• Term project, short paper, peer review!• Midterms• FinalRepeat with me: “I am not an undergrad…”Mini-Conference (icml-2003)Choose a research projectDo the research and some background readingPrepare a 4-page paper (ICML format)Submit the paper after midtermsEach paper will have 3 anonymous reviewersReviewers submit their reviews, discussBest ~25% of papers presented to class.Additional 25% presented as posters.All authors revise papers based on reviews.4Example: SnowballsThe snowball game: You are having asnowball battle with a neighbor over yourbackyard fence. You have a catapult withforce and angle controls. You need to hityour neighbor’s fort 10 times to win.Snowballs: DetailsYou catapult’s controls (force, angle) are setrandomly. You decide:• don’t fire (costs 1 unit)• fire (costs 2 units)• get distance travelled (costs 5 units)• check hit or miss (costs 100 units)How minimize cost to achieve the objective?5Some Types of LearningSupervised, classification: vectorÆBooleanSupervised, regression: vectorÆrealSupervised, general: vectorÆvectorUnsupervised, discrete: vector ÆclusterReinforcement, associative: vectorÆaction, via realReinforcement, temporal: vectorÆaction, viadelayed realProject Ideas (pg. 1)Given a web page that (probably) contains glossaryentries and definitions, extract the fields.• http://www.twjc.co.uk/glossary.html• http://www.cs.unc.edu/~helser/juggler-0.81/glossary.htmlGiven multiple database with addresses, create aunified database of places.Create a more accurate battery power indicator.Extract titles, authors, references from pdf files.Self organization of a peer-to-peer network.6Project Ideas (pg. 2)Predict server response time for nodes in a wirelessnetwork.In RL, there are several algorithms that trade offexploration and exploitation in a theoreticallymotivated way. Evaluate them empirically.Compare existing RL techniques for “mountain car”or Tetris.Figure out how to beat a fixed set of TAC agents.Compare techniques for merging probabilitydistributions theoretically.Project Ideas (pg. 3)To solve multiple choice synonym questions, we’veshown that multiple experts is a smart way to dothis. Training is done using supervised data. Canthe multiple modules be used to train each other?(”Labeling via collectives of sufficiently accuratemodules”).How about modules for RL? Is there an advantagefor doing policy search, table, neural net alltogether?7Project Ideas (pg. 4)Applications: Natural language dialog,robotics, financial trading, networkdiagnosis, object recognition, combiningspeech and commands and images, problemsolving (Sokoban), video gameWhat data do you have?Meet the Class (pg. 1)Signed up:Vasilios Daskalopoulos (CS)Jixin Li (CS)Mark Rogaski (CS)Georgios Sakkis (CS)Mark Sharp (SCILS)Alexander Strehl (CS)Zhi Wei (CS)Jiankuan Ye (CS)Permission:Bing Bai (CS/SCILS)Chan su Lee (CBIM)Igor Gierymski (Bus.) XLu Liu (SCILS)Paul Batchis (CS)Christopher Peery (CS)Rong Xu (CS)Rui Huang (CS/CBIM)…8Meet the Class (pg. 2)Sabrina R. Li (Stat)Dave LeRoux (CS)Xiaoxia Ren (CS)Zhiguo Li (CS/CBIM)Sign up Sheet9Next TimeMe:• syllabus• web page• snowball simulatorTA• available ML resourcesYou:• Ch.