1CS 559: Machine LearningCS 559: Machine Learning Fundamentals and Applications1stSet of NotesInstructor: Philippos MordohaiWebpage: www.cs.stevens.edu/~mordohaiEmail:Philippos Mordohai@stevens eduE-mail: [email protected]: Lieb 215ObjectivesObjectives•Obtain hands-on experience with and beObtain handson experience with and be able to implement fundamental algorithmsUseful for everyday problems–Useful for everyday problems• Be able to use state of the art machine learning and pattern recognition tools for advanced problems2Important PointsImportant Points•This is an elective course You chose to beThis is an elective course. You chose to be here.•Expect to work and to be challenged•Expect to work and to be challenged.• Exams won’t be based on recall. They will b b k d ill b dbe open book and you will be expected to solve new problems.3Important Points IIImportant Points II• Always ask:–What are we classifying?– What is known, what is unknown?–Which are the classes/labels/options?Which are the classes/labels/options?– What is the objective function?•At any point ask me WHY?yp• You can ask me anything about the course in class, during a break, in my office, by email. If thi k h k i t ki t l i–If you think a homework is taking too long or is wrong.– If you can’t decide on a project.–Etc. etc.Etc. etc.4LogisticsLogistics•Office hours: Tuesday 5-6 and by emailOffice hours: Tuesday 56 and by email• Evaluation:4 h k t (20%)–4 homework sets (20%)– Project (25%)–Pop-up quizzes and participation (10%)– Midterm (20%)–Final exam (25%)5ProjectProject•Picktopic BEFORE middleof thePick topic BEFORE middle of the semester•I will suggest ideas and datasets in nextI will suggest ideas and datasets in next lectures•Deliverables:Deliverables:– Project proposal–Presentation in classese tat o c ass– Poster in CS department event–Final report (around 8 pages)p( pg)6Project ExamplesProject Examples•Face detection using boostingFace detection using boosting7Project ExamplesProject Examples•Detecting bots in Quake 2Detecting bots in Quake 28Project ExamplesProject Examples•Spam filteringSpam filtering• Gender identification from emailsH d ii ii•Handwriting recognition– Also cool demo, but requires hardware• Speech recognition• Malicious website detectiona c ous ebs te detect o9PrerequisitesPrerequisites•Probability theoryProbability theory • Matlab or C/C++ programming–This could be“language of your choice”butThis could be language of your choice, but then you are responsible for debugging etc.•Some linear algebraSome linear algebra– Must not be afraid of eigenvalues• Your grade will be affected by any weaknesses in theseweaknesses in these10Pattern ClassificationA lot of material in these slides was taken fromPattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley & Sons, 2000with the permission of the authors and the publisherWhat is Pattern Recognition?What is Pattern Recognition?•InformallyInformally– Recognize patterns in dataMore formally•More formally– Assign an object or an event to one of the several prespecifiedcategories(a category isseveral pre-specified categories(a category is usually called a class)Pattern Classification, Chapter 1 12Many of these slides are borrowed from Olga Veksler (U. of Western Ontario)Example: Male or Female?Example: Male or Female?Pattern Classification, Chapter 1 13Example: Photograph or Not?Example: Photograph or Not?Pattern Classification, Chapter 1 14Character RecognitionCharacter Recognition• In this case the classes are all characters in the alphabet digits etcin the alphabet, digits etc.Pattern Classification, Chapter 1 15Speech UnderstandingSpeech Understanding• In this case the classes are all phonemesPattern Classification, Chapter 1 16Machine Learning ResearchMachine Learning Research•Speech recognitionSpeech recognition • Natural language processingCii•Computer vision• Medical outcomes analysis• Robot control•Computational biologyComputational biology• Sensor networksPattern Classification, Chapter 1 17Real-life ApplicationsReallife Applications•Loan applicationsLoan applications• Recommendation systems A N tfli–Amazon, Netflix• Targeted advertising– countless examples…18The Netflix PrizeThe Netflix Prize• Predict how much someone is going to enjoy a movie based on their movie preferencesmovie based on their movie preferences– $1M awarded in Sept. 2009•Can software recommend movies to customers?Can software recommend movies to customers?– Not Rambo to Woody Allen fans–Not Saw VI if you’ve seen all previous Saw moviesyp•Can we also have automatic customer serviceCan we also have automatic customer service for mobile providers, banks, travel agencies?19Chapter 1: Introduction to Pattern Recognition (Sections 1 1-16)Recognition (Sections 1.11.6)•Machine Perception•Machine Perception• An ExamplePtt R iti S t•Pattern Recognition Systems• The Design Cycle• Learning and Adaptation• ConclusionPattern Classification, Chapter 1 20Machine PerceptionMachine Perception• Build a machine that can recognize patterns:gp– Speech recognition– Computer Vision: object recognition, face detection– Fingerprint identificationOCR (O ti l Ch t R iti )–OCR (Optical Character Recognition)– DNA sequence identification Pattern Classification, Chapter 1 21An ExampleAn Example• “Sorting incoming Fish on a conveyorSorting incoming Fish on a conveyor according to species using optical sensing”Sea bassSpeciesSalmonPattern Classification, Chapter 1 22TrainingTraining• Set up a camera and take some sample images –Label these images by handLabel these images by hand• Extract features– Length–Lightness– Width– Number and shape of fins– Position of the mouth, etc…–Test whether this set of features is useful for a classifierPattern Classification, Chapter 1 23PreprocessingPreprocessing• Use a segmentation operation to isolate ff ffishes from one another and from the background•Informationfrom a single fish is sent to a•Information from a single fish is sent to a feature extractor whose purpose is to reduce the data by measuring certainquantitiesthe data by measuring certain quantities•The features are passed to a classifierThe features are passed to a classifier Pattern Classification, Chapter 1 24Pattern Classification, Chapter 1