Slide 1Examples of PatternsSlide 3Slide 4Slide 5Slide 6Slide 7Slide 8Slide 9Slide 10Slide 11ApplicationsTwo Schools of ThinkingLevels of taskSchools and streamsAn example of Pattern RecognitionFeatures and DistributionsDecision/classification BoundariesMain Issues in Pattern RecognitionWhat is a pattern?What is a patternSlide 22Lecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningLecture 1:Introduction to Pattern Recognition 1. Examples of patterns in nature. 2. Issues in pattern recognition and an example of pattern recognition 3. Schools in pattern recognition 4. Pattern theoryLecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningExamples of PatternsCrystal patterns at atomic and molecular levelsTheir structures are represented by 3D graphs and can be described by deterministic grammar or formal languageLecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningExamples of PatternsConstellation patterns in the sky.The constellation patterns are represented by 2D (often planar) graphsHuman perception has strong tendency to find patterns from anything. We see patterns from even random noise --- we are more likely to believe a hidden pattern than denying it when the risk (reward) for missing (discovering) a pattern is often high.Lecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningExamples of PatternsBiology pattern ---morphologyLandmarks are identified from biologic forms and these patterns are then represented by a list of points. But for other forms, like the root of plants,Points cannot be registered crossing instances.Applications: biometrics, computational anatomy, brain mapping, …Lecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningExamples of PatternsPattern discovery and associationStatistics show connections between the shape of one’s face (adults) and his/her Character. There is also evidence that the outline of children’s face is related to alcohol abuse during pregnancy.Lecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningExamples of PatternsWe may understand patterns of brain activity and find relationships between brain activities, cognition, and behaviorsPatterns of brain activities:Lecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningExamples of PatternsPatterns with variations: 1. Expression –geometric deformation 2. lighting --- photometric deformation 3. 3D pose transform 4. Noise and occlusionLecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningExamples of PatternsA wide variety of texture patterns are generated by various stochastic processes. How are these patterns represented in human brain?Lecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningExamples of PatternsSpeech signal and Hidden Markov modelLecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningExamples of PatternsNatural language and stochastic grammar.Lecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningExamples of PatternsLecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningApplicationsLie detector,Handwritten digit/letter recognitionBiometrics: voice, iris, finger print, face, and gait recognitionSpeech recognitionSmell recognition (e-nose, sensor networks)Defect detection in chip manufacturingReading DNA sequencesFruit/vegetable recognitionMedical diagnosisNetwork traffic modeling, intrusion detection… …Lecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningTwo Schools of Thinking1. Generative methods: Bayesian school, pattern theory. 1). Define patterns and regularities (graph spaces), 2). Specify likelihood model for how signals are generated from hidden structures 3). Learning probability models from ensembles of signals 4). Inferences.2. Discriminative methods: The goal is to tell apart a number of patterns, say 100 people in a company, 10 digits for zip-code reading. These methods hit the discriminative target directly, without having to understand the patterns (their structures) or to develop a full mathematical description. For example, we may tell someone is speaking English or Chinese in the hallway without understanding the words he is speaking. “You should not solve a problem to an extent more than what you need”Lecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningLevels of taskFor example, there are many levels of tasks related to human face patterns 1. Face authentication (hypothesis test for one class) 2. Face detection (yes/no for many instances). 3. Face recognition (classification) 4. Expression recognition (smile, disgust, surprise, angry) identifiability problem. 5. Gender and age recognition-------------------------------------------------------------- 6. Face sketch and from images to cartoon --- needs generative models. 7. Face caricature … … The simple tasks 1-4 may be solved effectively using discriminative methods, but the difficult tasks 5-7 will need generative methods.Lecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningSchools and streamsSchools for pattern recognition can be divided in three axes: Axis I: generative vs discriminative (Bayesian vs non-Bayesian) (--- modeling the patterns or just want to tell them apart) Axis II: deterministic vs stochastic (logic vs statistics) (have rigid regularity and hard thresholds or have soft constraints on regularity and soft thresholding) Axis III: representation---algorithm---implementationExamples: Bayesian decision theory, neural networks, syntactical pattern recognition (AI), decision trees, Support vector machines, boosting techniques,Lecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningAn example of Pattern Recognition Classification of fish into two classes: salmon and Sea Bass by discriminative methodLecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningFeatures and DistributionsLecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningDecision/classification BoundariesLecture note for Stat 231-CS276A: Pattern Recognition and Machine LearningMain Issues in Pattern Recognition1. Feature selection and extraction --- What are good discriminative features?2. Modeling and learning 3. Dimension reduction, model complexity4. Decisions and risks5. Error