Slide 1Slide 2Slide 3Slide 4Slide 5Pattern Classification via Density EstimationECE 539 Jin Woo Yim12/14/2010Motivation:In class, we discussed MAP classifier and ML classifier along with the subject of pattern classification. In practice, the underlying densities of training sets are often unknown. We discussed many approaches to deal with this situation. However, the most fundamental approach would be estimating densities of training sets. Then we could use, for example when 2 classes exist, binary hypothesis testing.How to estimate density?Histogram Density Estimator: The most intuitive way. (e.g. feat. dim. = 2, bins =16)e.g. Training data = 100, number of training samples in ith bin: (3,2) = 15. Bin 1 Bin 2 Bin 3i th bin: (3,2)Bin 16How good is this estimation?It will be tested using Mean Squared Error (Expected value of L2 norm difference between real density and estimated density). Mean Squared Error can be divided into bias and variance terms. These terms will be affected by the number of feature space dimension, the number of training samples, and the number of bins.Comparison b/w Theory and PracticeWrite a matlab code to estimate Gaussian density(and possibly more densities) with n Gaussian training samples. Observe how estimated density performs with respect to increasing number of samples and number of feature spaces. Compare the simulated results with the theoretical
View Full Document
Unlocking...