# UB CSE 666 - Approximating class densities, Bayesian classifier (23 pages)

Previewing pages*1, 2, 22, 23*of 23 page document

**View the full content.**## Approximating class densities, Bayesian classifier

Previewing pages *1, 2, 22, 23*
of
actual document.

**View the full content.**View Full Document

## Approximating class densities, Bayesian classifier

0 0 49 views

- Pages:
- 23
- School:
- University at Buffalo, The State University of New York
- Course:
- Cse 666 - Biometrics & Image Analysis

**Unformatted text preview: **

http www cubs buffalo edu Pattern Recognition Approximating class densities Bayesian classifier http www cubs buffalo edu Bayesian classification Suppose we have 2 classes and we know probability density functions of their feature vectors How some new pattern should be classified Bayes classification rule classify x to the class wi which has biggest posterior probability P wi x P w1 x P w2 x w1 w2 posterior Using Bayes formula we can rewrite classification rule p x w1 P w1 p x w2 P w2 w1 w2 likelihood prior http www cubs buffalo edu Estimating probability density function Parametric pdf estimation model unknown probability density function p x wi of class wi by some parametric function pi x and determine parameters based on 1 training samples x 2 1 Example Gaussian function p x 2 l 2 e 2 Non parametric pdf estimation 1 Histogram 2 K nearest neighbor 3 Kernel methods Parzen kernels or windows 1 p x N 1 xi x h i 1 h N N is the number of training samples 4 Other methods estimating cumulative distribution function first SVM density estimation etc http www cubs buffalo edu Estimating kernel width Non parametric pdf estimation Fixed kernels 1 p x N 1 xi x h i 1 h 1 p x N 1 x xi i 1 hi hi 1 p x N 1 x xi i i 1 hi hi Adaptive kernels or N N N http www cubs buffalo edu Estimating kernel width Recall we used maximum likelihood method for parametric pdf estimation N max p X max p x1 x2 x N max p xk k 1 Can we use same method for estimating the kernel width h No the max is not achievable p x N max p xk h h k 1 N max h if k 1 1 N 1 xi xk N i 1 h h h 0 http www cubs buffalo edu Estimating kernel width Solution separate model data kernel centers from testing data cross validation technique 1 x x 1 i k max h N h h k 1 i k N p x http www cubs buffalo edu Estimating kernel width Tried maximum likelihood cross validation and still diverges 1 x x 1 i k max h N h h k 1 i k N This might happen if data is somewhat discrete p x Solution truly separate model data from testing data 1 1 xi xk

View Full Document