Derek Hoiem Feb 3 2005 10701 Recitation Backpropagation Example Cross validation Overfitting Measuring error Intrinsic error Bias and Variance components Where does error come from Last homework Next homework Class material Questions Topics Questions Loss Confidence 0 1 Error vs Squared Error n 1 2 y x w t n n Tricky Stuff e y x w t 2 p t x dtdx 1 N Modeling Error Intrinsic Error e y x w E t x 2 p x dx E t 2 x E t x 2 p x dx Infinite Data Squared Error e N Squared Error Example Intrinsic Error and Modeling Error Stephen Covey Spectacled Friend 1 Modeling error huh Sounds suspicious What is in your circle of concern Spectacled Friend 2 There s nothing I can do e y x w E t x 2 p x dx E t 2 x E t x 2 p x dx Modeling Error Bias2 Variance Bias2 Variance ED em ED y x w E t x 2 p x dx E D y x w E D y x w 2 p x dx Tricky Stuff e y x w E t x 2 p x dx E t 2 x E t x 2 p x dx Modeling Error Bias2 Variance How can we tell if its too complex or not complex enough 2 Cross validation or validation set reduce variance 1 Sufficiently complex classifier for problem low bias Practical Solution to the BiasVariance Problem K fold Leave One Out How many folds Cross Validation Cross Validation Example x Num inputs Num hidden layers Hidden nodes per layer Num outputs 1 Activation functions Defined by h22 h21 h11 h12 1 1 Neural Networks y w1h12 x w0h12 1 1 h12 w1h11 h11 w0h11 h22 h21 1 i y 1 x w 1 exp w0 wi x i Neural Networks Sigmoid Function w0 shift i kw slope 1 x w 1 exp w0 wi x i Neural Networks Sigmoid Function i 1 x w 1 exp w0 wi x i Neural Networks Sigmoid Function x 1 h22 h21 h11 h12 1 1 y Neural Networks Backpropagation x 1 1 1 x 0 5 y 0 h 1 Training Input x 1 y 1 2 1 y Neural Networks Simple Example
View Full Document
Unlocking...