Chapter IV - Designing and Training MLPs2. Controlling Learning in Practice3. Other Search Procedures4. Stop Criteria5. How good are MLPs as learning machines?6. Error Criterion7. Network Size and Generalization8. Project: Application of the MLP to real world data9. Conclusionalgorithm locality and distributed systems system identification versus modelinggood initial weight valuesMinskowski measurescross entropy criterionearly stopping and model complexitylearning rate annealingshallow networksEq.6outliersEq.8activationdualfan-inSimon Haykinnonconvexconfusion matrixgeneralizationVladimir VapnikBarronsaliencyHessiancommitteessimulated annealingfirst ordervalidationclassification errorrobustOccamVC dimensionGenetic AlgorithmsLuenbergerScott FahlmanCampbellR. A. Fisherline search methodsBishopEq. 24FletcherHorst, Pardalos and ThoaiShepherdPearlmutterHertz, Krogh, PalmerLeCun, Denker and SollaPerroneCoverLeCun, Simard, PearlmutterSilva e AlmeidaAlmeida’s adaptive stepsizeTable of Contents CHAPTER IV - DESIGNING AND TRAINING MLPS ...................................................................................3 2. CONTROLLING LEARNING IN PRACTICE.............................................................................................4 3. OTHER SEARCH PROCEDURES ......................................................................................................15 4. STOP CRITERIA.............................................................................................................................29 5. HOW GOOD ARE MLPS AS LEARNING MACHINES? ...........................................................................33 6. ERROR CRITERION........................................................................................................................38 7. NETWORK SIZE AND GENERALIZATION ...........................................................................................45 8. PROJECT: APPLICATION OF THE MLP TO REAL WORLD DATA............................................................51 9. CONCLUSION ................................................................................................................................58 ALGORITHM LOCALITY AND DISTRIBUTED SYSTEMS ..............................................................................61 SYSTEM IDENTIFICATION VERSUS MODELING .......................................................................................62 GOOD INITIAL WEIGHT VALUES............................................................................................................62 MINSKOWSKI MEASURES ...................................................................................................................63 CROSS ENTROPY CRITERION ..............................................................................................................63 EARLY STOPPING AND MODEL COMPLEXITY .........................................................................................64 LEARNING RATE ANNEALING ...............................................................................................................65 SHALLOW NETWORKS ........................................................................................................................65 EQ.6 ................................................................................................................................................65 OUTLIERS .........................................................................................................................................65 EQ.8 ................................................................................................................................................65 ACTIVATION.......................................................................................................................................65 DUAL ................................................................................................................................................66 FAN-IN ..............................................................................................................................................66 SIMON HAYKIN..................................................................................................................................66 NONCONVEX .....................................................................................................................................66 CONFUSION MATRIX...........................................................................................................................66 GENERALIZATION...............................................................................................................................66 VLADIMIR VAPNIK..............................................................................................................................67 BARRON ...........................................................................................................................................67 SALIENCY..........................................................................................................................................67 HESSIAN...........................................................................................................................................67 COMMITTEES.....................................................................................................................................67 SIMULATED ANNEALING......................................................................................................................67 FIRST ORDER ....................................................................................................................................68 VALIDATION.......................................................................................................................................68 CLASSIFICATION ERROR.....................................................................................................................68 ROBUST............................................................................................................................................68 OCCAM.............................................................................................................................................68 VC DIMENSION..................................................................................................................................69 GENETIC
View Full Document