Slide 1Slide 2Slide 3Slide 4Slide 5Optimal Brain Surgeon AlgorithmPayne Y. ChangWhat is OBS(Optical Brain Surgeon) ?What is OBS(Optical Brain Surgeon) ?• To solve real-world problems with ANN :• Usually Need• Highly connected• Larger• Not alwayse good • OBS algorithm• An approach to prune the ANN• Goal : 1. Set some synaptic weights to zero 2. Modify the remaining weights 3. Maintain good performanceHow does OBS work ?1. Train the MLP2. Calculate the saliency ( Si ) of each synaptic weight ( wi ) SSii : The increase in the cost function from the deletion of w : The increase in the cost function from the deletion of wii3. If Si is smaller than a threshold, delete wi4. Go to step 25. Stop when no more weights can be deletedProgram : OBS.exe1. C++2. FLTKInputInputOutputOutputHidden NHidden NBias (1)Bias (1)Deleted wDeleted wResults and SummaryBefore OBS After OBSData file Structure Mean S.D. # of weights Mean S.D. # of weightsData1 ( HW#2 ) 2-10-2 79.06 0.98002 52 78.14 1.7715 19Data2 2-10-2 93.95555 1.68688 52 91.81111 2.09451 22Data3 2-10-2 92 4.13656 52 90.2 6.5963 24Data3 2-10-10-2 94.3 4.24395 162 94.4 3.56526 57• Learning Rate = 0.01• Momentum = 0.8• Activation function : Hyperbolic tangen1. Efficient ( Time & Storage Saving )1. Efficient ( Time & Storage Saving )2. Better Generalization2. Better
View Full Document
Unlocking...