Optimal Brain SurgeonBackgroundBenefitsSteps to Perform OBSTests Performed and ResultsConclusionOptimal Brain SurgeonOptimal Brain SurgeonECE 539 Final ProjectECE 539 Final ProjectMark SlosarekMark SlosarekBackgroundBackgroundOptimal Brain Surgeon Algorithm (OBS) is Optimal Brain Surgeon Algorithm (OBS) is a pruning algorithma pruning algorithmReduces weights to reduce overall Reduces weights to reduce overall complexity of networkcomplexity of networkBenefitsBenefitsA pruned network has several benefitsA pruned network has several benefitsQuicker CalculationsQuicker CalculationsLess Storage Space requiredLess Storage Space requiredA pruned network should not have A pruned network should not have significantly more error than an non-significantly more error than an non-pruned networkpruned networkSteps to Perform OBSSteps to Perform OBSTrain the given MLP to minimize mean-Train the given MLP to minimize mean-square errorsquare errorCalculate the cost of the equationCalculate the cost of the equationCompute the inverse HessianCompute the inverse HessianFind the smallest SaliencyFind the smallest SaliencyIf Saliency is much smaller than mean-If Saliency is much smaller than mean-square, delete that weight and repeat for square, delete that weight and repeat for next weight, other go to next stepnext weight, other go to next stepUpdate all weightsUpdate all weightsTests Performed and ResultsTests Performed and ResultsTo test the effectiveness of the OSB, I To test the effectiveness of the OSB, I used the wine data from the homeworkused the wine data from the homeworkThe pruned network contained The pruned network contained approximately 60% fewer weightsapproximately 60% fewer weightsResults from both networks, however Results from both networks, however results of pruning were very inconclusiveresults of pruning were very inconclusiveThe results were not very consistantThe results were not very consistantConclusionConclusionA pruned network can save much space A pruned network can save much space and timeand timeMy algorithm is not perfect, and could be My algorithm is not perfect, and could be recoded to be more efficientrecoded to be more efficientResults not noticeable in simple networks, Results not noticeable in simple networks, but in real world problems, could be but in real world problems, could be results could be greatresults could be
View Full Document