Implementation of Nonlinear Conjugate Gradient Method for MLPIntroductionThe AlgorithmThe Algorithm (Continued)Software ImplementationImplementation of Nonlinear Conjugate Gradient Method for MLPMatt PetersonECE 539December 10, 2001IntroductionSteepest Descent Gradient training methodCan oscillateCan get caught at local minimumsNonlinear Conjugate Gradient Method“Optimization” approachConverges quickerThe AlgorithmInitializationSelect initial weight vector w(0)Use BP to compute gradient vector g(0)Set s(0) = r(0) = -g(0)Use line search to find η(n) that minimizes errorTest for convergence ( ||r(n)|| < ε||r(0)|| )Update Weights ( w(n+1) = w(n) + η(n)s(n) )The Algorithm (Continued)Compute new gradient vector g(n+1)Set r(n+1) = -g(n+1)Calculate β(n+1) using Polak-Ribiére methodβ(n+1)=max( (rT(n+1)(r(n+1)-r(n))/rT(n)r(n) , 0 )Update direction vector s(n+1) = r(n+1) + β(n+1)s(n)Software ImplementationWritten in Matlab codeSimilar structure and user interface as
View Full Document
Unlocking...