Lecture 9 MLP (I): Feed-forward ModelOutlineMulti-Layer Perceptron StructureTwo Layer Perceptron XOR GateDecision Boundaries of XORMLP Nonlinear MappingMLP Feed-forward model NotationMLP Feed-forward modelApplications to ClassificationApplications to ApproximationIntro. ANN & Fuzzy SystemsLecture 9 MLP (I): Feed-forward Model(C) 2001 by Yu Hen Hu2Intro. ANN & Fuzzy SystemsOutline•Multi-Layer Perceptron Structure•Feed Forward Model•XOR Example•MLP Applications(C) 2001 by Yu Hen Hu3Intro. ANN & Fuzzy SystemsMulti-Layer Perceptron Structure A Three Layer Feed-forward Multi-Layer Perceptronwwwwww111M122M1NMNOutput LayerHidden Layer #2Input LayerHidden Layer #1(C) 2001 by Yu Hen Hu4Intro. ANN & Fuzzy SystemsTwo Layer Perceptron XOR Gate Let x1, x2 {0, 1}, theny1 = sgn(x1 – x2 – 0.5) = x1 AND y2 = sgn(x2 – x1 – 0.5) = x2 AND z = sgn(y1 + y2 – 0.5) = y1 OR y2+++zx xyy11221-1-11-0.5 -0.511-0.52x1x(C) 2001 by Yu Hen Hu5Intro. ANN & Fuzzy SystemsDecision Boundaries of XOR•Linear Hyper-planes as decision boundaries x1 – x2 – 0.5 = 0; and x2 – x1 – 0.5 = 0 -0.500.511.5-0.500.511.5zx1x2(C) 2001 by Yu Hen Hu6Intro. ANN & Fuzzy SystemsMLP Nonlinear Mapping 12.5z1xy134525515111(C) 2001 by Yu Hen Hu7Intro. ANN & Fuzzy SystemsMLP Feed-forward model Notation •(k) – Index of individual feature vectors, 1 k K. •() -- Layer index, superscript, 0 L. = 0 input layer, = L output layer •i, j – ith and jth neuron in each layer, subscript Example: •zi()(k): the output of ith neuron in the th layer corresponding to the kth feature vector. •wij(): the value of the synaptic weight that connect the output of the jth neuron at 1th layer to the jth neuron at the th layer. The value of the weight is updated once every epoch.(C) 2001 by Yu Hen Hu8Intro. ANN & Fuzzy SystemsMLP Feed-forward model •Note that , and •The input layer usually consists of linear elements. Thus, a 2-layer MLP will have two layers of non-linear neurons: the hidden layer, and the output layer. 1)()1(0kz )1()(0iiwNjjijiNjjijiiiikzwkawkukukufkz0)1()()1(1)1()()()()()()()()()])(exp[1/(1))(()((C) 2001 by Yu Hen Hu9Intro. ANN & Fuzzy SystemsApplications to Classification •Classification: Match output class to target class. MLP assigns each input feature vector to a membership of a particular class i. Multi-LayerPerceptron0 1 0 0 00 0 1 0 0TargetclassOutputclassInput feature vector(C) 2001 by Yu Hen Hu10Intro. ANN & Fuzzy SystemsApplications to Approximation •Approximation (regression, modeling) : Targets are real numbers instead of binary class membership. Multi-LayerPerceptron.1 .1 .1 .2 .3.3 .4 .1 .9 .99TargetOutputInput
View Full Document