Neuro Fuzzy Systems: State-of-the-art Modeling Techniques Ajith Abraham School of Computing & Information Technology Monash University, Churchill 3842, Australia http://ajith.softcomputing.net Email: [email protected] Abstract: Fusion of Artificial Neural Networks (ANN) and Fuzzy Inference Systems (FIS) have attracted the growing interest of researchers in various scientific and engineering areas due to the growing need of adaptive intelligent systems to solve the real world problems. ANN learns from scratch by adjusting the interconnections between layers. FIS is a popular computing framework based on the concept of fuzzy set theory, fuzzy if-then rules, and fuzzy reasoning. The advantages of a combination of ANN and FIS are obvious. There are several approaches to integrate ANN and FIS and very often it depends on the application. We broadly classify the integration of ANN and FIS into three categories namely concurrent model, cooperative model and fully fused model. This paper starts with a discussion of the features of each model and generalize the advantages and deficiencies of each model. We further focus the review on the different types of fused neuro-fuzzy systems and citing the advantages and disadvantages of each model. 1. Introduction Neuro Fuzzy (NF) computing is a popular framework for solving complex problems. If we have knowledge expressed in linguistic rules, we can build a FIS, and if we have data, or can learn from a simulation (training) then we can use ANNs. For building a FIS, we have to specify the fuzzy sets, fuzzy operators and the knowledge base. Similarly for constructing an ANN for an application the user needs to specify the architecture and learning algorithm. An analysis reveals that the drawbacks pertaining to these approaches seem complementary and therefore it is natural to consider building an integrated system combining the concepts. While the learning capability is an advantage from the viewpoint of FIS, the formation of linguistic rule base will be advantage from the viewpoint of ANN. In section 2 we present cooperative NF system and concurrent NF system followed by the different fused NF models in section 3. Some discussions and conclusions are provided towards the end. 2. Cooperative and Concurrent Neuro-Fuzzy Systems In the simplest way, a cooperative model can be considered as a preprocessor wherein ANN learning mechanism determines the FIS membership functions or fuzzy rules from the training data. Once the FIS parameters are determined, ANN goes to thebackground. The rule based is usually determined by a clustering approach (self organizing maps) or fuzzy clustering algorithms. Membership functions are usually approximated by neural network from the training data. In a concurrent model, ANN assists the FIS continuously to determine the required parameters especially if the input variables of the controller cannot be measured directly. In some cases the FIS outputs might not be directly applicable to the process. In that case ANN can act as a postprocessor of FIS outputs. Figures 1 and 2 depict the cooperative and concurrent NF models. Figure 1. Cooperative NF model Figure 2. Concurrent NF model 3. Fused Neuro Fuzzy Systems In a fused NF architecture, ANN learning algorithms are used to determine the parameters of FIS. Fused NF systems share data structures and knowledge representations. A common way to apply a learning algorithm to a fuzzy system is to represent it in a special ANN like architecture. However the conventional ANN learning algorithms (gradient descent) cannot be applied directly to such a system as the functions used in the inference process are usually non differentiable. This problem can be tackled by using differentiable functions in the inference system or by not using the standard neural learning algorithm. Some of the major woks in this area are GARIC [9], FALCON [8], ANFIS [1], NEFCON [7], FUN [3], SONFIN [2], FINEST [4], EFuNN [5], dmEFuNN[5], evolutionary design of neuro fuzzy systems [10], and many others. • Fuzzy Adaptive learning Control Network (FALCON) FALCON [8] has a five-layered architecture as shown in Figure 3. There are two linguistic nodes for each output variable. One is for training data (desired output) and the other is for the actual output of FALCON. The first hidden layer is responsible for the fuzzification of each input variable. Each node can be a single node representing a simple membership function (MF) or composed of multilayer nodes that compute a complex MF. The Second hidden layer defines the preconditions of the rule followed by rule consequents in the third hidden layer. FALCON uses a hybrid-learning algorithm comprising of unsupervised learning to locate initial membership functions/ rule base and a gradient descent learning to optimally adjust the parameters of the MF to produce the desired outputs.R1R2R3x1xnR1y1y'1ymy'm Figure 3. Architecture of FALCON Figure 4. Structure of ANFIS • Adaptive Neuro Fuzzy Inference System (ANFIS) ANFIS [1] implements a Takagi Sugeno FIS and has a five layered architecture as shown in Figure 2. The first hidden layer is for fuzzification of the input variables and T-norm operators are deployed in the second hidden layer to compute the rule antecedent part. The third hidden layer normalizes the rule strengths followed by the fourth hidden layer where the consequent parameters of the rule are determined. Output layer computes the overall input as the summation of all incoming signals. ANFIS uses backpropagation learning to determine premise parameters (to learn the parameters related to membership functions) and least mean square estimation to determine the consequent parameters. A step in the learning procedure has got two parts: In the first part the input patterns are propagated, and the optimal consequent parameters are estimated by an iterative least mean square procedure, while the premise parameters are assumed to be fixed for the current cycle through the training set. In the second part the patterns are propagated again, and in this epoch, backpropagation is used to modify the premise parameters, while the consequent parameters remain fixed. This procedure is then iterated. ξ1ξ2R1R2R3R4η Figure 5. ASN of GARIC Figure 6. Architecture of NEFCON • Generalized Approximate Reasoning based Intelligent Control (GARIC) GARIC [9] implements a neuro-fuzzy controller by using two neural