eFSM - A novel online neural-fuzzy semantic memory model

IEEE Transactions on Neural Networks

Volumn 21, Issue 1, 2010, Pages 136-157

  Tung, Whye Loon ^a   Quek, Chai ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Evolving fuzzy system; Incremental sequential learning; Neural fuzzy semantic memory; Neural fuzzy system (NFS)

Indexed keywords

ADAPTIVE STRUCTURE; ASSOCIATIVE MAPPING; BENCHMARK APPLICATIONS; COMPACT SETS; DATA-DRIVEN; DYNAMIC NATURE; EFSM MODEL; EVOLVING FUZZY SYSTEM; FUZZY RULE-BASED SYSTEMS; FUZZY SEMANTICS; GENETIC FUZZY SYSTEMS; INCREMENTAL LEARNING; INCREMENTAL SEQUENTIAL LEARNING; INTENSIVE RESEARCH; MAMDANI; MODELING PERFORMANCE; NONSTATIONARY; OBSERVED DATA; OPERATING ENVIRONMENT; PARAMETER LEARNING; REAL-WORLD PROBLEM; RESEARCH EFFORTS; RULE BASE; RULE BASIS; SEMANTIC KNOWLEDGE; SEQUENTIAL LEARNING; TIME VARYING; TRAINING DATA;

BENCHMARKING; FUZZY CONTROL; FUZZY RULES; FUZZY SETS; FUZZY SYSTEMS; KNOWLEDGE ACQUISITION; REAL TIME SYSTEMS; SEMANTICS; TUNING;

FUZZY LOGIC;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; FUZZY LOGIC; HUMAN; LEARNING; MEMORY; NONLINEAR SYSTEM; ONLINE SYSTEM; PHYSIOLOGY; PSYCHOLOGICAL MODEL; SEMANTICS;

ALGORITHMS; COMPUTER SIMULATION; FUZZY LOGIC; HUMANS; MEMORY; MODELS, PSYCHOLOGICAL; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; ONLINE SYSTEMS; SEMANTICS; VERBAL LEARNING;

EID: 73949115619     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2009.2035116     Document Type: Article

References (75)

1. C. C. Lee, "Fuzzy logic in control systems: Fuzzy logic controller- Part II," IEEE Trans. Syst. Man Cybern., vol. 20, no. 2, pp. 419-435, Mar./Apr. 1990.
2. J. Casillas, O. Cordon, F. Herrera, and L. Magdalena, Eds., Interpretability Issues in Fuzzy Modeling. Berlin, Germany: Springer-Verlag, 2003.
3. J. J. Buckley and Y. Hayashi, "Fuzzy neural networks:Asurvey," Fuzzy Sets Syst., vol. 66, no. 1, pp. 1-13, 1994.
4. J. J. Buckley and Y. Hayashi, "Neural nets for fuzzy systems," Fuzzy Sets Syst., vol. 71, no. 3, pp. 265-276, 1995.
5. C. T. Lin and C. S. G. Lee, Neural Fuzzy Systems-A Neuro-Fuzzy Synergism to Intelligent Systems. Englewood Cliffs, NJ: Prentice-Hall, 1996.
6. J. S. R. Jang, "ANFIS: Adaptive-network-based fuzzy inference system," IEEE Trans. Syst. Man Cybern., vol. 23, no. 3, pp. 665-685, May/Jun. 1993.
7. C. Quek and R. W. Zhou, "POPFNN: A pseudo outer-product based fuzzy neural network," Neural Netw., vol. 9, no. 9, pp. 1569-1581, 1996.
8. C. Quek and R. Zhou, "POPFNN-AARS(S): A pseudo outer-product based fuzzy neural network," IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 29, no. 6, pp. 859-870, Dec. 1999.
9. K. K. Ang, C. Quek, and M. Pasquier, "POPFNN-CRI(S): Pseudo outer product based fuzzy neural network using the compositional rule of inference and singleton fuzzifier," IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 33, no. 6, pp. 838-849, Dec. 2003.
10. C. Quek and R. Zhou, "Structure and learning algorithms of a nonsingleton input fuzzy neural network based on the approximate analogical reasoning schema," Fuzzy Sets Syst., vol. 157, no. 13, pp. 1814-1831, 2006.
11. A. Singh, C. Quek, and S. Y. Cho, "DCT-Yager FNN: A novel Yagerbased fuzzy neural network with the discrete clustering technique," IEEE Trans. Neural Netw., vol. 19, no. 4, pp. 625-644, Apr. 2008.
12. C. J. Lin and C. T. Lin, "An ART-based fuzzy adaptive learning control network," IEEE Trans. Fuzzy Syst., vol. 5, no. 4, pp. 477-496, Aug. 1997.
13. C. Quek and W. L. Tung, "A novel approach to the derivation of fuzzy membership functions using the Falcon-MART architecture," Pattern Recognit. Lett., vol. 22, no. 9, pp. 941-958, 2001.
14. W. L. Tung and C. Quek, "Falcon: Neural fuzzy control and decision systems using FKP and PFKP clustering algorithms," IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 34, no. 1, pp. 686-695, Feb. 2004.
15. J. Kim and N. Kasabov, "HyFIS: Adaptive neuro-fuzzy inference systems and their application to nonlinear dynamic systems," Neural Netw., vol. 12, no. 9, pp. 1301-1319, 1999.
16. M. J. Watts, "A decade of Kasabov's evolving connectionist systems: A review," IEEE Trans. Syst. Man Cybern. C, Appl. Rev., vol. 39, no. 3, pp. 253-269, May 2009.
17. N. Kasabov, "Evolving fuzzy neural networks for supervised/ unsupervised online knowledge-based learning," IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 31, no. 6, pp. 902-918, Dec. 2001. (Pubitemid 34035554)
18. N. Kasabov and Q. Song, "DENFIS: Dynamic evolving neural-fuzzy inference system and its application for time-series prediction," IEEE Trans. Fuzzy Syst., vol. 10, no. 2, pp. 144-154, Apr. 2002.
19. W. L. Tung and C. Quek, "GenSoFNN: A generic self-organizing fuzzy neural network," IEEE Trans. Neural Netw., vol. 13, no. 5, pp. 1075-1086, Sep. 2002.
20. W. L. Tung, C. Quek, and P. Y. K. Cheng, "GenSo-EWS: A novel neural-fuzzy based early warning system for predicting bank failures," Neural Netw., vol. 17, no. 4, pp. 567-587, 2004.
21. W. L. Tung and C. Quek, "GenSo-FDSS: A neural-fuzzy decision support system for pediatric ALL cancer subtype identification using gene expression data," Artif. Intell. Med., vol. 33, no. 1, pp. 61-88, 2005.
22. W. L. Tung and C. Quek, "A brain-inspired fuzzy semantic memory model for learning and reasoning with uncertainty," Neural Comput. Appl., vol. 16, no. 6, pp. 559-569, 2007.
23. R. J. Oentaryo, M. Pasquier, and C. Quek, "GenSoFNN-Yager: A novel brain-inspired generic self-organizing neuro-fuzzy system realizing Yager inference," Expert Syst. Appl., vol. 35, no. 4, pp. 1825-1840, 2008.
24. J. Sim, W. L. Tung, and C. Quek, "FCMAC-Yager: A novel Yager inference scheme based fuzzy CMAC," IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1394-1410, Dec. 2006.
25. S. D. Teddy, E.M. K. Lai, and C. Quek, "Hierarchically clustered adaptive quantization CMAC and its learning convergence," IEEE Trans. Neural Netw., vol. 18, no. 6, pp. 1658-1682, Dec. 2007.
26. S. D. Teddy, C. Quek, and E. M. K. Lai, "PSECMAC: A novel self-organizing multi-resolution associative memory architecture," IEEE Trans. Neural Netw., vol. 19, no. 4, pp. 689-712, Apr. 2008.
27. S. D. Teddy, E. M. K. Lai, and C. Quek, "A cerebellar associative memory approach to option pricing and arbitrage trading," Neurocomputing, vol. 71, no. 16-18, pp. 3303-3315, 2008.
28. C. W. Ting and C. Quek, "A novel blood glucose regulation using TSK -FCMAC: A fuzzy CMAC based on the zero-ordered TSK fuzzy inference scheme," IEEE Trans. Neural Netw., vol. 20, no. 5, pp. 856-871, May 2009.
29. S. Mitra and Y. Hayashi, "Neuro-fuzzy rule generation: Survey in soft computing framework," IEEE Trans. Neural Netw., vol. 11, no. 3, pp. 748-768, May 2000.
30. ε fuzzy rule systems from data using evolution strategies," IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 29, no. 6, pp. 829-845, Dec. 1999.
31. O. Cordon, F. Gomide, F. Herrera, F. Hoffmann, and L. Magdalena, "Ten years of genetic fuzzy systems: Current framework and new trends," Fuzzy Sets Syst., vol. 141, no. 1, pp. 5-31, 2004.
32. F. Herrera, "Genetic fuzzy systems: Status, critical considerations and future directions," Int. J. Comput. Intell. Res., vol. 1, no. 1, pp. 59-67, 2005.
33. O. Cordon and F. Herrera, "A two-stage evolutionary process for designing TSK fuzzy rule-based systems," IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 29, no. 6, pp. 703-715, Dec. 1999.
34. O. Cordon, F. Herrera, F. Hoffmann, and L. Magdalena, Genetic Fuzzy Systems: Evolutionary Tuning and Learning of Fuzzy Knowledge Bases, ser. Advances in Fuzzy Systems-Applications & Theory, K. Hirota, G. J. Klir, E. Sanchez, P. Z. Wang, and R. R. Yager, Eds. Singapore: World Scientific, 2001, vol. 19.
35. H. Wang, S. Kwong, Y. Jin, W. Wei, and K. F. Man, "Multi-objective hierarchical genetic algorithm for interpretable fuzzy rule-based knowledge extraction," Fuzzy Sets Syst., vol. 149, no. 1, pp. 149-186, 2005.
36. C. Pena-Reyes and M. Sipper, "Fuzzy CoCo: A cooperative- coevolutionary approach to fuzzy modeling," IEEE Trans. Fuzzy Syst., vol. 9, no. 5, pp. 727-737, Oct. 2001.
37. G. Leng, T. M. McGinnity, and G. Prasad, "Design for self-organizing fuzzy neural networks based on genetic algorithms," IEEE Trans. Fuzzy Syst., vol. 14, no. 6, pp. 755-766, Dec. 2006.
38. J. Casillas, B. Carse, and L. Bull, "Fuzzy-XCS: A Michigan genetic fuzzy system," IEEE Trans. Fuzzy Syst., vol. 15, no. 4, pp. 536-550, Aug. 2007.
39. C. J. Lin, C. H. Chen, and C. T. Lin, "Efficient self-evolving evolutionary learning for neurofuzzy inference systems," IEEE Trans. Fuzzy Syst., vol. 16, no. 6, pp. 1476-1490, Dec. 2008.
40. E. G. Mansoori, M. J. Zolghadri, and S. D. Katebi, "SGERD: A steadystate genetic algorithm for extracting fuzzy classification rules from data," IEEE Trans. Fuzzy Syst., vol. 16, no. 4, pp. 1061-1071, Aug. 2008.
41. H. M. Huang, M. Pasquier, and C. Quek, "Financial market trading system with a hierarchical coevolutionary fuzzy predictive model," IEEE Trans. Evol. Comput., vol. 13, no. 1, pp. 56-70, Feb. 2009.
42. R. M. French, "Catastrophic forgetting in connectionist networks," in Encyclopedia of Cognitive Science, L. Nadel, Ed. London, U.K.: Nature Publishing Group, 2003, vol. 1, pp. 431-435.
43. S. Grossberg, Studies of Mind and Brain: Neural Principles of Learning, Perception, Development, Cognition, and Motor Control. Boston, MA: Reidel, 1982.
44. P. P. Angelov and D. P. Filev, "An approach to online identification of Takagi-Sugeno fuzzy models," IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 34, no. 1, pp. 484-498, Feb. 2004.
45. C. F. Juang and C. T. Lin, "An online self-constructing neural fuzzy inference network and its applications," IEEE Trans. Fuzzy Syst., vol. 6, no. 1, pp. 12-32, Feb. 1998.
46. H. J. Rong, N. Sundararajan, G. B. Huang, and P. Saratchandran, "Sequential adaptive fuzzy inference system (SAFIS) for nonlinear system identification and prediction," Fuzzy Sets Syst., vol. 157, no. 9, pp. 1260-1275, 2006.
47. G. Panoutsos and M. Mahfouf, "An incremental learning structure using granular computing and model fusion with application to materials processing," in Intelligent Techniques and Tools for Novel System Architectures, ser. Studies in Computational Intelligence (SCI). Berlin, Germany: Springer-Verlag, 2008, vol. 109, pp. 139-153.
48. J. C. de Barros and A. L. Dexter, "On-line identification of computationally undemanding evolving fuzzy models," Fuzzy Sets Syst., vol. 158, no. 18, pp. 1997-2012, 2007.
49. E. D. Lughofer, "FLEXFIS: A robust incremental learning approach for evolving Takagi-Sugeno fuzzy models," IEEE Trans. Fuzzy Syst., vol. 16, no. 6, pp. 1393-1410, Dec. 2008.
50. C. F. Juang and Y. W. Tsao, "A self-evolving interval type-2 fuzzy neural network with online structure and parameter learning," IEEE Trans. Fuzzy Syst., vol. 16, no. 6, pp. 1411-1424, Dec. 2008.
51. P. P. Angelov and X. W. Zhou, "Evolving fuzzy-rule-based classifiers from data streams," IEEE Trans. Fuzzy Syst., vol. 16, no. 6, pp. 1462-1475, Dec. 2008.
52. S. Mohagheghi, G. K. Venayagamoorthy, and R. G. Harley, "Fully evolvable optimal neurofuzzy controller using adaptive critic designs," IEEE Trans. Fuzzy Syst., vol. 16, no. 6, pp. 1450-1461, Dec. 2008.
53. W.Wang and J. J. Vrbanek, "An evolving fuzzy predictor for industrial applications," IEEE Trans. Fuzzy Syst., vol. 16, no. 6, pp. 1439-1449, Dec. 2008.
54. D. Nauck, F. Klawonn, and R. Kruse, Foundations of Neuro-Fuzzy Systems. New York: Wiley, 1997.
55. J. L. Castro, "Fuzzy logic controllers are universal approximators," IEEE Trans. Syst. Man Cybern., vol. 25, no. 4, pp. 629-635, Apr. 1995.
56. E. Mamdani, "Application of fuzzy logic to approximate reasoning using linguistic systems," IEEE Trans. Comput., vol. C-26, no. 12, pp. 1182-1191, Dec. 1977.
57. J. R. Whitlock, A. J. Heynen, M. G. Shuler, and M. F. Bear, "Learning induces long-term potentiation in the hippocampus," Science, vol. 313, no. 5790, pp. 1093-1097, 2006.
58. M. F. Bear and W. C. Abraham, "Long-term depression in hippocampus," Annu. Rev. Neurosci., vol. 19, pp. 437-462, 1996.
59. W. L. Tung and C. Quek, "Analysis of the gradient descent approach to parameter learning in eFSM," Centre Comput. Intell., School Comput. Eng., Nanyang Technol. Univ., Nanyang, Singapore, Tech. Rep. C2i-TR-09-001, 2009 [Online]. Available: http://www.c2i.ntu.edu.sg/Resources/TR/C2i-TR-09-001.pdf
60. S. Guillaume, "Designing fuzzy inference systems from data: An interpretability- oriented review," IEEE Trans. Fuzzy Syst., vol. 9, no. 3, pp. 426-443, Jun. 2001.
61. J. Yen, L. Wang, and C. Gillespie, "Improving the interpretability of TSK fuzzy models by combining global learning and local learning," IEEE Trans. Fuzzy Syst., vol. 6, no. 4, pp. 530-537, Aug. 1998.
62. W. L. Tung and C. Quek, "A localized approach to parameter-learning in Mamdani-type neural-fuzzy systems," 2009, to be published.
63. G. K. Tan, "Feasibility of predicting congestion states with neural networks," School Civil Structural Eng., Nanyang Technol. Univ., Singapore, Final Year Project, 1997.
64. R. Goldman and J.Weinberg, Statistics: An Introduction. Englewood Cliffs, NJ: Prentice-Hall, 1985.
65. F. Liu, C. Quek, and G. S. Ng, "A novel generic Hebbian ordering based fuzzy rule base reduction approach to Mamdani neuro-fuzzy system," Neural Comput., vol. 19, no. 6, pp. 1656-1680, 2007.
66. K. K. Ang and C. Quek, "RSPOP: Rough set-based pseudo outerproduct fuzzy rule identification algorithm," Neural Comput., vol. 17, no. 1, pp. 205-243, 2005.
67. Z. Pawlak, Rough Sets: Theoretical Aspects of Reasoning About Data. Boston, MA: Kluwer, 1991.
68. Q. Shen and A. Chouchoulas, "Combining rough sets and data-drivenfuzzy learning for generation of classification rules," Pattern Recognit., vol. 32, no. 12, pp. 2073-2076, 1999.
69. T. Takagi and M. Sugeno, "Fuzzy identification of systems and its applications to modeling and control," IEEE Trans. Syst. Man Cybern., vol. SMC-15, no. 1, pp. 116-132, Feb. 1985.
70. J. Casillas, O. Cordon, F. Herrera, and L. Magdalena, "Interpretability improvements to find the balance interpretability- accuracy in fuzzy modeling: An overview," in Interpretability Issues in Fuzzy Modeling, J. Casillas, O. Cordon, F. Herrera, and L. Magdalena, Eds. NewYork: Springer-Verlag, 2003, pp. 3-22.
71. E. Mamdani and S. Assilian, "An experiment in linguistic synthesis with a fuzzy logic controller," Int. J. Man-Mach. Studies, vol. 7, pp. 1-13, 1975.
72. Y. W. Lu, N. Sundararajan, and P. Saratchandran, "A sequential learning scheme for function approximation using minimal radial basis function (RBF) neural networks," Neural Comput., vol. 9, no. 3, pp. 461-478, 1997.
73. V. Kadirkamanathan and M. Niranjan, "A function estimation approach to sequential learning with neural networks," Neural Comput., vol. 5, no. 6, pp. 954-975, 1993.
74. J. Platt, "A resource allocating network for function interpolation," Neural Comput., vol. 3, no. 2, pp. 213-225, 1991.
75. W. L. Tung and C. Quek, "Online identification of a nonlinear dynamic system with time-varying characteristics using eFSM," Centre Comput. Intell., School Comput. Eng., Nanyang Technol. Univ., Nanyang, Singapore, Tech. Rep. C2i-TR-09-002, 2009 [Online]. Available: http://www.c2i.ntu.edu.sg/Resources/TR/C2i-TR-09-002.pdf