A survey on universal approximation and its limits in soft computing techniques

International Journal of Approximate Reasoning

Volumn 33, Issue 2, 2003, Pages 185-202

  Tikk, Domonkos ^a   Kóczy, László T ^a   Gedeon, Tamás D ^b  

^a BUDAPEST UNIVERSITY OF TECHNOLOGY AND ECONOMICS   (Hungary)

^b MURDOCH UNIVERSITY   (Australia)

Author keywords

Approximation behaviour of soft computing techniques; Approximation rates; Constructive proofs; Course of dimensionality; Kolmogorov's theorem; Nowhere denseness; Universal approximation performed by fuzzy systems and neural networks

Indexed keywords

APPROXIMATION THEORY; COMPUTATIONAL METHODS; NEURAL NETWORKS; THEOREM PROVING;

SOFT COMPUTING TECHNIQUES;

FUZZY CONTROL;

EID: 0037544056     PISSN: 0888613X     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0888-613X(03)00021-5     Document Type: Article

References (55)

1. Arnold V.I. On functions of three variables. Dokl. Akad. Nauk USSR. 114:1957;679-681.
2. Barron A.R. Universal approximation bounds for superpositions of sigmoidal functions. IEEE Trans. Inform. Theory. 39(3):1993;930-945.
3. Bauer P., Klement E.P., Leikermoser A., Moser B. Modeling of control functions by fuzzy controllers. Nguyen H., Sugeno M., Tong R., Yager R.R. Theoretical Aspects of Fuzzy Control. 1995;91-116 Wiley, New York.
4. Blum E.K., Li L.K. Approximation theory and feedforward networks. Neural Networks. 4(4):1991;511-515.
5. Buckley J.J. Sugeno type controllers are universal controllers. Fuzzy Sets Syst. 53:1993;299-304.
6. Buckley J.J. System stability and the fuzzy controller. Nguyen H., Sugeno M., Tong R., Yager R.R. Theoretical Aspects of Fuzzy Control. 1995;51-63 Wiley, New York.
7. Castro J.L. Fuzzy logic controllers are universal approximators. IEEE Trans. SMC. 25:1995;629-635.
8. Castro J.L., Mantas C.J., Benítez J.M. Neural networks with a continuous squashing function in the output are universal approximators. Neural Networks. 13:2000;561-563.
9. Cybenko G. Approximation by superposition of sigmoidal functions. Math. Control Signals Syst. 2:1989;303-314.
10. Ding Y.-S., Ying H., Shao S.-H. Necessary conditions on minimal system configuration for general MISO mamdani fuzzy systems as universal approximators. IEEE Trans. Syst. Man Cybernet. Part B. 30(6):2000;857-864.
11. De Figueiredo R.J.P. Implications and applications of Kolmogorov's superposition theorem. IEEE Trans. Automat. Control. 1980;1227-1230.
12. Funahashi K.-I. On the approximate realization of continuous mappings by neural networks. Neural Networks. 2:1989;183-192.
13. Haykin S. Neural Networks: A Comprehensive Foundation. second ed. 1999;Prentice Hall, Upper Saddle River, NJ.
14. R. Hecht-Nielsen, Kolmogorov's mapping neural network existence theorem, in: Proc. of the International Conference on Neural Networks, New York, vol. III, 1987, pp. 11-14.
15. Hellendoorn H., Driankov D., Reinfrank M. An Introduction to Fuzzy Control. 1993;Springer, Berlin.
16. Hornik K. Approximation capabilities of multilayer feedforward networks. Neural Networks. 4:1991;251-257.
17. Hornik K., Stinchcombe M., White H. Multilayer feedforward networks are universal approximators. Neural Networks. 2:1989;359-366.
18. Jang J.-S.R., Sun C.-T., Mizutani E. Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence. 1997;Prentice Hall, Upper Saddle River, NJ.
19. Jones L.K. A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training. Ann. Stat. 20:1992;608-613.
20. Joo M.G., Lee J.S. Universal approximation by hierarchical fuzzy system with constraints on the fuzzy rule. Fuzzy Sets Syst. 130:2002;175-188.
21. Kainen P.C., Kůrková V., Vogt A. Approximation by neural networks is not continuous. Neurocomputing. 29:1999;47-56.
22. S. Kawamoto, K. Tada, N. Onoe, A. Ishigame, T. Taniguchi, Construction of exact fuzzy system for nonlinear system and its stability analysis, in: Proc. of the 8th Fuzzy System Symposium, Hiroshima, 1992, pp. 517-520 (in Japanese).
23. Klement E.P., Kóczy L.T., Moser B. Are fuzzy systems universal approximators? Int. J. Gen. Syst. 28(2-3):1999;259-282.
24. Kóczy L.T., Zorat A. Fuzzy systems and approximation. Fuzzy Sets Syst. 85:1997;203-222.
25. L.T. Kóczy, A. Zorat, T.D. Gedeon, The cat and mouse problem - optimizing the size of fuzzy rule bases, in: Proc. of the Conf. on Current Issues of Fuzzy Technologies'95, Trento, Italy, 1995, pp. 139-151.
26. Kolmogorov A.N. On the representation of continuous functions of many variables by superpositions of continuous functions of one variable and addition. Dokl. Akad. SSSR. 114:1957;953-956. (in Russian).
27. B. Kosko, Fuzzy systems as universal approximators, in: Proc. of the IEEE Int. Conf. on Fuzzy Systems, San Diego, 1992, pp. 1153-1162.
28. Kůrková V. Kolmogorov's theorem and multilayer neural networks. Neural Networks. 5:1992;501-506.
29. Kůrková V. Approximation of functions by perceptron networks with bounded number of hidden units. Neural Networks. 8(5):1995;745-750.
30. Li Y.-M., Shi Z.-K., Li Z.-H. Approximation theory of fuzzy systems based upon genuine many-valued implications - SISO cases. Fuzzy Sets Syst. 130:2002;147-157.
31. Li Y.-M., Shi Z.-K., Li Z.-H. Approximation theory of yyfuzzy systems based upon genuine many-valued implications - MIMO cases. Fuzzy Sets Syst. 130:2002;159-174.
32. Lorentz G.G. Approximation of Functions. 1966;Holt, Reinhard and Winston, New York.
33. D.F. McCaffrey, A.R. Gallant, Convergence rate for single hidden layer feedforward networks, Technical report, RAND Corp., Santa Monica, CA and Department of Statistics, North Carolina State University, 1992.
34. Mhaskar H.N., Micchelli C.A. Approximation by superposition of a sigmoidal function. Adv. Appl. Math. 13:1992;350-373.
35. Moser B. Sugeno controllers with a bounded number of rules are nowhere dense. Fuzzy Sets Syst. 104(2):1999;269-277.
36. H.T. Nguyen, V. Kreinovich, On approximations of controls by fuzzy systems, Technical Report TR 92-93/302, LIFE Chair of Fuzzy Theory, Tokyo Institute of Technology, Tokyo, 1992.
37. Poggio T., Girosi F. Representation properties of networks: Kolmogorov's theorem is irrelevant. Neural Comput. 1(4):1989;465-469.
38. Raju G.V.S., Zhou J., Kisner R.A. Hierarchical fuzzy control. Int. J. Control. 54:1991;1201-1216.
39. Sprecher D.A. On the structure of continuous functions of several variables. Trans. Am. Math. Soc. 115:1965;340-355.
40. D. Tikk, Notes on the approximation rate of KH controllers, Fuzzy Sets Syst., in press [ doi:10.1016/S0165-0114(02)00387-1 ].
41. Tikk D. On nowhere denseness of certain fuzzy controllers containing prerestricted number of rules. Tatra Mountains Math. Publ. 16:1999;369-377.
42. D. Tikk, P. Baranyi, R.J. Patton, Polytopic and TS model are nowhere dense in the approximation model space, in: Proc. of the Int. Conf. on Systems, Man and Cybernetics (SMC 2002), Hammamet, Tunisia, October 2002, p. 5.
43. D. Tikk, P. Baranyi, Y. Yam, L.T. Kóczy, Stability of a new interpolation method, in: Proc. of the IEEE Int. Conf. on System, Man, and Cybernetics (IEEE SMC'99), Tokyo, Japan, vol. III, 1999, pp. 7-9.
44. Tikk D., Joó I., Kóczy L.T., Várlaki P., Moser B., Gedeon T.D. Stability of interpolative fuzzy KH-controllers. Fuzzy Sets Syst. 125(1):2002;105-119.
45. L.X. Wang, Fuzzy systems are universal approximators, in: Proc. of the IEEE Int. Conf. on Fuzzy Systems, San Diego, 1992, pp. 1163-1169.
46. Wang L.X. Universal approximation by hierarchical fuzzy systems. Fuzzy Sets Syst. 93:1998;223-230.
47. Wang L.X. Analysis and design of hierarchical fuzzy systems. IEEE Trans. Fuzzy Syst. 7(5):1999;617-624.
48. L.X. Wang, J. Mendel, Generating fuzzy rules from numerical data with supplications, Technical Report TR USC-SIPI #169, Signal and Image Processing Institute, University of Southern California, 1991.
49. Wei C., Wang L.X. A note on universal approximation by hierarchical fuzzy systems. Inform. Sci. 123:2000;241-248.
50. R.R. Yager, V. Kreinovich, Universal approximation theorem for uninorm-based fuzzy systems modeling, Fuzzy Sets Syst., in press [doi:10.1016/S0165-0114(02)00521-3].
51. Ying H. General SISO Takagi-Sugeno fuzzy systems with linear rule consequents are universal approximators. IEEE Trans. Fuzzy Syst. 6(4):1998;582-587.
52. Ying H. General Takagi-Sugeno fuzzy systems with simplified linear rule consequents are universal controllers, models and filters. J. Inform. Sci. 108:1998;91-107.
53. Ying H. Sufficient conditions on uniform approximation of multivariate functions by general Takagi-Sugeno fuzzy systems with linear rule consequents. IEEE Trans. SMC Part A. 28(4):1998;515-520.
54. Zadeh L.A. Outline of a new approach to the analysis of complex systems and decision processes. IEEE Trans. SMC. 3:1973;28-44.
55. Zeng K., Zhang N.-Y., Xu W.-L. A comparative study on sufficient conditions for Takagi-Sugeno fuzzy systems as universal approximators. IEEE Trans. Fuzzy Syst. 8(6):2000;773-780.