Bounds on the complexity of neural-network models and comparison with linear methods

International Journal of Adaptive Control and Signal Processing

Volumn 17, Issue 2, 2003, Pages 179-194

  Hlaváckova Schindler, Kateřina ^a   Sanguineti, Marcello ^b  

^a VIENNA UNIVERSITY OF TECHNOLOGY   (Austria)

^b UNIVERSITY OF GENOA   (Italy)

Author keywords

Curse of dimensionality; Neural networks; Non linear models; Polynomially bounded complexity

Indexed keywords

ADAPTIVE CONTROL SYSTEMS; COMPUTATIONAL COMPLEXITY; CONTROL SYSTEM ANALYSIS; MATHEMATICAL MODELS; NEURAL NETWORKS; POLYNOMIALS;

NONLINEAR MODELS; POLYNOMIALLY BOUNDED COMPLEXITY;

NONLINEAR CONTROL SYSTEMS;

EID: 0038681934     PISSN: 08906327     EISSN: None     Source Type: Journal    
DOI: 10.1002/acs.746     Document Type: Article

References (35)

1. Kanellakopoulos I, Kokotovic PV, Morse AS. Systematic design of adaptive controllers for feedback linearizable systems. IEEE Transactions on Automatic Control 1991; 11:1241-1253.
2. Sastry SS, Isidori A. Adaptive control of linearizable systems. IEEE Transactions on Automatic Control 1989; 34:1123-1131.
3. Chen FC, Liu CC. Adaptively controlling non-linear continuous-time systems using multilayer neural networks. IEEE Transactions on Automatic Control 1994; 39:1306-1310.
4. Chen FC, Khalil H. Adaptive control of a class of non-linear discrete-time systems using multilayer neural networks. IEEE Transactions on Automatic Control 1995; 40:791-801.
5. Narendra KS, Parthasarathi K. Identification and control of dynamical systems using neural networks. IEEE Transactions on Neural Networks 1990; 1:4-26.
6. Narendra KS, Mukhopadhyay S. Adaptive control using neural networks and approximate models. IEEE Transactions on Neural Networks 1997; 8:475-485.
7. Narendra KS, Balakrishnan J, Ciliz KM. Adaptation and learning using multiple models, switching, and tuning. IEEE Control Systems Magazine 1995, 15:37-51.
8. Pinkus A. Approximation theory of the MLP model in neural networks. Acta Numerica 1999; 8:143-196.
9. Kurková V. Universality and complexity of approximation of multivariable functions by feedforward networks. In Softcomputing and Industry: Recent Applications, Roy R, Koeppen M, Ovaska S, Furuhashi T, Hoffmann F (eds). Springer-Verlag: London, 2002; 13-24.
10. Bellman R. Dynamic Programming. Princeton University Press: Princeton, NJ, 1957.
11. Burr DJ. Experiments on neural net recognition of spoken and written text. IEEE Transactions on Acoustics, Speech, and Signal Processing 1988; 36:1162-1168.
12. Parisini T, Zoppoli R. Neural networks for feedback feedforward non-linear control systems. IEEE Transactions on Neural Networks 1994; 5:436-449.
13. Sejnowski TJ, Rosenberg C. Parallel networks that learn to pronounce English text. Complex Systems 1987; 1:145-168.
14. Zoppoli R, Parisini T. Learning techniques and neural networks for the solution of N-stage non-linear nonquadratic optimal control problems. In Systems, Models and Feedback: Theory and Applications, Isidori A, Tarn TJ (eds). Birkhäuser: Basel, 1992; 193-210.
15. Zoppoli R, Sanguineti M, Parisini T. Approximating networks and extended Ritz method for the solution of functional optimization problems. Journal of Optimization Theory and Applications 2002; 112:403-440.
16. Barron AR. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information Theory 1993; 39:930-945.
17. Kurková V, Sanguineti M. Comparison of worst case errors in linear and neural network approximation. IEEE Transactions on Information Theory 2002; 48:264-275.
18. Sjöberg J, Zhang Q, Ljung L, Benveniste A, Delyon B, Glorennec P-Y, Hjalmarsson H, Juditsky A. Nonlinear black-box models in system identification: a unified overview. Automatica 1995; 31:1691-1724.
19. Juditsky A, Hjalmarsson H, Benveniste A, Delyon B, Ljung L, Sjöberg J, Zhang Q. Nonlinear black-box models in system identification: mathematical foundations. Automatica 1995; 31:1725-1750.
20. Friedman A. Foundations of Modern Analysis. Dover: New York, 1982.
21. Pinkus A. N - Widths in Approximation Theory. Springer: New York, 1986.
22. DeVore R, Howard R, Micchelli C. Optimal non-linear approximation. Manuscripta Mathematica 1989; 63:469-478.
23. Kainen PC, Kurková V, Vogt A. Approximation by neural networks is not continuous. Neurocomputing 1999; 29:47-56.
24. Pisier G. Remarques sur un resultat non publié de B. Maurey. Seminaire d'Analyse Fonctionelle 1980-81; 1(12) École Polytechnique, Centre de Mathématiques, Palaiseau.
25. Jones LK. A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training. Annals of Statistics 1992; 20:608-613.
26. Kurková V. Dimension-independent rates of approximation by neural networks. In Computer-Intensive Methods in Control and Signal Processing. The Curse of Dimensionality, Warwick K, Kárný M (eds). Birkhäuser: Basel, 1997; 261-270.
27. Kurková V, Savický P, Hlaváčková K. Representations and rates of approximation of real-valued Boolean functions by neural networks. Neural Networks 1998; 11:651-659.
28. Breiman L. Hinging hyperplanes for regression, classification, and function approximation. IEEE Transactions on Information Theory 1993; 39:993-1013.
29. Giulini S, Sanguineti M. On dimension-independent approximation by neural networks and linear approximators. Proceedings of the International Joint Conference on Neural Networks, Como, Italy, 2000; I:283-288.
30. Lorentz GG. Approximation of Functions (2nd edn). Chelsea Publ. Company: New York, 1986.
31. Timan AF. Theory of Approximation of Functions of a Real Variable. Dover Publications Inc.: New York, 1993.
32. Wahlberg B. Laguerre and Kautz models. Proceedings of IFAC Symposium on System Identification; Copenhagen, Denmark, 1994; 965-976.
33. Mityagin BS. The approximate dimension and bases in nuclear spaces. Russian Mathematical Surveys 1961; 16:59-127 (Original Russian in Uspekhi Matematicheskikh Nauk. 1961, Tom XVI, 4:63-132).
34. Kurková V, Sanguineti M. Tight bounds on rates of variable-basis approximation via estimates of covering numbers. Research Report ICS-02-865 - Institute of Computer Science - Academy of Sciences of the Czech Republic, 2002.
35. Kurková V, Sanguineti M. Bounds on rates of variable-basis and neural-network approximation. IEEE Transactions on Information Theory 2001; 47:2659-2665.