Some comparisons of complexity in dictionary-based and linear computational models

Neural Networks

Volumn 24, Issue 2, 2011, Pages 171-182

  Gnecco, Giorgio ^a   Kůrková, Věra ^b   Sanguineti, Marcello ^a  

^a UNIVERSITY OF GENOA   (Italy)

^b INSTITUTE OF COMPUTER SCIENCE   (Czech Republic)

Author keywords

Kernel models; Linear approximation schemes; Model complexity; Neural networks; Variable basis approximation schemes; Worst case errors

Indexed keywords

APPROXIMATION SCHEME; KERNEL MODELS; LINEAR APPROXIMATIONS; MODEL COMPLEXITY; WORST-CASE ERRORS;

ERRORS; ORTHOGONAL FUNCTIONS; POLYNOMIALS;

NEURAL NETWORKS;

ACCURACY; ANALYTICAL ERROR; ARTICLE; ARTIFICIAL NEURAL NETWORK; INTERMETHOD COMPARISON; KERNEL METHOD; LINEAR REGRESSION ANALYSIS; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; PRIORITY JOURNAL;

COMPUTATIONAL BIOLOGY; DICTIONARIES AS TOPIC; LINEAR MODELS; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER); STATISTICS, NONPARAMETRIC;

EID: 79251629004     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2010.10.002     Document Type: Article

References (44)

1. Akhiezer N.I., Glazman I.M. Theory of linear operators in Hilbert space 1993, Dover, New York.
2. Barron A.R. Neural net approximation. Proc. 7th Yale workshop on adaptive and learning systems 1992, 69-72. Yale University Press.
3. Barron A.R. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information Theory 1993, 39:930-945.
4. Courant R. Differential and integral calculus 1988, Vol. II. Wiley.
5. Friedman A. Foundations of modern analysis 1982, Dover, New York.
6. Girosi F., Anzellotti G. Rates of convergence for radial basis functions and neural networks. Artificial neural networks for speech and vision 1993, 97-113. Chapman & Hall. R.J. Mammone (Ed.).
7. Giulini, S., & Sanguineti, M. (2000). On dimension-independent approximation by neural networks and linear approximators. In Proc. int. joint conference on neural networks (pp. I283-I288).
8. Giulini S., Sanguineti M. Approximation schemes for functional optimization problems. Journal of Optimization Theory and Applications 2009, 140:33-54.
9. Gnecco G., Kůrková V., Sanguineti M. Some comparisons of model complexity in linear and neural-network approximation. LNCS 2010, Vol. 6352:358-367. Springer-Verlag. K. Diamantaras, W. Duch, L. Iliadis (Eds.).
10. Gnecco, G., & Sanguineti, M. On a variational norm tailored to variable-basis approximation schemes. IEEE Transactions on Information Theory (in press).
11. Golub G.H., Loan C.F.V. Matrix computations 1996, John Hopkins University Press, London.
12. Gribonval R., Vandergheynst P. On the exponential convergence of matching pursuits in quasi-incoherent dictionaries. IEEE Transactions on Information Theory 2006, 52:255-261.
13. Jones L.K. A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training. Annals of Statistics 1992, 24:608-613.
14. Kainen P.C., Kůrková V. An integral upper bound for neural network approximation. Neural Computation 2009, 21:2970-2989.
15. Kainen P.C., Kůrková V., Sanguineti M. Complexity of Gaussian radial basis networks approximating smooth functions. Journal of Complexity 2009, 25:63-74.
16. Kainen P.C., Kůrková V., Vogt A. Best approximation by Heaviside perceptron networks. Neural Networks 2000, 13:695-697.
17. Kainen P.C., Kůrková V., Vogt A. Geometry and topology of continuous best and near best approximations. Journal of Approximation Theory 2000, 105:252-262.
18. Kainen P.C., Kůrková V., Vogt A. Continuity of approximation by neural networks in Lp-spaces. Annals of Operations Research 2001, 101:143-147.
19. Kainen P.C., Kůrková V., Vogt A. A Sobolev-type upper bound for rates of approximation by linear combinations of Heaviside plane waves. Journal of Approximation Theory 2007, 147:1-10.
20. Kecman V. Learning and soft computing 2001, MIT Press, Cambridge.
21. Kolmogorov A.N. Über die beste Annäherung von Funktionen einer gegebenen Funktionenklasse. Annals of Mathematics 1936, 37(1):107-110. (English translation: "On the best approximation of functions of a given class", In: V.M. Tikhomirov (Ed.), Selected works of A.N. Kolmogorov: Vol. I (pp. 202-205) Kluwer. 1991).
22. Kolmogorov A.N., Fomin S.V. Introductory real analysis 1970, Dover, New York.
23. Kůrková V. Dimension-independent rates of approximation by neural networks. Computer-intensive methods in control and signal processing. The curse of dimensionality 1997, 261-270. Birkhäuser, Boston. K. Warwick, M. Kárný (Eds.).
24. Kůrková V. High-dimensional approximation and optimization by neural networks. Advances in learning theory: methods, models and applications 2003, 69-88. IOS Press, Amsterdam, (Chapter 4). J. Suykens, G. Horváth, S. Basu, C. Micchelli, J. Vandewalle (Eds.).
25. Kůrková V. Model complexity of neural networks and integral transforms. Lecture notes in computer science 2009, Vol. 5768:708-718. Springer, Berlin, Heidelberg. M. Polycarpou, C. Panayiotou, C. Alippi, G. Ellinas (Eds.).
26. Kůrková, V. (2010). Integral transforms and norms induced by computational units (submitted for publication).
27. Kůrková V., Kainen P.C., Kreinovich V. Estimates of the number of hidden units and variation with respect to half-spaces. Neural Networks 1997, 10:1061-1068.
28. Kůrková V., Sanguineti M. Bounds on rates of variable-basis and neural-network approximation. IEEE Transactions on Information Theory 2001, 47:2659-2665.
29. Kůrková V., Sanguineti M. Comparison of worst case errors in linear and neural network approximation. IEEE Transactions on Information Theory 2002, 48:264-275.
30. Kůrková V., Sanguineti M. Error estimates for approximate optimization by the extended Ritz method. SIAM Journal on Optimization 2005, 15:461-487.
31. Kůrková V., Sanguineti M. Geometric upper bounds on rates of variable-basis approximation. IEEE Transactions on Information Theory 2008, 54:5681-5688.
32. Kůrková V., Savický P., Hlaváčková K. Representations and rates of approximation of real-valued Boolean functions by neural networks. Neural Networks 1998, 11:651-659.
33. Makovoz Y. Random approximants and neural networks. Journal of Approximation Theory 1996, 85:98-109.
34. Oden J.T., Demkowicz L.F. Applied functional analysis 1996, CRC Press.
35. Papoulis A. The Fourier integral and its applications 1962, McGraw-Hill, USA.
36. Pinkus A. n-widths in approximation theory 1985, Springer, Berlin, Heidelberg.
37. Pinkus A. Negative theorems in approximation theory. American Mathematical Monthly 2003, 110:900-911.
38. Pisier, G. (1981). Remarques sur un résultat non publié de B. Maurey. In Séminaire d'analyse fonctionnelle 1980-81: Vol. I, no. 12. École Polytechnique, Centre de Mathématiques, Palaiseau, France.
39. Rudin W. Real and complex analysis 1987, McGraw-Hill, New York.
40. Shubin M.A. Pseudodifferential operators and spectral theory 2001, Springer, Berlin, Heidelberg.
41. Smith K.A. Neural networks for combinatorial optimization: a review of more than a decade of research. INFORMS Journal on Computing 1999, 11:15-34.
42. Vandebril R., Van Barel M., Mastronardi N. Matrix computations and semiseparable matrices: linear systems 2008, The Johns Hopkins University Press.
43. Weidmann J. Linear operators in Hilbert spaces 1980, Springer, New York.
44. Zoppoli R., Sanguineti M., Parisini T. Approximating networks and extended Ritz method for the solution of functional optimization problems. Journal of Optimization Theory and Applications 2002, 112:403-439.