Can dictionary-based computational models outperform the best linear ones?

Neural Networks

Volumn 24, Issue 8, 2011, Pages 881-887

  Gnecco, Giorgio ^a   Kůrková, Věra ^b   Sanguineti, Marcello ^a  

^a UNIVERSITY OF GENOA   (Italy)

^b INSTITUTE OF COMPUTER SCIENCE   (Czech Republic)

Author keywords

Dictionary based approximation; Kolmogorov width; Linear approximation; Perceptron networks; Rates of approximation; Worst case error

Indexed keywords

DICTIONARY-BASED APPROXIMATION; KOLMOGOROV WIDTH; LINEAR APPROXIMATIONS; PERCEPTRON NETWORK; RATES OF APPROXIMATION; WORST-CASE ERRORS;

ERRORS;

COMPUTATIONAL METHODS;

ANALYTICAL ERROR; ARTICLE; DICTIONARY BASED COMPUTATIONAL MODEL; GEOMETRY; INTERMETHOD COMPARISON; LINEAR SYSTEM; MATHEMATICAL ANALYSIS; MATHEMATICAL MODEL; PRIORITY JOURNAL;

ALGORITHMS; COMPUTER SIMULATION; LINEAR MODELS; NEURAL NETWORKS (COMPUTER); REPRODUCIBILITY OF RESULTS;

EID: 80051786839     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2011.05.014     Document Type: Article

References (25)

1. Alessandri A., Sanguineti M. Optimization of approximating networks for optimal fault diagnosis. Optimization Methods and Software 2005, 20:235-260.
2. Barron A.R. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information Theory 1993, 39:930-945.
3. Bellman R. Dynamic programming 1957, Princeton University Press, Princeton, NJ.
4. Giulini S., Sanguineti M. Approximation schemes for functional optimization problems. Journal of Optimization Theory and Applications 2009, 140:33-54.
5. Gnecco G., Kůrková V., Sanguineti M. Some comparisons of model complexity in linear and neural-network approximation. Lecture notes in computer science 2010, Vol. 6354:358-367. Springer, Berlin, Heidelberg.
6. Gnecco G., Sanguineti M. Accuracy of suboptimal solutions to kernel principal component analysis. Computational Optimization and Applications 2009, 42:265-287.
7. Gnecco G., Sanguineti M. Regularization techniques and suboptimal solutions to optimization problems in learning from data. Neural Computation 2010, 22:793-829.
8. Gribonval R., Vandergheynst P. On the exponential convergence of matching pursuits in quasi-incoherent dictionaries. IEEE Transactions on Information Theory 2006, 52:255-261.
9. Jones L.K. A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training. Annals of Statistics 1992, 24:608-613.
10. Kainen P.C., Kůrková V., Sanguineti M. Complexity of Gaussian radial basis networks approximating smooth functions. Journal of Complexity 2009, 25:63-74.
11. Kainen P.C., Kůrková V., Sanguineti M. On tractability of neural-network approximation. Lecture notes in computer science 2009, Vol. 5495:11-21.
12. Kainen P.C., Kůrková V., Vogt A. A Sobolev-type upper bound for rates of approximation by linear combinations of Heaviside plane waves. Journal of Approximation Theory 2007, 147:1-10.
13. Knuth D.E. Big omicron and big omega and big theta. SIGACT News 1976, 8:18-24.
14. Kolmogorov A.N. Über die beste annäherung von funktionen einer gegebenen funktionenklasse. Annals of Mathematics 1936, 37:107-110. English translation: "On the best approximation of functions of a given class", in Selected works of A. N. Kolmogorov, vol. I; V. M. Tikhomirov Ed., pp. 202-205. Kluwer, 1991.
15. Kolmogorov A.N., Fomin S.V. Introductory real analysis 1970, Dover, New York.
16. Kůrková V. Dimension-independent rates of approximation by neural networks. Computer-intensive methods in control and signal processing. the curse of dimensionality 1997, 261-270. Birkhäuser, Boston. K. Warwick, M. Kárný (Eds.).
17. Kůrková V. High-dimensional approximation and optimization by neural networks. Advances in learning theory: methods, models and applications 2003, 69-88. IOS Press, Amsterdam, Chapter 4. J. Suykens, G. Horváth, S. Basu, C. Micchelli, J. Vandewalle (Eds.).
18. Kůrková V., Sanguineti M. Bounds on rates of variable-basis and neural-network approximation. IEEE Transactions on Information Theory 2001, 47:2659-2665.
19. Kůrková V., Sanguineti M. Comparison of worst case errors in linear and neural network approximation. IEEE Transactions on Information Theory 2002, 48:264-275.
20. Kůrková V., Sanguineti M. Approximate minimization of the regularized expected error over kernel models. Mathematics of Operations Research 2008, 33:747-756.
21. Kůrková V., Sanguineti M. Geometric upper bounds on rates of variable-basis approximation. IEEE Transactions on Information Theory 2008, 54:5681-5688.
22. Lavretsky E. On the geometric convergence of neural approximations. IEEE Transactions on Neural Networks 2002, 13:274-282.
23. Pinkus A. n-Widths in approximation theory 1985, Springer, Berlin, Heidelberg.
24. Pisier, G. (1981). Remarques sur un résultat non publié de B. Maurey. In: Séminaire d'Analyse Fonctionnelle 1980-1981, vol. I, no. 12. École Polytechnique, Centre de Mathématiques, Palaiseau, France.
25. Zoppoli R., Sanguineti M., Parisini T. Approximating networks and extended Ritz method for the solution of functional optimization problems. Journal of Optimization Theory and Applications 2002, 112:403-439.