Approximation methods for supervised learning

Foundations of Computational Mathematics

Volumn 6, Issue 1, 2006, Pages 3-58

  DeVore, Ronald ^a   Kerkyacharian, Gerard ^b   Picard, Dominique ^c   Temlyakov, Vladimir ^a  

^a UNIVERSITY OF SOUTH CAROLINA   (United States)

^b UNIVERSITÉ PARIS OUEST NANTERRE LA DÉFENSE   (France)

^c UNIVERSITÉ PARIS DESCARTES   (France)

Author keywords

Entropy; Learning theory; Nonlinear methods; Regression estimation; Upper and lower estimates

Indexed keywords

APPROXIMATION METHODS; KULLBACK-LEIBLER INFORMATION; LEARNING THEORY; NON-LINEAR METHODS; NONLINEAR APPROXIMATION; RATE OF APPROXIMATION; REGRESSION ESTIMATION; UPPER AND LOWER ESTIMATES;

ALGORITHMS; ENTROPY; OPTIMIZATION;

ESTIMATION;

EID: 31944434635     PISSN: 16153375     EISSN: 16153383     Source Type: Journal    
DOI: 10.1007/s10208-004-0158-6     Document Type: Article

References (42)

1. P. S. Alexandroff, Combinatorial Topology, Vol. 1, Graylock Press, Rochester, NY, 1956.
2. S. Bernstein, The Theory of Probabilities, Gastehizdat, Moscow, 1946.
3. L. Birgé. Approximation dans les espaces métriques et théorie d l'estimation, Z. Wahrsch. Verw. Gebiete 65 (1983), 181-237.
4. L. Birgé and P. Massart, Rates of convergence for minimum contrast estimators, Probab. Theory Related Fields 97 (1993), 113-150.
5. P. Binev, A. Cohen, W. Dahmen, R. DeVore, and V. Temlyakov, Universal algorithms for learning theory. Part 1: Piecewise constant functions, Manuscript (2004), 1-24.
6. A. Cohen, I. Daubechies, and P. Vial, Wavelets and fast wavelet transforms on an interval, Appl. Comput. Harmon. Anal. 1 (1993), 54-81.
7. B. Carl, Entropy numbers, s-numbers, and eigenvalue problems, J. Funct. Anal. 41 (1981), 290-306.
8. A. Cohen, R. DeVore, and R. Hochmuth, Restricted nonlinear approximation, Constr. Approx. 16 (2000), 85-113.
9. A. Cohen, W. Dahmen, I. Daubechies, and R. DeVore, Tree approximation and encoding, ACHA 11 (2001), 192-226.
10. F. Cucker and S. Smale, On the mathematical foundations of learning theory, Bull. Amer. Math. Soc. 39 (2002), 1-49.
11. I. Daubechies, Ten Lectures on Wavelets, CBMS-NSF Regional Coference Series in Applied Mathematics, SLAM, Philadelphia, 1992.
12. A. Dembo and O.Zeitouni, Large Deviation Techniques and Applications, Springer-Verlag, New York, 1998.
13. R. DeVore, Nonlinear approximation, Acta Numer. 7 (1998), 51-150.
14. R. DeVore, R. Howard, and C. Micchelli, Optimal non-linear approximation, Manuscripta Math. 63 (1989), 469-478.
15. d, Trans. Amer. Math. Soc. 335 (1993), 843-864.
16. R. DeVore and B. Lucier, Wavelets, Acta Numer. 1 (1992), 1-56.
17. D. Donoho and I. Johnstone, Ideal spatial adaptation by wavelet shrinkage, Biometrika 81 (1994), 425-455.
18. D. Donoho, I. Johnstone, G. Kerkyacharian, and D. Picard, Wavelet shrinkage: Asymptopia? J. Roy. Stat. Soc. Ser. B 57 (1995), 301-369.
19. L. Györfi, M. Kohler, A. Krzyzak, and H. Walk, A Distribution-free Theory of Nonparametric Regression, Springer Series in Statistics, Springer-Verlag, New York, 2002.
20. A. Gushin, On Fano lemma and similar inequalities for the minimax risk, Theory Probab. Math. Statist, (to appear).
21. I. A. Ibragimov and R. Z. Hasminskii, Statistical Estimation: Asymptotic Theory, Springer-Verlag, New York, 1981.
22. G. Kerkyacharian and D. Picard, Entropy, Universal coding, approximation and bases properties, Constr. Approx. 20 (2004), 1-37.
23. B. S. Kashin and V. N. Temlyakov, On a norm and approximation characteristics of classes of functions of several variables, in Metric Theory of Functions and Related Problems in Analysis, Izd. Nauchno-Issled. Aktuarno-Finans. Tsentra (AFTs), Moscow, 1999, pp. 69-99.
24. S. Konyagin and V. Temlyakov, Greedy approximation with regard to bases and general minimal systems, Serdica Math. J. 28 (2002), 305-328.
25. S. Konyagin and V. Temlyakov, Some error estimates in learning theory, IMI Preprint 05 (2004), 1-18.
26. S. Konyagin and V. Temlyakov, The entropy in learning theory: Error estimates, IMI Preprint 09 (2004), 1-25.
27. L. Le Cam, Convergence of estimates under dimensionality restriction, Ann. Statist. 1 (1973), 38-53.
28. M. Ledoux and M. Talagrand, Probability in Banach Spaces: Isoperimetry and Processes, Springer-Verlag, New York, 1991.
29. G. Lorentz, M. Von Golitschek, and Yu. Makovoz, Constructive Approximation: Advanced. Problems, Grundlehren der Mathematischen Wissenschaften, Vol. 304, Springer-Verlag, Berlin, 1996.
30. Y. Meyer, Ondelettes et Operateurs I. Hermann, Paris, 1990.
31. T. Poggio and S. Smale, The mathematics of learning: Dealing with data, Notices Amer. Math. Soc. 50 (2003), 537-544.
32. S. Smale and D-X. Zhou, Estimating the approximation error in learning theory, Anal. Appl 1 (2003), 17-41.
33. E. Stein, Singular Integrals and Differentiability Properties of Functions, Princeton University Press, Princeton, NJ, 1970.
34. M. Talagrand, New concentration inequalities in product spaces, Invent. Math. 126 (1996), 505-563.
35. V. Temlyakov, Nonlinear Kolmogorov widths, Matem. Zametki 63 (1998), 891-902.
36. V. Temlyakov, Approximation by elements of a finite dimensional subspace of functions from various Sobolev or Nikol'skii spaces, Math. Notes 43 (1988), 444-454.
37. V. Temlyakov, The best m-term approximation and greedy algorithms, Adv. Comput. Math. 8 (1998), 249-265
38. S. Van de Geer, Empirical Process in M-Estimaton, Cambridge University Press, New York, 2000.
39. P. Wojtaszczyk, Greedy algorithm for general biorthogonal systems, J. Approx. Theory 107 (2000), 293-314.
40. d), Proc. London Math. Soc. 87 (2003), 471-497.
41. Y. Yang and A. Barron, Information-theoretic determination of minimax rates of convergence, Ann. Statist. 27 (1999), 1564-1599.
42. W. P. Ziemer, Weakly Differentiable Functions, Springer-Verlag, New York, 1989.