Aggregation for Gussian regression

Annals of Statistics

Volumn 35, Issue 4, 2007, Pages 1674-1697

  Bunea, Florentina ^a   Tsybakov A, Alexandre B ^b   Wegkamp, Marten H ^a  

^a FLORIDA STATE UNIVERSITY   (United States)

^b SORBONNE UNIVERSITÉ   (France)

Author keywords

Aggregation; Lasso estimator; Minimax risk; Model averaging; Model selection; Nonparametric regression; Oracle inequalities; Penalized least squares

Indexed keywords

EID: 38049043619     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/009053606000001587     Document Type: Article

References (45)

1. AKAIKE, H. (1974). A new look at the statistical model identification. IEEE Trans. Automat. Control 19 716-723.
2. ANTONIADIS, A. and FAN, J. (2001). Regularization of wavelet approximations (with discussion). J. Amer. Statist. Assoc. 96 939-967.
3. AUDIBERT, J.-Y. (2004). Aggregated estimators and empirical complexity for least square regression. Ann. Inst. H. Poincaré Probab. Statist. 40 685-736.
4. BARAUD, Y. (2000). Model selection for regression on a fixed design. Probab. Theory Related Fields 117 467-493.
5. BARAUD, Y. (2002). Model selection for regression on a random design. ESAIM Probab. Statist. 6 127-146.
6. BARRON, A. R. (1993). Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inform. Theory 39 930-945.
7. BARRON, A., BIRGÉ, L. and MASSART, P. (1999). Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 301-413.
8. BARTLETT, P. L., BOUCHERON, S. and LUGOSI, G. (2000). Model selection and error estimation. In Proc. 13th Annual Conference on Computational Learning Theory 286-297. Morgan Kaufmann, San Francisco.
9. BIRGÉ, L. (2006). Model selection via testing: An alternative to (penalized) maximum likelihood estimators. Ann. Inst. H. Poincaré Probab. Statist. 42 273-325.
10. BIRGÉ, L. and MASSART, P. (2001). Gaussian model selection. J. Eur. Math. Soc. 3 203-268.
11. p criterion for Gaussian model selection. Prépublication 647, Laboratoire de Probabilités et Modèles Aléatoires, Univ. Paris 6 and Paris 7. Available at www.proba.jussieu.fr/ mathdoc/preprints/index.html# 2001.
12. BUNEA, F. (2004). Consistent covariate selection and postmodel selection inference in semi-parametric regression. Ann. Statist. 32 898-927.
13. BUNEA, F. and NOBEL, A. B. (2005). Sequential procedures for aggregating arbitrary estimators of a conditional mean. Technical Report M984, Dept. Statistics, Florida State Univ.
14. BUNEA, F., TSYBAKOV, A. and WEGKAMP, M. H. (2004). Aggregation for regression learning. Available at www.arxiv.org/abs/ math/0410214. Prépublication 948, Laboratoire de Probabilités et Modèles Aléatoires, Univ. Paris 6 and Paris 7. Available at hal.ccsd.cnrs.fr/ccsd-00003205.
15. CATONI, O. (2004). Statistical Learning Theory and Stochastic Optimization. École d'Eté de Probabilités de Saint-Flour 2001. Lecture Notes in Math. 1851. Springer, Berlin.
16. CAVALIER, L., GOLUBEV, G. K., PICARD, D. and TSYBAKOV, A. B. (2002). Oracle inequalities for inverse problems. Ann. Statist. 30 843-874.
17. CHEN, S., DONOHO, D. and SAUNDERS, M. (2001). Atomic decomposition by basis pursuit. SIAM Rev. 43 129-159.
18. DEVROYE, L., GYÖRFI, L. and LUGOSI, G. (1996). A Probabilistic Theory of Pattern Recognition. Springer, New York.
19. DONOHO, D. L., ELAD, M. and TEMLYAKOV, V. (2006). Stable recovery of sparse over-complete representations in the presence of noise. IEEE Trans. Inform. Theory 52 6-18.
20. EFRON, B., HASTIE, T., JOHNSTONE, I. and TIBSHIRANI, R. (2004). Least angle regression (with discussion). Ann. Statist. 32 407-499.
21. FOSTER, D. and GEORGE, E. (1994). The risk inflation criterion for multiple regression. Ann. Statist. 22 1947-1975.
22. GILBERT, E. N. (1952). A comparison of signalling alphabets. Bell System Tech. J. 31 504-522.
23. GYÖRFI, L., KOHLER, M., KRZYŻAK, A. and WALK, H. (2002). A Distribution-Free Theory of Nonparametric Regression. Springer, New York.
24. HÄRDLE, W., KERKYACHARIAN, G., PICARD, D. and TSYBAKOV, A. (1998). Wavelets, Approximation and Statistical Applications. Lecture Notes in Statist. 129. Springer, New York.
25. JUDITSKY, A., NAZIN, A., TSYBAKOV, A. and VAYATIS, N. (2005). Recursive aggregation of estimators by the mirror descent method with averaging. Problems Inform. Transmission 41 368-384.
26. JUDITSKY, A. andNEMIROVSKI, A. (2000). Functional aggregation for nonparametric regression. Ann. Statist. 28 681-712.
27. JUDITSKY, A., RIGOLLET, P. and TSYBAKOV, A. (2005). Learning by mirror averaging. Prépublication du Laboratoire de Probabilités et Modèles Aléatoires, Univ. Paris 6 and Paris 7. Available at hal.ccsd.cnrs.fr/ccsd-00014097.
28. KNEIP, A. (1994). Ordered linear smoothers. Ann. Statist. 22 835-866.
29. KOLTCHINSKII, V. (2006). Local Rademacher complexities and oracle inequalities in risk minimization (with discussion). Ann. Statist. 34 2593-2706.
30. LEUNG, G. and BARRON, A. R. (2006). Information theory and mixing least-squares regressions. IEEE Trans. Inform. Theory 52 3396-3410.
31. LOUBES, J.-M. and VAN DE GEER, S. A. (2002). Adaptive estimation with soft thresholding penalties. Statist. Neerlandica 56 454-479.
32. LUGOSI, G. and NOBEL, A. (1999). Adaptive model selection using empirical complexities. Ann. Statist. 27 1830-1864.
33. P. Technometrics 15 661-675.
34. NEMIROVSKI, A. (2000). Topics in non-parametric statistics. Lectures on Probability Theory and Statistics (Saint-Flour, 1998). Lecture Notes in Math. 1738 85-277. Springer, Berlin.
35. OSBORNE, M., PRESNELL, B. and TURLACH, B. (2000). On the LASSO and its dual. J. Comput. Graph. Statist. 9 319-337.
36. RAO, C. R. and WU, Y. (2001). On model selection (with discussion). In Model Selection (P. Lahiri, ed.) 1-64. IMS, Beachwood, OH.
37. SCHAPIRE, R. E. (1990). The strength of weak learnability. Machine Learning 5 197-227.
38. SCHWARZ, G. (1978). Estimating the dimension of a model. Ann. Statist. 6 461-464.
39. TIBSHIRANI, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267-288.
40. TSYBAKOV, A. B. (2003). Optimal rates of aggregation. In Learning Theory and Kernel Machines. Lecture Notes in Artificial Intelligence 2777 303-313. Springer, Heidelberg.
41. TSYBAKOV, A. B. (2004). Introduction à l'estimation non-paramétrique. Springer, Berlin.
42. WEGKAMP, M. H. (2003). Model selection in nonparametric regression. Ann. Statist. 31 252-273.
43. YANG, Y. (2000). Combining different procedures for adaptive regression. J. Multivariate Anal. 74 135-161.
44. YANG, Y. (2001). Adaptive regression by mixing. J. Amer. Statist. Assoc. 96 574-588.
45. YANG, Y. (2004). Aggregating regression procedures to improve performance. Bernoulli 10 25-47.