Optimal aggregation of affine estimators

Journal of Machine Learning Research

Volumn 19, Issue , 2011, Pages 635-660

  Salmon, Joseph ^a   Dalalyan, Arnak ^b  

^a UNIVERSITÉ PARIS DESCARTES   (France)

^b UNIVERSITÉ PARIS EST   (France)

Author keywords

List of keywords

Indexed keywords

ESTIMATION; GAUSSIAN NOISE (ELECTRONIC); INVERSE PROBLEMS; REGRESSION ANALYSIS;

ADAPTIVE ESTIMATORS; KERNEL RIDGE REGRESSIONS; LEAST SQUARE REGRESSION; LIST OF KEYWORDS; NON-PARAMETRIC REGRESSION; ORACLE INEQUALITIES; STATISTICAL INVERSE PROBLEMS; TUNING PARAMETER;

AGGREGATES;

EID: 84898450217     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (49)

1. P. Alquier and K. Lounici. Pac-bayesian bounds for sparse regression estimation with exponential weights. Electron. J. Statist., 5:127-145, 2010.
2. Y. Amit and D. Geman. Shape quantization and recognition with randomized trees. Neural Comput., 9:1545-1588, October 1997. (Pubitemid 127462804)
3. S. Arlot and F. Bach. Data-driven calibration of linear estimators with minimal penalties. In NIPS, pages 46-54, 2009.
4. J-Y. Audibert. Fast learning rates in statistical inference through aggregation. Ann. Statist., 37 (4):1591-1646, 2009.
5. F. Bach. Consistency of the group lasso and multiple kernel learning. J. Mach. Learn. Res., 9: 1179-1225, 2008.
6. Y. Baraud, Ch. Giraud, and S. Huet. Estimator selection in the gaussian setting, submitted, 2010.
7. A. R. Barron, L. Birgé, and P Massart. Risk bounds for model selection via penalization. Probab. Theory Related Fields, 113(3):301-413, 1999.
8. L. Breiman. Bagging predictors. Mach. Learn., 24(2):123-140, 1996. (Pubitemid 126724382)
9. A. Buades, B. Coll, and J-M. Morel. A review of image denoising algorithms, with a new one. Multiscale Model. Simul., 4(2):490-530, 2005. (Pubitemid 43890582)
10. F. Bunea, A. B. Tsybakov, and M. H. Wegkamp. Aggregation for Gaussian regression. Ann. Statist., 35(4): 1674-1697, 2007.
11. T. T. Cai. Adaptive wavelet estimation: a block thresholding and oracle inequality approach. Ann. Statist., 27(3):898-924, 1999. ISSN 0090-5364.
12. O. Catoni. Statistical learning theory and stochastic optimization, volume 1851 of Lecture Notes in Mathematics. Springer-Verlag, Berlin, 2004.
13. L. Cavalier, G. K. Golubev, D. Picard, and A. B. Tsybakov. Oracle inequalities for inverse problems. Ann. Statist., 30(3):843-874, 2002. (Pubitemid 37095353)
14. A. Cohen. All admissible linear estimates of the mean vector. The Annals of Mathematical Statistics, 37(2):458-463, 1966. ISSN 00034851.
15. A. S. Dalalyan and J. Salmon. Sharp oracle inequalities for aggregation of affine estimators. Technical Report arXiv:1104.3969v2 [math.ST], April 2011.
16. A. S. Dalalyan and A. B. Tsybakov. Aggregation by exponential weighting, sharp oracle inequalities and sparsity. In COLT, pages 97-111, 2007.
17. A. S. Dalalyan and A. B. Tsybakov. Aggregation by exponential weighting, sharp pac-bayesian bounds and sparsity. Mach. Learn., 72(1-2):39-61, 2008.
18. A. S. Dalalyan and A. B. Tsybakov. Sparse regression learning by aggregation and Langevin Monte-Carlo. In COLT, 2009.
19. D. L. Donoho and I. M. Johnstone. Ideal spatial adaptation by wavelet shrinkage. Biometrilca, 81(3):425-455, 1994.
20. D. L. Donoho and I. M. Johnstone. Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc., 90(432):1200-1224, 1995.
21. S. Y. Efromovich. On nonparametric regression for IID observations in a general setting. Ann. Statist., 24(3):1125-1144, 1996.
22. S. Y. Efromovich and M. S. Pinsker. A self-training algorithm for nonparametric filtering. Avtomat. i Telemekh., 1 (11) :585, 1984.
23. S. Y. Efromovich and M. S. Pinsker. Sharp-optimal and adaptive estimation for heteroscedastic nonparametric regression. Statist. Sinica, 6(4):925-942, 1996.
24. Y. Freund. Boosting a weak learning algorithm by majority. In Proceedings of the third annual workshop on Computational learning theory, COLT, pages 202-216, 1990.
25. E. I. George. Minimax multiple shrinkage estimation. Ann. Statist., 14(1):188-205, 1986.
26. Ch. Giraud. Mixing least-squares estimators when the variance is unknown. Bernoulli, 14(4): 1089-1107, 2008.
27. G. K. Golubev. Nonparametric estimation of smooth densities of a distribution in L-2. Problemy Peredachi Informatsii, 28(1):52-62, 1992.
28. Y. Golubev. On universal oracle inequalities related to high-dimensional linear models. Ann. Statist., 38(5):2751-2780, 2010.
29. A. B. Juditsky and A. S. Nemirovski. Functional aggregation for nonparametric regression. Ann. Statist., 28(3):681-712, 2000.
30. A. B. Juditsky and A. S. Nemirovski. Nonparametric denoising of signals with unknown local structure. I. Oracle inequalities. Appil Comput. Harmon. Anal., 27(2):157-179, 2009.
31. G. R. G. Lanckriet, N. Cristianini, P Bartlett, L. El Ghaoui, and M. Jordan. Learning the kernel matrix with semidefinite programming. J. Mach. Learn. Res., 5:27-72 (electronic), 2003/04.
32. G. Lecué. Optimal rates of aggregation in classification under low noise assumption. Bernoulli, 13(4):1000-1022, 2007.
33. G. Leung. Information Theory and Mixing Least Squares Regression. PhD thesis, Yale University, 2004.
34. G. Leung and A. R. Barron. Information theory and mixing least-squares regressions. IEEE Trans. Inf. Theory, 52(8):3396-3410, 2006. (Pubitemid 44145107)
35. K. Lounici. Generalized mirror averaging and D-convex aggregation. Math. Methods Statist., 16 (3):246-259, 2007.
36. A. S. Nemirovski. Topics in non-parametric statistics, volume 1738 of Lecture Notes in Math. Springer, Berlin, 2000.
37. M. S. Pinsker. Optimal filtration of square-integrable signals in Gaussian noise. Probl. Peredachi Inf, 16(2):52-68, 1980.
38. Ph. Rigollet and A. B. Tsybakov. Linear and convex aggregation of density estimators. Math. Methods Statist., 16(3):260-280, 2007.
39. Ph. Rigollet and A. B. Tsybakov. Exponential screening and optimal rates of sparse estimation. Ann. Statist., 39(2):731-471, 2011.
40. J. Salmon and E. Le Pennec. NL-Means and aggregation procedures. In ICIP, pages 2977-2980, 2009.
41. J. Shawe-Taylor and N. Cristianini. An introduction to support vector machines : and other Kernel-based learning methods. Cambridge University Press, 2000.
42. C. M. Stein. Estimation of the mean of a multivariate normal distribution. Ann. Statist., 9(6): 1135-1151, 1981.
43. A. B. Tsybakov. Optimal rates of aggregation. In COLT, pages 303-313, 2003.
44. A. B. Tsybakov. Introduction to nonparametric estimation. Springer, New York, 2009.
45. Y. Yang. Combining different procedures for adaptive regression. J. Multivariate Anal., 74(1): 135-161, 2000.
46. Y. Yang. Regression with multiple candidate models: selecting or mixing? Statist. Sinica, 13(3): 783-809, 2003. (Pubitemid 37155854)
47. Y. Yang. Aggregating regression procedures to improve performance. Bernoulli, 10(l):25-47, 2004. (Pubitemid 44242782)
48. M. Yuan and Y. Lin. Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B Stat. Methodol., 68(1):49-67, 2006. (Pubitemid 43415335)
49. T. Zhang. Information-theoretic upper and lower bounds for statistical estimation. IEEE Trans. Inform. Theory, 52(4):1307-1321, 2006. ISSN 1557-9654.