Generalized mirror averaging and D-convex aggregation

Mathematical Methods of Statistics

Volumn 16, Issue 3, 2007, Pages 246-259

  Lounici, K ^a  

^a UNIVERSITÉ PARIS DESCARTES   (France)

Author keywords

aggregation; learning; mirror averaging; sparsity; stochastic optimization

Indexed keywords

EID: 79952905234     PISSN: 10665307     EISSN: 19348045     Source Type: Journal    
DOI: 10.3103/S1066530707030040     Document Type: Article

References (32)

1. J.-Y. Audibert, "A randomized online learning algorithm for better variance control", in Proc. 19th Annual Conf. on Learning Theory (COLT) (2006), pp. 392-407.
2. P. L. Bartlett, S. Boucheron, and G. Lugosi, "Model selection and error estimation", Machine Learning 48, 85-113 (2002).
3. L. Birgé and P. Massart, "Gaussian model selection", J. European Math. Soc. 3, 203-268 (2001).
4. S. Boucheron, O. Bousquet, and G. Lugosi, "Theory of classification: some recent advances", ESAIM Probab. and Statist. 9, 323-375 (2005).
5. F. Bunea and A. B. Nobel, Sequential Procedures for Aggregating Arbitrary Estimators of a Conditional Mean, Tech. Report no. 05-014 (Dept. of Statist. and Operations Research, UNC, Chapel Hill, 2005).
6. F. Bunea, A. B. Tsybakov, and M. H. Wegkamp, "Aggregation for Gaussian regression", Ann. Statist. 35, 1674-1697 (2007).
7. F. Bunea, A. B. Tsybakov, and M. H. Wegkamp, "Sparsity oracle inequalities for the Lasso", 2006 (manuscript).
8. O. Catoni, Universal Aggregation Rules with Exact Bias Bounds, Tech. Report no. 510 (Laboratoire de Probab. et Modèles Aléatoires, Univ. Paris 6 et Paris 7, 1999).
9. O. Catoni, Statistical Learning Theory and Stochastic Optimization, in Ecole d' Eté de Probab. de Saint-Flour XXXI-2001, Lecture Notes in Math. (Springer, New York, 2004), Vol. 1851.
10. N. Cesa-Bianchi and G. Lugosi, Prediction, Learning, and Games (Cambridge Univ. Press, Cambridge, 2006).
11. A. Dembo and O. Zeitouni, Large Deviations Techniques and Applications, 2nd ed. (Springer, New York, 1998).
12. W. Feller, An Introduction to Probability Theory and Its Application, 3rd ed. (Wiley, New York, 1968), Vol. 1.
13. D. Haussler, J. Kivinen, and M. K. Warmuth, "Sequential prediction of individual sequences under general loss functions", IEEE Trans. Inform. Theory 44(5), 1906-1925 (1998).
14. A. Juditsky and A. Nemirovski, "Functional aggregation for nonparametric regression", Ann. Statist. 28, 681-712 (2000).
15. A. Juditsky, A. Nazin, A. B. Tsybakov, and N. Vayatis, "Recursive aggregation of estimators by mirror descent algorithm with averaging," Problems of Inform. Transmission 41(4), 368-384 (2005).
16. A. Juditsky, Ph. Rigollet, and A. B. Tsybakov, Learning by Mirror Averaging, Tech. Report no. 1034 (Laboratoire de Probab. et Modèles Aléatoires, Univ. Paris 6 et Paris 7, 2006).
17. J. Kivinen and M. K. Warmuth, "Averaging expert predictions", in Proc. 4th European Conf. on Comput. Learning Theory, Ed. by H. U. Simon and P. Fischer, Lecture Notes in Artificial Intelligence (Springer, Berlin, 1999), Vol. 1572, pp. 153-167.
18. G. Leung and A. Barron, "Information theory and mixing least-square regressions", IEEE Trans. Inform. Theory 52(8), 3396-3410 (2006).
19. G. Lugosi and M. Wegkamp, "Complexity regularization via localized random penalties", Ann. Statist. 32, 1679-1697 (2004).
20. D. A. McAllester, "PAC-Bayesian Model Averaging", in Proc. 12th Annual Conf. on Comput. Learning Theory (1999), pp. 164-170.
21. A. Nemirovski, "Topics in nonparametric statistics", in Lectures on probability theory and statistics (Saint Flour, 1998). Lecture Notes in Math. (Springer, Berlin, 2000), Vol. 1738, 85-277.
22. A. S. Nemirovski and D. B. Yudin, Problem Complexity and Method Efficiency in Optimization (Wiley-Interscience, New York, 1983).
23. P. Rigollet and A. B. Tsybakov, Linear and Convex Aggregation of Density Estimators, Tech. Report (Laboratoire de Probab. et Modèles Aléatoires, Univ. Paris 6, 2006).
24. P. Rigollet and P. Zhao, "Continuous mirror averaging", 2006 (manuscript).
25. A. Samarov and A. Tsybakov, Aggregation of Density Estimators and Dimension Reduction, Tech. Report no. 1035 (Laboratoire de Probab. et Modèles Aléatoires, Univ. Paris 6 et Paris 7, 2005).
26. A. B. Tsybakov, Introduction à l'estimation non paramétrique (Springer, 2004).
27. A. B. Tsybakov, "Optimal rates of aggregation", in Proc. Comput. Learning Theory and Kernel Machines, Ed. by B. Scholkopf and M. Warmuth, Lecture Notes in Artificial Intelligence (Springer, Heidelberg, 2003), Vol. 2777, 303-313.
28. V. Vovk, "Aggregating strategies", in Proc. 3rd Annual Workshop on Comput. Learning Theory (Morgan Kaufman, San Mateo, 1990), pp. 372-383.
29. M. Wegkamp, "Model selection in nonparametric regression", Ann. Statist. 31, 252-273 (2003).
30. Y. Yang, "Combining different procedures for adaptive regression", J. Multivariate Anal. 74, 135-161 (2000).
31. Y. Yang, "Aggregating procedures for a better performance", Bernoulli 10, 25-47 (2004).
32. T. Zhang, "From epsilon-entropy to KL-complexity: analysis of minimum information complexity density estimation", Ann. Statist. 34, 2180-2210 (2006).