Kullback-leibler aggregation and misspecified generalized linear models1

Annals of Statistics

Volumn 40, Issue 2, 2012, Pages 639-665

  Rigollet, Philippe ^a  

^a Princeton University   (United States)

Author keywords

Aggregation; Classification; Finite sample bounds; Generalized linear models; Logistic regression; Minimax lower bounds; Oracle inequalities; Regression

Indexed keywords

EID: 84866861743     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/11-AOS961     Document Type: Article

References (43)

1. Akaike, H. (1973). information Theory and An Extension of The Maximum Likelihood Principle. in Second international Symposium on information Theory (Tsahkadsor, 1971) 267-281. Akad. Kiadó, Budapest. Mr0483125
2. Alquier, P. and Lounici, K. (2011). Pac-Bayesian Bounds for Sparse Regression Estimation with Exponential Weights. Electron. J. Stat. 5 127-145. Mr2786484
3. Audibert, J. Y. (2008). Progressive Mixture Rules are Deviation Suboptimal. in Advances in Neural information Processing Systems 20 (Y. S. J. Platt D. Koller and S. Roweis, Eds.) 41-48. Mit Press, Cambridge, Ma.
4. Barndorff-Nielsen, O. (1978). information and Exponential Families in Statistical Theory. Wiley, Chichester. Mr0489333
5. Bartlett, P. L.,Mendelson, S. and Neeman, J. (2012). 1-Regularized Linear Regression: Persistence and Oracle inequalities. Probab. Theory Related Fields. to Appear.
6. Belomestny, D. and Spokoiny, V. (2007). Spatial Aggregation of Local Likelihood Estimates with Applications to Classification. Ann. Statist. 35 2287-2311. Mr2363972
7. Boucheron, S., Bousquet, O. and Lugosi, G. (2005). Theory of Classification: A Survey of Some Recent Advances. Esaim Probab. Stat. 9 323-375. Mr2182250
8. Breiman, L. (1999). Prediction Games and Arcing Algorithms. Neural Comput. 11 1493-1517.
9. Brown, L. D. (1986). Fundamentals of Statistical Exponential Families with Applications in Statistical Decision Theory. institute of Mathematical Statistics Lecture Notes - Monograph Series 9. Ims, Hayward, Ca. Mr0882001
10. Bunea, F., Tsybakov, A. B. andwegkamp, M. H. (2007). Aggregation for Gaussian Regression. Ann. Statist. 35 1674-1697. Mr2351101
11. Catoni, O. (2004). Statistical Learning Theory and Stochastic Optimization. Lecture Notes in Math. 1851. Springer, Berlin. Lecture Notes From The 31St Summer School on Probability Theory Held in Saint-Flour, July 8-25, 2001. Mr2163920
12. Dalalyan, A. and Salmon, J. (2011). Sharp Oracle inequalities for Aggregation of Affine Estimators. Available At Arxiv:1104.3969.
13. Dalalyan, A. S. and Tsybakov, A. B. (2007). Aggregation By Exponential Weighting and Sharp Oracle inequalities. in Learning Theory. Lecture Notes in Computer Science 4539 97-111. Springer, Berlin. Mr2397581
14. Ekeland, I. and TéMam, R. (1999). Convex Analysis and Variational Problems. Classics in Applied Mathematics 28. Siam, Philadelphia, Pa. Mr1727362
15. Fahrmeir, L. andkaufmann, H. (1985). Consistency and Asymptotic Normality of The Maximum Likelihood Estimator in Generalized Linear Models. Ann. Statist. 13 342-368. Mr0773172
16. Freund, Y. and Schapire, R. E. (1996). Experiments with A New Boosting Algorithm. in international Conference on Machine Learning 148-156. Mr1473055
17. Friedman, J., Hastie, T. and Tibshirani, R. (2000). Additive Logistic Regression: A Statistical View of Boosting (with Discussion). Ann. Statist. 28 337-407. Mr1790002
18. Greenshtein, E. (2006). Best Subset Selection, Persistence in High-Dimensional Statistical Learning and Optimization Under L1 Constraint. Ann. Statist. 34 2367-2386. Mr2291503
19. Greenshtein, E. and Ritov, Y. (2004). Persistence in High-Dimensional Linear Predictor Selection and The Virtue of Overparametrization. Bernoulli 10 971-988. Mr2108039
20. Juditsky, A. and Nemirovski, A. (2000). Functional Aggregation for Nonparametric Regression. Ann. Statist. 28 681-712. Mr1792783
21. Juditsky, A., Rigollet, P. and Tsybakov, A. B. (2008). Learning By Mirror Averaging. Ann. Statist. 36 2183-2206. Mr2458184
22. Koltchinskii, V. (2011). Oracle inequalities in Empirical Risk Minimization and Sparse Recovery Problems. Lecture Notes in Math. 2033. Springer, Heidelberg. Mr2829871
23. Lecam, L. (1953). on Some Asymptotic Properties of Maximum Likelihood Estimates and Related Bayes' Estimates. Univ. California Publ. Statist. 1 277-329. Mr0054913
24. Lecué, G. (2007). Simultaneous Adaptation to The Margin and to Complexity in Classification. Ann. Statist. 35 1698-1721. Mr2351102
25. Lecué, G. (2012). Empirical Risk Minimization Is Optimal for The Convex Aggregation Problem. Bernoulli. to Appear.
26. Lecué, G. and Mendelson, S. (2009). Aggregation Via Empirical Risk Minimization. Probab. Theory Related Fields 145 591-613. Mr2529440
27. Lehmann, E. L. and Casella, G. (1998). Theory of Point Estimation, 2Nd Ed. Springer, New York. Mr1639875
28. Lounici, K. (2007). Generalized Mirror Averaging and D-Convex Aggregation. Math. Methods Statist. 16 246-259. Mr2356820
29. Massart, P. (2007). Concentration inequalities and Model Selection. Lecture Notes in Math. 1896. Springer, Berlin. Mr2319879
30. Mccullagh, P. and Nelder, J. A. (1989). Generalized Linear Models, 2Nd Ed. Chapman and Hall, London.
31. Mease, D. andwyner, A. (2008). Evidence Contrary to The Statistical View of Boosting. J. Mach. Learn. Res. 9 131-156.
32. Mitchell, C. and Van De Geer, S. (2009). General Oracle inequalities for Model Selection. Electron. J. Stat. 3 176-204. Mr2485876
33. Nemirovski, A. (2000). topics in Non-Parametric Statistics. in Lectures on Probability Theory and Statistics (Saint-Flour, 1998). Lecture Notes in Math. 1738 85-277. Springer, Berlin. Mr1775640
34. Nemirovski, A., Juditsky, A., Lan, G. and Shapiro, A. (2008). Robust Stochastic Approximation Approach to Stochastic Programming. Siam J. Optim. 19 1574-1609. Mr2486041
35. Raskutti, G., Wainwright, M. J. and Yu, B. (2011). Minimax Rates of Estimation for High-Dimensional Linear Regression Over Q -Balls. Ieee Trans. inform. Theory 57 6976-6994. Mr2882274
36. Rigollet, P. (2012). Supplement to "Kullback-Leibler Aggregation and Misspecified Generalized Linear Models." Doi:10.1214/11-Aos961Supp.
37. Rigollet, P. and Tsybakov, A. B. (2007). Linear and Convex Aggregation of Density Estimators. Math. Methods Statist. 16 260-280. Mr2356821
38. Rigollet, P. and Tsybakov, A. (2011). Exponential Screening and Optimal Rates of Sparse Estimation. Ann. Statist. 39 731-771. Mr2816337
39. Rigollet, P. and Tsybakov, A. (2012). Sparse Estimation By Exponential Weighting. Statist. Sci. to Appear.
40. Tsybakov, A. B. (2003). Optimal Rates of Aggregation. in Colt (B. SchöLkopf andm. K.Warmuth, Eds.). Lecture Notes in Computer Science 2777 303-313. Springer, Berlin.
41. White, H. (1982). Maximum Likelihood Estimation of Misspecified Models. Econometrica 50 1-25. Mr0640163
42. Yang, Y. (2000). Mixing Strategies for Density Estimation. Ann. Statist. 28 75-87. Mr1762904
43. Yang, Y. (2004). Aggregating Regression Procedures to Improve Performance. Bernoulli 10 25-47. Mr2044592