Minimax risks for sparse regressions: Ultra-high dimensional phenomenons

Electronic Journal of Statistics

Volumn 6, Issue , 2012, Pages 38-90

  Verzelen, Nicolas ^a  

^a UMR 729 MISTEA   (France)

Author keywords

Adaptive estimation; Dimension reduction; High dimensional geometry; Highdimensional regression; Minimax risk

Indexed keywords

EID: 84857090762     PISSN: 19357524     EISSN: None     Source Type: Journal    
DOI: 10.1214/12-EJS666     Document Type: Article

References (50)

1. Abramovich, F. and Grinshtein, V. (2010). MAP model selection in Gaussian regression. Electron. J. Stat. 4, 932-949. http://dx.doi.org/10.1214/10-EJS573. MR2721039 (2011j:62028)
2. Aldous, D. J. (1985). Exchangeability and related topics, École d'été de probabilités de Saint Flour XIII. Lecture Notes in Mathematics, Vol. 1117. Springer-Verlag, Berlin.
3. Arias-Castro, E., Candès, E. J., and Plan, Y. (2010). Global testing and sparse alternatives: Anova, multiple comparisons and the higher criticism. arXiv:1007.1434.
4. Baraniuk, R., Davenport, M., DeVore, R., and Wakin, M. (2008). A simple proof of the restricted isometry property for random matrices. Constr. Approx. 28, 3, 253-263. http://dx.doi.org/10.1007/s00365-007-9003-x. MR2453366
5. Baraud, Y. (2002). Non-asymptotic rates of testing in signal detection. Bernoulli 8, 5, 577-606.
6. Baraud, Y., Giraud, C., and Huet, S. (2009). Gaussian model selection with an unknown variance. Ann. Statist. 37, 2, 630-672.
7. Baraud, Y., Huet, S., and Laurent, B. (2003). Adaptive tests of linear hypotheses by model selection. Ann. Statist. 31, 1, 225-251. MR1962505 (2004a:62091)
8. Bickel, P., Ritov, Y., and Tsybakov, A. (2009). Simultaneous analysis of lasso and Dantzig selector. Ann. Statist. 37, 4, 1705-1732. http://dx.doi.org/10.1214/08-AOS620. MR2533469
9. Birgé, L. (2001). An alternative point of view on Lepski's method. In State of the art in probability and statistics (Leiden, 1999). IMS Lecture Notes Monogr. Ser., Vol. 36. Inst. Math. Statist., Beachwood, OH, 113-133. http://dx.doi.org/10.1214/lnms/1215090065. MR1836557 (2002j:62049)
10. Birgé, L. (2005). A new lower bound for multiple hypothesis testing. IEEE Trans. Inform. Theory 51, 4, 1611-1615. MR2241522 (2007b:62024)
11. Birgé, L. and Massart, P. (2001). Gaussian model selection. J. Eur. Math. Soc. (JEMS) 3, 3, 203-268. MR1848946 (2002i:62072)
12. Birgé, L. and Massart, P. (2007).Minimal penalties for Gaussian model selection. Probab. Theory Related Fields 138, 1-2, 33-73. http://dx.doi.org/10.1007/s00440-006-0011-8. MR2288064 (2008g:62070)
13. Candès, E. and Plan, Y. (2009). Near-ideal model selection by ℓ1 minimization. Ann. Statist. 37, 5A, 2145-2177. http://dx.doi.org/10.1214/08-AOS653. MR2543688
14. Candes, E. J. and Tao, T. (2005). Decoding by linear programming. IEEE Trans. Inform. Theory 51, 12, 4203-4215. http://dx.doi.org/10.1109/TIT.2005.858979. MR2243152 (2007b:94313)
15. Candes, E. J. and Tao, T. (2007). The Dantzig selector: statistical estimation when p is much larger than n. Ann. Statist. 35, 6, 2313-2351. MR2382644
16. Chu, J.-H., Weiss, S., Carey, V., and Rabyl, B. (2009). A graphical model approach for inferring large-scale networks integrating gene expression and genetic polymorphism. BMC Systems Biology 3, 55.
17. Cook, R. D. and Li, B. (2002). Dimension reduction for conditional mean in regression. Ann. Statist. 30, 2, 455-474. http://dx.doi.org/10.1214/aos/1021379861. MR1902895 (2003c:62087)
18. Dalalyan, A. and Tsybakov, A. (2008). Aggregation by exponential weighting, sharp oracle inequalities and sparsity. Machine Learning 72, 1-2, 39-61.
19. Davidson, K. R. and Szarek, S. J. (2001). Local operator theory, random matrices and Banach spaces. In Handbook of the ge-ometry of Banach spaces, Vol. I. North-Holland, Amsterdam, 317-366. MR1863696 (2004f:47002a)
20. Donoho, D. and Jin, J. (2004). Higher criticism for detecting sparse heterogeneous mixtures. Ann. Statist. 32, 3, 962-994. http://dx.doi.org/10.1214/009053604000000265. MR2065195 (2005e:62066)
21. Donoho, D. and Johnstone, I. (1994). Minimax risk over lp-balls for lq-error. Probab. Theory Related Fields 99, 2, 277-303. http://dx.doi.org/10.1007/BF01199026. MR1278886 (95g:62019)
22. Donoho, D. and Tanner, J. (2009). Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing. Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 367, 1906, 4273-4293. With electronic supplementary materials available online, http://dx.doi.org/10.1098/rsta.2009.0152. MR2546388 (2010k:62407)
23. Donoho, D. L. (2006). Compressed sensing. IEEE Trans. Inform. Theory 52, 4, 1289-1306. http://dx.doi.org/10.1109/TIT.2006.871582. MR2241189 (2007e:94013)
24. Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. (2004). Least angle regression. Ann. Statist. 32, 2, 407-499. With discussion, and a rejoinder by the authors.MR2060166 (2005d:62116)
25. Fan, J. and Lv, J. (2008). Sure independence screening for ultrahigh dimensional feature space. J. R. Statist. Soc. B 70, 5,.
26. Fukumizu, K., Bach, F., and Jordan, M. I. (2009). Kernel dimension reduction in regression. Ann. Statist. 37, 4, 1871-1905. http://dx.doi.org/10.1214/08-AOS637. MR2533474
27. Giraud, C. (2008). Estimation of Gaussian graphs by model selection. Electron. J. Stat. 2, 542-563.
28. Ingster, Y. (1993a). Asymptotically minimax hypothesis testing for nonparametric alternatives I. Math. Methods Statist. 2, 85-114.
29. Ingster, Y. (1993b). Asymptotically minimax hypothesis testing for nonparametric alternatives II. Math. Methods Statist. 3, 171-189.
30. Ingster, Y. (1993c). Asymptotically minimax hypothesis testing for nonparametric alternatives III. Math. Methods Statist. 4, 249-268.
31. n-balls. Math. Methods Statist. 7, 4, 401-428 (1999). MR1680087 (2000f:62012)
32. Ingster, Y. (2001). Adaptive detection of a signal of growing dimension I. Math. Methods Statist. 10, 4, 395-421.
33. Ingster, Y. (2002). Adaptive detection of a signal of growing dimension II. Math. Methods Statist. 11, 1, 37-68.
34. Ingster, Y., Tsybakov, A., and Verzelen, N. (2010). Detection boundary in sparse regression. Electron. J. Stat. 4, 1476-1526.
35. Johnstone, I. (1994). On minimax estimation of a sparse normal mean vector. Ann. Statist. 22, 1, 271-289. http://dx.doi.org/10.1214/aos/1176325368. MR1272083 (95g:62020)
36. Lounici, K. (2008). Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators. Electron. J. Stat. 2, 90-102. http://dx.doi.org/10.1214/08-EJS177. MR2386087 (2009a:62287)
37. Massart, P. (2007). Concentration inequalities and model selection. Lecture Notes in Mathematics, Vol. 1896. Springer, Berlin. Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, July 6-23, 2003, With a foreword by Jean Picard. MR2319879 (2010a:62008)
38. Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. Ann. Statist. 34, 3, 1436-1462. MR2278363 (2008b:62044)
39. q balls. Tech. rep., UC Berkeley.
40. Rigollet, P. and Tsybakov, A. (2011). Exponential screening and optimal rates of sparse estimation. Ann. Statist. 39, 2, 731-771.
41. Rudelson, M. and Vershynin, R. (2009). Smallest singular value of a random rectangular matrix. Comm. Pure Appl. Math. 62, 12, 1707-1739. http://dx.doi.org/10.1002/cpa.20294. MR2569075
42. Verzelen, N. (2010a). High-dimensional gaussian model selection on a gaussian design. Ann. Inst. H. Poincaré Probab. Statist. 46, 2, 480-524.
43. Verzelen, N. (2010b). Technical Appendix to "Minimax risks for sparse regressions: Ultra-high-dimensional phenomenons."
44. Verzelen, N. and Villers, F. (2010). Goodness-of-fit tests for high-dimensional Gaussian linear models. Ann. Statist. 38, 2, 704-752. MR2604699
45. Wainwright, M. (2009a). Information-theoretic limits on sparsity recovery in the high-dimensional and noisy setting. IEEE Trans. Inform. The-ory 55, 12, 5728-5741. http://dx.doi.org/10.1109/TIT.2009.2032816. MR2597190
46. Wainwright, M. (2009b). Sharp thresholds for high-dimensional and noisy sparsity recovery using ℓ1-constrained quadratic programming (lasso). IEEE Trans. Inform. Theory 55, 5, 2183-2202.
47. Ye, F. and Zhang, C.-H. (2010). Rate minimaxity of the Lasso and Dantzig selector for the ℓq loss in ℓr balls. J. Mach. Learn. Res. 11, 3519-3540. MR2756192
48. Yu, B. (1997). Assouad, Fano, and Le Cam. In Festschrift for Lucien Le Cam. Springer, New York, 423-435. MR1462963 (99c:62137)
49. Zhao, P. and Yu, B. (2006). On model selection consistency of Lasso. J. Mach. Learn. Res. 7, 2541-2563. MR2274449
50. Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B Stat. Methodol. 67, 2, 301-320. MR2137327