Slope—adaptive variable selection via convex optimization

Annals of Applied Statistics

Volumn 9, Issue 3, 2015, Pages 1103-1140

  Bogdan, Małgorzata ^a   van den Berg, Ewout ^b   Sabatti, Chiara ^c   Su, Weijie ^c   Candès, Emmanuel J ^c  

^a WROCLAW UNIVERSITY OF TECHNOLOGY   (Poland)

^b IBM T J WATSON RESEARCH CENTER   (United States)

^c STANFORD UNIVERSITY   (United States)

Author keywords

False discovery rate; Lasso; Sorted 1 penalized estimation (SLOPE); Sparse regression; Variable selection

Indexed keywords

EID: 84946556176     PISSN: 19326157     EISSN: 19417330     Source Type: Journal    
DOI: 10.1214/15-AOAS842     Document Type: Article

References (55)

1. ABRAMOVICH, F. and BENJAMINI, Y. (1995). Thresholding of wavelet coefficients as multiple hypotheses testing procedure. In Wavelets and Statistics. Lecture Notes in Statistics 103 5-14. Springer, Berlin.
2. ABRAMOVICH, F., BENJAMINI, Y., DONOHO, D. L. and JOHNSTONE, I. M. (2006). Adapting to unknown sparsity by controlling the false discovery rate. Ann. Statist. 34 584-653. MR2281879
3. AKAIKE, H. (1974). A new look at the statistical model identification. IEEE Trans. Automat. Control AC-19 716-723. System identification and time-series analysis. MR0423716
4. BARLOW, R. E., BARTHOLOMEW, D. J., BREMNER, J. M. and BRUNK, H. D. (1972). Statistical Inference Under Order Restrictions. The Theory and Application of Isotonic Regression. Wiley, New York. MR0326887
5. BAUER, P., PÖTSCHER, B. M. and HACKL, P. (1988). Model selection by multiple test procedures. Statistics 19 39-44. MR0921623
6. BECK, A. and TEBOULLE, M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2 183-202. MR2486527
7. BECKER, S. R., CANDÈS, E. J. and GRANT, M. C. (2011). Templates for convex cone problems with applications to sparse signal recovery. Math. Program. Comput. 3 165-218. MR2833262
8. BENJAMINI, Y. and GAVRILOV, Y. (2009). A simple forward selection procedure based on false discovery rate control. Ann. Appl. Stat. 3 179-198. MR2668704
9. BENJAMINI, Y. and HOCHBERG, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289-300. MR1325392
10. BENJAMINI, Y. and YEKUTIELI, D. (2005). False discovery rate-adjusted multiple confidence intervals for selected parameters. J. Amer. Statist. Assoc. 100 71-93. MR2156820
11. BERK, R., BROWN, L., BUJA, A., ZHANG, K. and ZHAO, L. (2013). Valid post-selection inference. Ann. Statist. 41 802-837. MR3099122
12. BEST, M. J. and CHAKRAVARTI, N. (1990). Active set algorithms for isotonic regression; a unifying framework. Math. Program. 47 425-439. MR1068274
13. BIRGÉ, L. and MASSART, P. (2001). Gaussian model selection. J. Eur. Math. Soc. (JEMS) 3 203- 268. MR1848946
14. BOGDAN, M., CHAKRABARTI, A., FROMMLET, F. and GHOSH, J. K. (2011). Asymptotic Bayesoptimality under sparsity of some multiple testing procedures. Ann. Statist. 39 1551-1579. MR2850212
15. BOGDAN, M., GHOSH, J. K. and ŻAK-SZATKOWSKA, M. (2008). Selecting explanatory variables with the modified version of Bayesian information criterion. Qual. Reliab. Eng. Int. 24 627-641.
16. BOGDAN, M., VAN DEN BERG, E., SABATTI, C., SU, W. and CANDÈS, E. J. (2015). Supplement to "SLOPE-Adaptive variable selection via convex optimization." DOI:10.1214/15- AOAS842SUPP.
17. 1 norm. Preprint. Available at arXiv:1310.1969v2.
18. BONDELL, H. D. and REICH, B. J. (2008). Simultaneous regression shrinkage, variable selection, and supervised clustering of predictors with OSCAR. Biometrics 64 115-123, 322-323. MR2422825
19. BÜHLMANN, P. (2013). Statistical significance in high-dimensional linear models. Bernoulli 19 1212-1242. MR3102549
20. CANDES, E. and TAO, T. (2007). The Dantzig selector: Statistical estimation when p is much larger than n. Ann. Statist. 35 2313-2351. MR2382644
21. CANDÈS, E. J., WAKIN, M. B. and BOYD, S. P. (2008). Enhancing sparsity by reweighted l1 minimization. J. Fourier Anal. Appl. 14 877-905. MR2461611
22. DE LEEUW, J.,HORNIK, K. andMAIR, P. (2009). Isotone optimization in R: Pool-adjacent-violators algorithm (PAVA) and active set methods. J. Stat. Softw. 32 1-24.
23. EFRON, B. (2011). Tweedie's formula and selection bias. J. Amer. Statist. Assoc. 106 1602-1614. MR2896860
24. FOSTER, D. P. and GEORGE, E. I. (1994). The risk inflation criterion for multiple regression. Ann. Statist. 22 1947-1975. MR1329177
25. FOSTER, D. P. and STINE, R. A. (1999). Local asymptotic coding and the minimum description length. IEEE Trans. Inform. Theory 45 1289-1293. MR1686271
26. FOYGEL-BARBER, R. andCANDÈS, E. J. (2014). Controlling the false discovery rate via knockoffs. Ann. Statist. To appear. Available at arXiv:1404.5609.
27. FROMMLET, F. andBOGDAN, M. (2013). Some optimality properties of FDR controlling rules under sparsity. Electron. J. Stat. 7 1328-1368. MR3063610
28. FROMMLET, F., RUHALTINGER, F., TWARÓG, P. and BOGDAN, M. (2012). Modified versions of Bayesian information criterion for genome-wide association studies. Comput. Statist. Data Anal. 56 1038-1051. MR2897552
29. GRAZIER G'SELL, M., HASTIE, T. and TIBSHIRANI, R. (2013). False variable selection rates in regression. Preprint. Available at arXiv:1302.2303.
30. GROTZINGER, S. J. andWITZGALL, C. (1984). Projections onto order simplexes. Appl. Math. Optim. 12 247-270. MR0768632
31. INGSTER, YU. I. (1998). Minimax detection of a signal for ln-balls. Math. Methods Statist. 7 401- 428. MR1680087
32. JAVANMARD, A. and MONTANARI, A. (2014a). Confidence intervals and hypothesis testing for high-dimensional regression. J. Mach. Learn. Res. 15 2869-2909. MR3277152
33. JAVANMARD, A. andMONTANARI, A. (2014b). Hypothesis testing in high-dimensional regression under the Gaussian random design model: Asymptotic theory. IEEE Trans. Inform. Theory 60 6522-6554. MR3265038
34. KRUSKAL, J. B. (1964). Nonmetric multidimensional scaling: A numerical method. Psychometrika 29 115-129. MR0169713
35. LOCKHART, R., TAYLOR, J., TIBSHIRANI, R. J. and TIBSHIRANI, R. (2014). A significance test for the Lasso. Ann. Statist. 42 413-468. MR3210970
36. MALLOWS, C. L. (1973). Some comments on cp. Technometrics 15 661-676.
37. MEINSHAUSEN, N. (2007). Relaxed Lasso. Comput. Statist. Data Anal. 52 374-393. MR2409990
38. MEINSHAUSEN, N. and BÜHLMANN, P. (2010). Stability selection. J. R. Stat. Soc. Ser. B. Stat. Methodol. 72 417-473. MR2758523
39. MEINSHAUSEN, N.,MEIER, L. and BÜHLMANN, P. (2009). p-values for high-dimensional regression. J. Amer. Statist. Assoc. 104 1671-1681. MR2750584
40. NESTEROV, Y. (2004). Introductory Lectures on Convex Optimization. A Basic Course. Kluwer Academic, Boston, MA. MR2142598
41. NESTEROV, Y. (2007). Gradient methods for minimizing composite objective function. CORE discussion paper. Center for Operations Research and Econometrics (CORE), Université Catholique de Louvain. Available at http://www.ecore.be/DPs/dp_1191313936.pdf.
42. PARIKH, N. and BOYD, S. (2013). Proximal algorithms. In Foundations and Trends in Optimization 1 123-231.
43. SARKAR, S. K. (2002). Some results on false discovery rate in stepwise multiple testing procedures. Ann. Statist. 30 239-257. MR1892663
44. SERVICE, S. K., TESLOVICH, T. M., FUCHSBERGER, C., RAMENSKY, V., YAJNIK, P., KOBOLDT, D. C., LARSON, D. E., ZHANG, Q., LIN, L., WELCH, R., DING, L., MCLELLAN, M. D., O'LAUGHLIN, M., FRONICK, C., FULTON, L. L.,MAGRINI, V., SWIFT, A., ELLIOTT, P., JARVELIN, M. R., KAAKINEN, M.,MCCARTHY, M. I., PELTONEN, L., POUTA, A., BONNYCASTLE, L. L., COLLINS, F. S., NARISU, N., STRINGHAM, H. M., TUOMILEHTO, J., RIPATTI, S., FULTON, R. S., SABATTI, C., WILSON, R. K., BOEHNKE, M. and FREIMER, N. B. (2014). Re-sequencing expands our understanding of the phenotypic impact of variants at GWAS loci. PLoS Genet. 10 e1004147.
45. 1 -penalization for mixture regression models. TEST 19 209-256. MR2677722
46. SUN, T. and ZHANG, C.-H. (2012). Scaled sparse linear regression. Biometrika 99 879-898. MR2999166
47. TIBSHIRANI, R. (1996). Regression shrinkage and selection via the Lasso. J. Roy. Statist. Soc. Ser. B 58 267-288. MR1379242
48. TIBSHIRANI, R. and KNIGHT, K. (1999). The covariance inflation criterion for adaptive model selection. J. R. Stat. Soc. Ser. B. Stat. Methodol. 61 529-546. MR1707859
49. VAN DE GEER, S., BÜHLMANN, P., RITOV, Y. and DEZEURE, R. (2014). On asymptotically optimal confidence regions and tests for high-dimensional models. Ann. Statist. 42 1166-1202. MR3224285
50. WASSERMAN, L. and ROEDER, K. (2009). High-dimensional variable selection. Ann. Statist. 37 2178-2201. MR2543689
51. WU, Z. and ZHOU, H. H. (2013).Model selection and sharp asymptotic minimaxity. Probab. Theory Related Fields 156 165-191. MR3055256
52. ZENG, X. and FIGUEIREDO, M. (2014). Decreasing weighted sorted l1 regularization. IEEE Signal Process. Lett. 1240-1244.
53. ZHANG, C.-H. and ZHANG, S. S. (2014). Confidence intervals for low dimensional parameters in high dimensional linear models. J. R. Stat. Soc. Ser. B. Stat. Methodol. 76 217-242. MR3153940
54. ZHONG, L. and KWOK, J. (2012). Efficient sparse modeling with automatic feature grouping. IEEE Trans. Neural Netw. Learn. Syst. 1436-1447.
55. ZOU, H. (2006). The adaptive Lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418-1429. MR2279469