Variable selection with error control: Another look at stability selection

Journal of the Royal Statistical Society. Series B: Statistical Methodology

Volumn 75, Issue 1, 2013, Pages 55-80

  Shah, Rajen D ^a   Samworth, Richard J ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

Complementary pairs stability selection; R concavity; Subagging; Subsampling; Variable selection

Indexed keywords

EID: 84871371181     PISSN: 13697412     EISSN: 14679868     Source Type: Journal    
DOI: 10.1111/j.1467-9868.2011.01034.x     Document Type: Article

References (29)

1. Alon, U., Barkai, N., Notterman, D. A., Gish, K., Ybarra, S., Mack, D. and Levine, A. J. (1999) Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proc. Natn. Acad. Sci. USA, 96, 6745-6750.
2. Amaratunga, D. and Cabrera, J. (2004) Exploration and Analysis of DNA Microarray and Protein Array Data. New York: Wiley-Interscience.
3. Bach, F. (2008) Bolasso: model consistent lasso estimation through the bootstrap. In Proc. 25th Int. Conf. Machine Learning, pp. 33-40. New York: Association for Computing Machinery.
4. Biau, G., Cérou, F. and Guyader, A. (2010) On the rate of convergence of the bagged nearest neighbor estimate. J. Mach. Learn. Res., 11, 687-712.
5. Breiman, L. (1996) Bagging predictors. Mach. Learn., 24, 123-140.
6. Breiman, L. (1999) Using adaptive bagging to debias regressions. Technical Report Department of Statistics, University of California, Berkeley.
7. Bühlmann, P. and Yu, B. (2002) Analyzing bagging. Ann. Statist., 30, 927-961.
8. Cule, M., Samworth, R. and Stewart, M. (2010) Maximum likelihood estimation of a multi-dimensional log-concave density (with discussion). J. R. Statist. Soc. B, 72, 545-607.
9. Dharmadhikari, S. and Joag-Dev, K. (1988) Unimodality, Convexity and Applications. Boston: Academic Press.
10. Dudoit, S., Fridlyand, J. and Speed, T. P. (2002) Comparison of discrimination methods for the classification of tumors using gene expression data. J. Am. Statist. Ass., 97, 77-87.
11. Dümbgen, L. and Rufibach, K. (2009) Maximum likelihood estimation of a log-concave density and its distribution function: basic properties and uniform consistency. Bernoulli, 15, 40-68.
12. Dümbgen, L., Samworth, R. J. and Schuhmacher, D. (2012) Stochastic search for semiparametric linear regression models. In From Probability to Statistics and Back: High-dimensional Models and Processes; a Festschrift in Honor of Jon Wellner (eds M. Banerjee and F. Bunea ). Beachwood: Institute of Mathematical Statistics. To be published.
13. Fan, J. and Lv, J. (2010) A selective overview of variable selection in high dimensional feature space. Statist. Sin., 20, 101-148.
14. Fan, J., Samworth, R. and Wu, Y. (2009) Ultrahigh dimensional feature selection: beyond the linear model. J. Mach. Learn. Res., 10, 2013-2038.
15. Friedman, J., Hastie, T. and Tibshirani, R. (2010) Regularization paths for generalized linear models via coordinate descent. J. Statist. Softwr., 33, 1-22.
16. Hall, P. and Samworth, R. J. (2005) Properties of bagged nearest neighbour classifiers. J. R. Statist. Soc. B, 67, 363-379.
17. Han, Y. and Yu, L. (2010) A variance reduction framework for stable feature selection. In Proc. 10th Int. Conf. Data Mining, pp. 206-215. Sydney: Institute of Electrical and Electronics Engineers Computer Society.
18. Kalousis, A., Prados, J. and Hilario, M. (2007) Stability of feature selection algorithms: a study on high-dimensional spaces. Knowl. Inform. Syst., 12, 95-116.
19. Koenker, R. and Mizera, I. (2010) Quasi-concave density estimation. Ann. Statist., 38, 2998-3027.
20. Kuncheva, L. (2007) A stability index for feature selection. In Proc. 25th Int. Multi-conf. Artificial Intelligence and Applications, pp. 390-395. Calgary: ACTA.
21. Lange, T., Braun, M., Roth, V. and Buhmann, J. (2003) Stability-based model selection. In Advances in Neural Information Processing Systems, vol. 15 (eds S. Becker, S. Thrun and K. Obermayer ), pp. 617-624. Cambridge: MIT Press.
22. Loscalzo, S. Yu, L. and Ding, C. (2009) Consensus group based stable feature selection. In Proc. 15th Int. Conf. Knowledge Discovery and Data Mining, pp. 567-576. New York: Association for Computing Machinery.
23. Meinshausen, N. and Bühlmann, P. (2010) Stability selection (with discussion). J. R. Statist. Soc. B, 72, 417-473.
24. R Development Core Team (2010) R: a Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing.
25. Saeys, Y. Abeel, T. and Peer, Y. V. (2008) Robust feature selection using ensemble feature selection techniques. In Proc. Eur. Conf. Machine Learning., pp. 313-325. Berlin: Springer.
26. Samworth, R. J. (2011) Optimal weighted nearest neighbour classifiers. Arxiv Preprint. (Available from http://arxiv.org/pdf/1101.5783
27. Seregin, A. and Wellner, J. A. (2010) Nonparametric estimation of convex-transformed densities. Ann. Statist., 38, 3751-3781.
28. Tibshirani, R. (1996) Regression shrinkage and selection via the lasso. J. R. Statist. Soc. B, 58, 267-288.
29. Walther, G. (2002) Detecting the presence of mixing with multiscale maximum likelihood. J. Am. Statist. Ass., 97, 508-513.