Covariance-regularized regression and classification for high dimensional problems

Journal of the Royal Statistical Society. Series B: Statistical Methodology

Volumn 71, Issue 3, 2009, Pages 615-636

  Witten, Daniela M ^a   Tibshirani, Robert ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

Classification; Covariance regularization; N p; Regression; Variable selection

Indexed keywords

EID: 66849143711     PISSN: 13697412     EISSN: 14679868     Source Type: Journal    
DOI: 10.1111/j.1467-9868.2009.00699.x     Document Type: Article

References (35)

1. Bair, E. Tibshirani, R. (2004) Semi-supervised methods to predict patient survival from gene expression data. PLOS Biol., 2, 511 522.
2. Banerjee, O., El Ghaoui, L. E. d'Aspremont, A. (2008) Model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data. J. Mach. Learn. Res., 9, 485 516.
3. Bickel, P. Levina, E. (2008) Covariance regularization by thresholding. Ann. Statist., to be published.
4. Breiman, L. (2001) Random forests. Mach. Learn., 45, 5 32. (Pubitemid 32933532)
5. Dey, D. Srinivasan, C. (1985) Estimation of a covariance matrix under Stein's loss. Ann. Statist., 13, 1581 1591.
6. Frank, I. Friedman, J. (1993) A statistical view of some chemometrics regression tools (with discussion). Technometrics, 35, 109 148.
7. Friedman, J. (1989) Regularized discriminant analysis. J. Am. Statist. Ass., 84, 165 175.
8. Friedman, J., Hastie, T. Tibshirani, R. (2007) Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9, 432 441.
9. Friedman, J., Hastie, T. Tibshirani, R. (2008) Regularization paths for generalized linear models via coordinate descent. To be published.
10. Green, P. J. (1984) Iteratively reweighted least squares for maximum likelihood estimation, and some robust and resistant alternatives. J. R. Statist. Soc. B, 46, 149 192.
11. Guo, Y., Hastie, T. Tibshirani, R. (2007) Regularized linear discriminant analysis and its application in microarrays. Biostatistics, 8, 86 100. (Pubitemid 44906104)
12. Haff, L. (1979) Estimation of the inverse covariance matrix: random mixtures of the inverse Wishart matrix and the identity. Ann. Statist., 7, 1264 1276.
13. Hinton, G., Osindero, S. Teh, Y. (2006) A fast learning algorithm for deep belief nets. Neur. Computn, 18, 1527 1553.
14. Hoerl, A. E. Kennard, R. (1970) Ridge regression: biased estimation for nonorthogonal problems. Technometrics, 12, 55 67.
15. Hummel, M., Bentink, S., Berger, H., Klappwe, W., Wessendorf, S., Barth, F. T. E., Bernd, H.-W., Cogliatti, S. B., Dierlamm, J., Feller, A. C., Hansmann, M. L., Haralambieva, E., Harder, L., Hasenclever, D., Kuhn, M., Lenze, D., Lichter, P., Martin-Subero, J. I., Moller, P., Muller-Hermelink, H.-K., Ott, G., Parwaresch, R. M., Pott, C., Rosenwald, A., Rosolowski, M., Schwaenen, C., Sturzenhofecker, B., Szczepanowski, M., Trautmann, H., Wacker, H.-H., Spang, R., Loefler, M., Trumper, L., Stein, H. Siebert, R. (2006) A biological definition of Burkitt's lymphoma from transcriptional and genomic profiling. New Engl. J. Med., 354, 2419 2430. (Pubitemid 43855045)
16. James, W. Stein, C. (1961) Estimation with quadratic loss. In Proc. 4th Berkeley Symp. Mathematics and Statistical Probability, vol. 1, pp. 361 379. Berkeley : University of California Press.
17. Kalbfleisch, J. Prentice, R. (1980) The Statistical Analysis of Failure Time Data. New York : Wiley.
18. Liang, F., Mukherjee, S. West, M. (2007) The use of unlabeled data in predictive modeling. Statist. Sci., 22, 189 205.
19. Mardia, K., Kent, J. Bibby, J. (1979) Multivariate Analysis. London : Academic Press.
20. McLachlan, G. J. (1992) Discriminant Analysis and Statistical Pattern Recognition. New York : Wiley.
21. Meinshausen, N. Bühlmann, P. (2006) High dimensional graphs and variable selection with the lasso. Ann. Statist., 34, 1436 1462.
22. Monti, S., Savage, K. J., Kutok, J. L., Feuerhake, F., Kurtin, P., Mihm, M., Wu, B., Pasqualucci, L., Neuberg, D., Aguiar, R. C. T., Dal Cin, P., Ladd, C., Pinkus, G. S., Salles, G., Harris, N. L., Dalla-Favera, R., Habermann, T. M., Aster, J. C., Golub, T. R. Shipp, M. A. (2005) Molecular profiling of diffuse large B-cell lymphoma identifies robust subtypes including one characterized by host inflammatory response. Blood, 105, 1851 1861. (Pubitemid 40731764)
23. O'Neill, T. (1978) Normal discrimination with unclassified observations. J. Am. Statist. Ass., 73, 821 826.
24. 1-regularization path algorithm for generalized linear models. J. R. Statist. Soc. B, 69, 659 677.
25. Ramaswamy, S., Tamayo, P., Rifkin, R., Mukherjee, S., Yeang, C., Angelo, M., Ladd, C., Reich, M., Latulippe, E., Mesirov, J., Poggio, T., Gerald, W., Loda, M., Lander, E. Golub, T. (2001) Multiclass cancer diagnosis using tumor gene expression signature. Proc. Natn. Acad. Sci. USA, 98, 15149 15154. (Pubitemid 34013984)
26. Rosenwald, A., Wright, G., Chan, W. C., Connors, J. M., Campo, E., Fisher, R. I., Gascoyne, R. D., Muller-Hermelink, H. K., Smeland, E. B. Staudt, L. M. (2002) The use of molecular profiling to predict survival after chemotherapy for diffuse large B-cell lymphoma. New Engl. J. Med., 346, 1937 1947. (Pubitemid 34651353)
27. Rothman, A., Levina, E. Zhu, J. (2008) Sparse permutation invariant covariance estimation. Electr. J. Statist., 2, 494 515.
28. Shipp, M. A., Ross, K. N., Tamayo, P., Weng, A. P., Kutok, J. L., Aguiar, R. C., Gaasenbeek, M., Angelo, M., Reich, M., Pinkus, G. S., Ray, T. S., Koval, M. A., Last, K. W., Norton, A., Lister, T. A., Mesirov, J., Neuberg, D. S., Lander, E. S., Aster, J. C. Golub, T. R. (2002) Diffuse large B-cell lymphoma outcome prediction by gene-expression profiling and supervised machine learning. Nat. Med., 8, 68 74. (Pubitemid 34101731)
29. Tibshirani, R. (1996) Regression shrinkage and selection via the lasso. J. R. Statist. Soc. B, 58, 267 288.
30. Tibshirani, R., Hastie, T., Narasimhan, B. Chu, G. (2002) Diagnosis of multiple cancer types by shrunken centroids of gene expression. Proc. Natn. Acad. Sci. USA, 99, 6567 6572. (Pubitemid 34526173)
31. Tibshirani, R., Hastie, T., Narasimhan, B. Chu, G. (2003) Class prediction by nearest shrunken centroids, with applications to DNA microarrays. Statist. Sci., 18, 104 117. (Pubitemid 41461824)
32. Tusher, V. G., Tibshirani, R. Chu, G. (2001) Significance analysis of microarrays applied to the ionizing radiation response. Proc. Natn. Acad. Sci. USA, 98, 5116 5121. (Pubitemid 32397092)
33. Zhao, P. Yu, B. (2006) On model selection consistency of lasso. J. Mach. Learn. Res., 7, 2541 2563. (Pubitemid 44866738)
34. Zhu, J. Hastie, T. (2004) Classification of gene microarrays by penalized logistic regression. Biostatistics, 5, 427 443. (Pubitemid 41180217)
35. Zou, H. Hastie, T. (2005) Regularization and variable selection via the elastic net. J. R. Statist. Soc. B, 67, 301 320. (Pubitemid 40465877)