Least angle regression

Annals of Statistics

Volumn 32, Issue 2, 2004, Pages 407-499

  Efron, Bradley ^a,b   Hastie, Trevor ^a,b   Johnstone, Iain ^a,b   Tibshirani, Robert ^a,b   Ishwaran, Hemant ^c   Knight, Keith ^d   Loubes, Jean Michel ^e,f   Massart, Pascal ^e,g   Madigan, David ^h,i   Ridgeway, Greg ^h,j   Rosset, Saharon ^a,k   Zhu, J I ^a,l   Stine, Robert A ^m,n   Turlach, Berwin A ^o   Weisberg, Sanford ^p  

^a STANFORD UNIVERSITY   (United States)

^b STANFORD LAW SCHOOL   (United States)

^c LERNER RESEARCH INSTITUTE   (United States)

^d UNIVERSITY OF TORONTO   (Canada)

^e UNIVERSITÉ PARIS SACLAY   (France)

^f CNRS   (France)

^g UNIVERSITÉ PARIS SUD   (France)

^h AVAYA LABS RESEARCH   (United States)

ⁱ RUTGERS UNIVERSITY   (United States)

^j RAND CORPORATION   (United States)

^k IBM T J WATSON RESEARCH CENTER   (United States)

^l UNIVERSITY OF MICHIGAN   (United States)

^m UNIVERSITY OF PENNSYLVANIA   (United States)

ⁿ UNIVERSITY OF PENNSYLVANIA   (United States)

^o UNIVERSITY OF WESTERN AUSTRALIA   (Australia)

^p UNIVERSITY OF MINNESOTA   (United States)

more..

Author keywords

Boosting; Coefficient paths; Lasso; Linear regression; Variable selection

Indexed keywords

EID: 3242708140     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/009053604000000067     Document Type: Article

References (71)

1. BREIMAN, L., FRIEDMAN, J., OLSHEN, R. and STONE, C. (1984). Classification and Regression Trees. Wadsworth, Belmont, CA.
2. EFRON, B. (1986). How biased is the apparent error rate of a prediction rule? J. Amer. Statist. Assoc. 81 461-470.
3. EFRON, B. and TIBSHIRANI, R. (1997). Improvements on cross-validation: The.632+ bootstrap method. J. Amer. Statist. Assoc. 92 548-560.
4. FREUND, Y. and SCHAPIRE, R. (1997). A decision-theoretic generalization of online learning and an application to boosting. J. Comput. System Sci. 55 119-139.
5. FRIEDMAN, J. (2001). Greedy function approximation: A gradient boosting machine. Ann. Statist. 29 1189-1232.
6. FRIEDMAN, J., HASTIE, T. and TIBSHIRANI, R. (2000). Additive logistic regression: A statistical view of boosting (with discussion). Ann. Statist. 28 337-407.
7. GOLUB, G. and VAN LOAN, C. (1983). Matrix Computations. Johns Hopkins Univ. Press, Baltimore, MD.
8. HASTIE, T., TIBSHIRANI, R. and FRIEDMAN, J. (2001). The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer, New York.
9. LAWSON, C. and HANSON, R. (1974). Solving Least Squares Problems. Prentice-Hall, Englewood Cliffs, NJ.
10. p. Technometrics 15 661-675.
11. MEYER, M. and WOODROOFE, M. (2000). On the degrees of freedom in shape-restricted regression. Ann. Statist. 28 1083-1104.
12. OSBORNE, M., PRESNELL, B. and TURLACH, B. (2000a). A new approach to variable selection in least squares problems. IMA J. Numer. Anal. 20 389-403.
13. OSBORNE, M. R., PRESNELL, B. and TURLACH, B. (2000b). On the LASSO and its dual. J. Comput. Graph. Statist. 9 319-337.
14. RAO, C. R. (1973). Linear Statistical Inference and Its Applications, 2nd ed. Wiley, New York.
15. STEIN, C. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135-1151.
16. TIBSHIRANI, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B. 58 267-288.
17. WEISBERG, S. (1980). Applied Linear Regression. Wiley, New York.
18. YE, J. (1998). On measuring and correcting the effects of data mining and model selection. J. Amer. Statist. Assoc. 93 120-131.
19. BREIMAN, L. (1992). The little bootstrap and other methods for dimensionality selection in regression: X-fixed prediction error. J. Amer. Statist. Assoc. 87 738-754.
20. GEORGE, E. I. and MCCULLOCH, R. E. (1993). Variable selection via Gibbs sampling. J. Amer. Statist. Assoc. 88 881-889.
21. ISHWARAN, H. and RAO, J. S. (2000). Bayesian nonparametric MCMC for large variable selection problems. Unpublished manuscript.
22. ISHWARAN, H. and RAO, J. S. (2003). Detecting differentially expressed genes in microarrays using Bayesian model selection. J. Amer. Statist. Assoc. 98 438-455.
23. p. Technometrics 15 661-675.
24. MITCHELL, T. J. and BEAUCHAMP, J. J. (1988). Bayesian variable selection in linear regression (with discussion). J. Amer. Statist. Assoc. 83 1023-1036.
25. SHAO, J. (1993). Linear model selection by cross-validation. J. Amer. Statist. Assoc. 88 486-494.
26. BREIMAN, L. (1996). Bagging predictors. Machine Learning 24 123-140.
27. BÜHLMANN, P. and YU, B. (2002). Analyzing bagging. Ann. Statist. 30 927-961.
28. ABRAMOVICH, F., BENJAMINI, Y., DONOHO, D. and JOHNSTONE, I. (2000). Adapting to unknown sparsity by controlling the false discovery rate. Technical Report 2000-19, Dept. Statistics, Stanford Univ.
29. AKAIKE, H. (1973). Maximum likelihood identification of Gaussian autoregressive moving average models. Biometrika 60 255-265.
30. BIRGÉ, L. and MASSART, P. (2001a). Gaussian model selection. J. Eur. Math. Soc. 3 203-268.
31. p criterion for Gaussian model selection. Technical Report 647, Univ. Paris 6 & 7.
32. FOSTER, D. and GEORGE, E. (1994). The risk inflation criterion for multiple regression. Ann. Statist. 22 1947-1975.
33. KNIGHT, K. and FU, B. (2000). Asymptotics for Lasso-type estimators. Ann. Statist. 28 1356-1378.
34. LOUBES, J.-M. and VAN DE GEER, S. (2002). Adaptive estimation with soft thresholding penalties. Statist. Neerlandica 56 453-478.
35. p. Technometrics 15 661-675.
36. VAN DE GEER, S. (2001). Least squares estimation with complexity penalties. Math. Methods Statist. 10 355-374.
37. BREIMAN, L. (2001). Random forests. Available at ftp://ftp.stat.berkeley. edu/pub/users/breiman/randomforest2001.pdf.
38. Fu, W. J. (1998). Penalized regressions: The Bridge versus the Lasso. J. Comput. Graph. Statist. 7 397-416.
39. OSBORNE, M. R., PRESNELL, B. and TURLACH, B. A. (2000). A new approach to variable selection in least squares problems. IMA J. Numer. Anal. 20 389-403.
40. RIDGEWAY, G. (2003). GBM 0.7-2 package manual. Available at http://cran.r-project.org/doc/packages/gbm.pdf.
41. BREIMAN, L. (1999). Prediction games and arcing algorithms. Neural Computation 11 1493-1517.
42. FREUND, Y. and SCHAPIRE, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. System Sci. 55 119-139.
43. FRIEDMAN, J. H. (2001). Greedy function approximation: A gradient boosting machine. Ann. Statist. 29 1189-1232.
44. MASON, L., BAXTER, J., BARTLETT, P. and FREAN, M. (2000). Boosting algorithms as gradient descent. In Advances in Neural Information Processing Systems 12 512-518. MIT Press, Cambridge, MA.
45. ROSSET, S. and ZHU, J. (2004). Piecewise linear regularized solution paths. Advances in Neural Information Processing Systems 16. To appear.
46. ROSSET, S., ZHU, J. and HASTIE, T. (2003). Boosting as a regularized path to a maximum margin classifier. Technical report, Dept. Statistics, Stanford Univ.
47. ZHU, J., ROSSET, S., HASTIE, T. and TIBSHIRANI, R. (2004). 1-norm support vector machines. Neural Information Processing Systems 16. To appear.
48. BENJAMINI, Y. and HOCHBERG, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289-300.
49. BLAKE, C. and MERZ, C. (1998). UCI repository of machine learning databases. Technical report, School Information and Computer Science, Univ. California, Irvine. Available at www.ics.uci.edu/~mlearn/MLRepository.html.
50. DONOHO, D. L. and JOHNSTONE, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika 81 425-455.
51. FOSTER, D. P. and GEORGE, E. I. (1994). The risk inflation criterion for multiple regression. Ann. Statist. 22 1947-1975.
52. FOSTER, D. P. and STINE, R. A. (1996). Variable selection via information theory. Technical Report Discussion Paper 1180, Center for Mathematical Studies in Economics and Management Science, Northwestern Univ.
53. SHAO, J. (1993). Linear model selection by cross-validation. J. Amer. Statist. Assoc. 88 486-494.
54. BREIMAN, L. (1995). Better subset regression using the nonnegative garrote. Technometrics 37 373-384.
55. MCCULLAGH, P. and NELDER, J. A. (1989). Generalized Linear Models, 2nd ed. Chapman and Hall, London.
56. MOORE, D. S. and MCCABE, G. P. (1999). Introduction to the Practice of Statistics, 3rd ed. Freeman, New York.
57. NELDER, J. A. (1977). A reformulation of linear models (with discussion). J. Roy. Statist. Soc. Ser. A 140 48-76.
58. NELDER, J. A. (1994). The statistics of linear models: Back to basics. Statist. Comput. 4 221-234.
59. COOK, R. D. (1998). Regression Graphics. Wiley, New York.
60. COOK, R. D. and WEISBERG, S. (1999a). Applied Regression Including Computing and Graphics. Wiley, New York.
61. COOK, R. D. and WEISBERG, S. (1999b). Graphs in statistical analysis: Is the medium the message? Amer. Statist. 53 29-37.
62. EFRON, B. (2001). Discussion of "Statistical modeling: The two cultures," by L. Breiman. Statist. Sci. 16 218-219.
63. LI, K. C. (1991). Sliced inverse regression for dimension reduction (with discussion). J. Amer. Statist. Assoc. 86 316-342.
64. p to individual cases. Technometrics 23 27-31.
65. WEISBERG, S. (2002). Dimension reduction regression in R. J. Statistical Software 7. (On-line journal available at www.jstatsoft.org. The software is available from cran.r-project.org.)
66. ABRAMOVICH, F., BENJAMINI, Y., DONOHO, D. and JOHNSTONE, I. (2000). Adapting to unknown sparsity by controlling the false discovery rate. Technical Report 2000-19, Dept. Statistics, Stanford Univ.
67. BIRGÉ, L. and MASSART, P. (2001). Gaussian model selection. J. Eur. Math. Soc. 3 203-268.
68. EFRON, B. (2004). The estimation of prediction error: Covariance penalties and cross-validation. J. Amer. Statist. Assoc. To appear.
69. FOSTER, D. and STINE, R. (1997). An information theoretic comparison of model selection criteria. Technical report, Dept. Statistics, Univ. Pennsylvania.
70. GEORGE, E. I. and FOSTER, D. P. (2000). Calibration and empirical Bayes variable selection. Biometrika 87 731-747.
71. LEBLANC, M. and TIBSHIRANI, R. (1998). Monotone shrinkage of trees. J. Comput. Graph. Statist. 7 417-433.