Rodeo: Sparse, greedy nonparametric regression

Annals of Statistics

Volumn 36, Issue 1, 2008, Pages 28-63

  Lafferty, John ^a,b   Wasserman, Larry ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b Carnegie Mellon University   (United States)

Author keywords

Bandwidth estimation; Local linear smoothing; Minimax rates of convergence; Nonparametric regression; Sparsity; Variable selection

Indexed keywords

EID: 50649123582     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/009053607000000811     Document Type: Article

References (31)

1. BREIMAN, L., FRIEDMAN, J. H., OLSHEN. R. A. and STONE, C. J. (1984). Classification and Regression Trees. Wadsworth, Belmont, CA. MR0726392
2. BÜP. and Yu, B. (2006). Sparse boosting. J. Mach. Learn. Res. 7 1001-1024.
3. 1-norm near-solution approximates the sparest near-solution. Comm. Pure Appl. Math. 59 797-829.
4. EFRON, B., HASTIE, T., JOHNSTONE, I. and TIBSHIRANI, R. (2004). Least angle regression (with discussion). Ann. Statist. 32 407-499. MR2060166
5. FAN, J. (1992). Design-adaptive nonparametric regression. J. Amer. Statist. Assoc. 87 998-1004. MR1209561
6. FAN, J. and LI, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348-1360. MR1946581
7. FAN, J. and PENG, H. (2004). Nonconcave penalized likelihood with a diverging number of parameters. Ann. Statist. 32 928-961. MR2065194
8. FRIEDMAN, J. H. (1991). Multivariate adaptive regression splines (with discussion). Ann. Statist. 19 1-141. MR1091842
9. FU, W. and KNIGHT, K. (2000). Asymptotics for lasso type estimators. Ann. Statist. 28 1356-1378. MR 1805787
10. GEORGE, E. I. and MCCULLOCH, R. E. (1997). Approaches for Bayesian variable selection. Statist. Sinica 7 339-373.
11. GIROSI, F. (1997). An equivalence between sparse approximation and support vector machines. Neural Comput. 10 1455-1480.
12. GYÖRFI, L., KOHLER, M., KRZYZAK, A. and WALK. H. (2002). A Distribution-Free Theory of Nonparametric Regression. Springer, Berlin.
13. HASTIE, T. and LOADER, C. (1993). Local regression: Automatic kernel carpentry. Statist. Sci. 8 120-129.
14. HASTIE, T., TIBSHIRANI, R. and FRIEDMAN, J. H . (2001). The Elements of Statistical Learning. Data Mining, Inference, and Prediction. Springer, Berlin. MR 1851606
15. HRISTACHE, M., JUDITSKY, A., POLZEHL, J. and SPOKOINY, V. (2001). Structure adaptive approach for dimension reduction. Ann. Statist. 29 1537-1566. MR 1891738
16. KERKYACHARIAN, K., LEPSKI, O. and PICARD, D. (2001). Nonlinear estimation in anisotropic multi-index denoising. Probab. Theory Related Fields 121 137-170. MR1863916
17. LAWRENCE, N. D., SEEGER, M. and HERBRICH, R. (2003). Fast sparse Gaussian process methods: The informative vector machine. In Advances in Neural Information Processing Systems 15 625-632.
18. LEPSKI, O. V., MAMMEN, E. and SPOKOINY, V. G. (1997). Optimal spatial adaptation to inhomogeneous smoothness: An approach based on kernel estimates with variable band-width selectors. Ann. Statist. 25 929-947. MR 1447734
19. LI, L., COOK, R. D. and NACHSTEIM, C. (2005). Model-free variable selection. J. Roy. Statist. Soc. Ser: B 67 285-299. MR2137326
20. RICE, J. (1984). Bandwidth choice for nonparametric regression. Ann. Statist. 12 1215-1230. MR0760684
21. RUPPERT, D. (1997). Empirical-bias bandwidths for local polynomial nonparametric regression and density estimation. J. Amer: Statist. Assoc. 92 1049-1062. MR1482136
22. RUPPERT, D. and WAND, M. P. (1994). Multivariate locally weighted least squares regression. Ann. Statist. 22 1346-1370. MR1311979
23. SAMAROV, A., SPOKOINY, V. and VIAL, C. (2005). Component identification and estimation in nonlinear high-dimensional regression models by structural adaptation. J. Amer: Statist. Assoc. 100 429-445. MR2160548
24. SMOLA, A. and BARTLETT, P. (2001). Sparse greedy Gaussian process regression. In Advances in Neural Information Processing Systems 13 619-625. MR1820960
25. STONE, C. J., HANSEN, M. H., KOOPERBERG, C. and TRUONG, Y. K. (1997). Polynomial splines and their tensor products in extended linear modeling (with discussion). Ann. Statist. 25 1371-1470. MR 1463561
26. TIBSHIRANI, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. Ser. B Stat. Methodol. 58 267-288. MR 1379242
27. TIPPING, M. (2001). Sparse Bayesian learning and the relevance vector machine. J. Machine Learning Research 1 211-244. MR1875838
28. TROPP, J. A. (2004). Greed is good: Algorithmic results for sparse approximation. IEEE Trans. Inform. Theory 50 2231-2241. MR2097044
29. TROPP, J. A. (2006). Just relax: Convex programming methods for identifying sparse signals. IEEE Trans. Inform. Theory 51 1030-1051. MR2238069
30. TURLACH, B. (2004). Discussion of "Least angle regression" by Efron, Hastie, Jonstone and Tibshirani. Ann. Statist. 32 494-499. MR2060166
31. ZHANG, H., WAHBA. G., LIN, Y., VOELKER, M., FERRIS, R. K. and KLEIN, B. (2005). Variable selection and model building via likelihood basis pursuit. J. Amer. Statist. Assoc. 99 659-672. MR2090901