Twin Boosting: Improved feature selection and prediction

Statistics and Computing

Volumn 20, Issue 2, 2010, Pages 119-138

  Bühlmann, Peter ^a   Hothorn, Torsten ^b  

^a ETH ZURICH   (Switzerland)

^b UNIVERSITY OF MUNICH   (Germany)

Author keywords

Classification; Gradient descent; High dimensional data; Regression; Regularization

Indexed keywords

EID: 77953318018     PISSN: 09603174     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11222-009-9148-5     Document Type: Article

References (33)

1. Breiman, L.: Arcing classifiers (with discussion). Ann. Stat. 26, 801-849 (1998).
2. Breiman, L.: Prediction games & arcing algorithms. Neural Comput. 11, 1493-1517 (1999).
3. Breiman, L.: Random forests. Mach. Learn. 45, 5-32 (2001).
4. Bühlmann, P.: Boosting for high-dimensional linear models. Ann. Stat. 34, 559-583 (2006).
5. Bühlmann, P., Hothorn, T.: Boosting algorithms: regularization, prediction and model fitting (with discussion). Stat. Sci. 22, 477-505 (2007).
6. Bühlmann, P., Meier, L.: Discussion of "One-step sparse estimates in nonconcave penalized likelihood models" (H. Zou and R. Li, auths.). Ann. Stat. 36, 1534-1541 (2008).
7. 2 loss: regression and classification. J. Am. Stat. Assoc. 98, 324-339 (2003).
8. Bühlmann, P., Yu, B.: Sparse boosting. J. Mach. Learn. Res. 7, 1001-1024 (2006).
9. Conlon, E., Liu, X., Lieb, J., Liu, J.: Integrating regulatory motif discovery and genome-wide expression analysis. Proc. Natl. Acad. Sci. USA 100, 3339-3344 (2003).
10. Cox, D.: Partial likelihood. Biometrika 62, 269-276 (1975).
11. Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression (with discussion). Ann. Stat. 32, 407-451 (2004).
12. Freund, Y., Schapire, R.: Experiments with a new boosting algorithm. In: Proceedings of the Thirteenth International Conference on Machine Learning. Morgan Kaufmann, San Francisco (1996).
13. Freund, Y., Schapire, R.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 119-139 (1997).
14. Friedman, J.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 29, 1189-1232 (2001).
15. Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: a statistical view of boosting (with discussion). Ann. Stat. 28, 337-407 (2000).
16. Guyon, I., Gunn, S., Nikravesh, M., Zadeh, L.: Feature Extraction, Foundations and Applications, Studies in Fuzziness and Soft Computing. Springer, Heidelberg (2006).
17. Hand, D.: Classifier technology and the illusion of progress (with discussion). Stat. Sci. 21, 1-34 (2006).
18. Huang, J., Ma, S., Zhang, C.-H.: Adaptive Lasso for sparse high-dimensional regression models. Stat. Sin. 18, 1603-1618 (2008).
19. Jamain, A., Hand, D.: The naive Bayes mystery. Pattern Recogn. Lett. 26, 1752-1760 (2005).
20. Lutz, R.: Logitboost with trees applied to the WCCI 2006 performance prediction challenge datasets. In: Proceedings of the IJCNN 2006.
21. Lutz, R. W., Bühlmann, P.: Conjugate direction boosting. J. Comput. Graph. Stat. 15, 287-311 (2006).
22. Meinshausen, N.: Relaxed Lasso. Comput. Stat. Data Anal. 52, 374-393 (2007).
23. Meinshausen, N., Bühlmann, P.: High-dimensional graphs and variable selection with the Lasso. Ann. Stat. 34, 1436-1462 (2006).
24. Meir, R., Rätsch, G.: An introduction to boosting and leveraging. In: Mendelson, S., Smola, A. (eds.) Advanced Lectures on Machine Learning. Lecture Notes in Computer Science. Springer, Berlin (2003).
25. Rätsch, G., Onoda, T., Müller, K.: Soft margins for AdaBoost. Mach. Learn. 42, 287-320 (2001).
26. Ridgeway, G.: The state of boosting. Comput. Sci. Stat. 31, 172-181 (1999).
27. Schapire, R.: The boosting approach to machine learning: an overview. In: Denison, D., Hansen, M., Holmes, C., Mallick, B., Yu, B. (eds.) MSRI Workshop on Nonlinear Estimation and Classification. Springer, Berlin (2002).
28. Tibshirani, R.: Regression shrinkage and selection via the Lasso. J. R. Stat. Soc., Ser. B 58, 267-288 (1996).
29. Tutz, G., Binder, H.: Generalized additive modeling with implicit variable selection by likelihood-based boosting. Biometrics 62, 961-971 (2006).
30. Tutz, G., Reithinger, F.: A boosting approach to flexible semiparametric mixed models. Stat. Med. 26, 2872-2900 (2007).
31. Zhao, P., Yu, B.: On model selection consistency of Lasso. J. Mach. Learn. Res. 7, 2541-2563 (2006).
32. Zou, H.: The adaptive Lasso and its oracle properties. J. Am. Stat. Assoc. 101, 1418-1429 (2006).
33. Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soc., Ser. B 67, 301-320 (2005).