Gradient boosted feature selection

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn , Issue , 2014, Pages 522-531

  Xu, Zhixiang ^a   Huang, Gao ^b   Weinberger, Kilian Q ^a   Zheng, Alice X ^c  

^a WASHINGTON UNIVERSITY   (United States)

^b TSINGHUA UNIVERSITY   (China)

^c GraphLab Inc   (United States)

Author keywords

feature selection; gradient boosting; large scale

Indexed keywords

EID: 84907030792     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2623330.2623635     Document Type: Conference Paper

References (34)

1. U. Alon, N. Barkai, D. A. Notterman, K. Gish, S. Ybarra, D. Mack, and A. J. Levine. Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proceedings of the National Academy of Sciences, 96(12):6745-6750, 1999.
2. L. Breiman. Classification and regression trees. Chapman & Hall/CRC, 1984.
3. O. Chapelle, P. Shivaswamy, S. Vadrevu, K. Weinberger, Y. Zhang, and B. Tseng. Boosted multi-task learning. Machine learning, 85(1):149-173, 2011.
4. N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, volume 1, pages 886-893. IEEE, 2005.
5. J. Duchi, S. Shalev-Shwartz, Y. Singer, and T. Chandra. Efficient projections onto the l1-ball for learning in high dimensions. In Proceedings of the 25th international conference on Machine learning, pages 272-279. ACM, 2008.
6. J. A. Etzel, V. Gazzola, and C. Keysers. An introduction to anatomical ROI-based fMRI classification analysis. Brain Research, 1282:114-125, 2009.
7. J. Friedman. Greedy function approximation: A gradient boosting machine. The Annals of Statistics, pages 1189-1232, 2001.
8. I. Guyon and A. Elisseeff. An introduction to variable and feature selection. The Journal of Machine Learning Research, 3:1157-1182, 2003.
9. T. Hastie, R. Tibshirani, and J. H. Friedman. The elements of statistical learning. Springer, 2009.
10. J. Huang, T. Zhang, and D. Metaxas. Learning with structured sparsity. The Journal of Machine Learning Research, 12:3371-3412, 2011.
11. S. Lee, H. Lee, P. Abbeel, and A. Y. Ng. Efficient l1 regularized logistic regression. In Proceedings of the National Conference on Artificial Intelligence, volume 21, page 401. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, 2006.
12. Y. Liu, M. Sharma, C. Gaona, J. Breshears, J. Roland, Z. Freudenburg, E. Leuthardt, and K. Q. Weinberger. Decoding ipsilateral finger movements from ecog signals in humans. In Advances in Neural Information Processing Systems, pages 1468-1476, 2010.
13. S. Ma, X. Song, and J. Huang. Supervised group lasso with applications to microarray data analysis. BMC bioinformatics, 8(1):60, 2007.
14. L. Meier, S. Van De Geer, and P. Bühlmann. The group lasso for logistic regression. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70(1):53-71, 2008.
15. F. Pan, T. Converse, D. Ahn, F. Salvetti, and G. Donato. Feature selection for ranking using boosted trees. In Proceedings of the 18th ACM conference on Information and knowledge management, pages 2025-2028. ACM, 2009.
16. M. Y. Park and T. Hastie. L1-regularization path algorithm for generalized linear models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69(4):659-677, 2007.
17. H. Peng, F. Long, and C. Ding. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 27(8):1226-1238, 2005.
18. S. Perkins, K. Lacker, and J. Theiler. Grafting: Fast, incremental feature selection by gradient descent in function space. The Journal of Machine Learning Research, 3:1333-1356, 2003.
19. V. Roth. The generalized lasso. Neural Networks, IEEE Transactions on, 15(1):16-28, 2004.
20. V. Roth and B. Fischer. The group-lasso for generalized linear models: Uniqueness of solutions and efficient algorithms. In Proceedings of the 25th international conference on Machine learning, pages 848-855. ACM, 2008.
21. Y. Saeys, I. Inza, and P. Larrañaga. A review of feature selection techniques in bioinformatics. bioinformatics, 23(19):2507-2517, 2007.
22. B. Schölkopf and A. Smola. Learning with kernels: Support vector machines, regularization, optimization, and beyond. MIT press, 2001.
23. L. Song, A. Smola, A. Gretton, J. Bedo, and K. Borgwardt. Feature selection via dependence maximization. The Journal of Machine Learning Research, 98888:1393-1434, 2012.
24. A. Spira, J. E. Beane, V. Shah, K. Steiling, G. Liu, F. Schembri, S. Gilman, Y.-M. Dumas, P. Calner, P. Sebastiani, et al. Airway epithelial gene expression in the diagnostic evaluation of smokers with suspect lung cancer. Nature medicine, 13(3):361-366, 2007.
25. S. Sra. Fast projections onto l1, q-norm balls for grouped feature selection. In Machine Learning and Knowledge Discovery in Databases, pages 305-317. Springer, 2011.
26. R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), pages 267-288, 1996.
27. E. Tuv, A. Borisov, G. Runger, and K. Torkkola. Feature selection with ensembles, artificial variables, and redundancy elimination. The Journal of Machine Learning Research, 10:1341-1366, 2009.
28. S. Tyree, K. Weinberger, K. Agrawal, and J. Paykin. Parallel boosted regression trees for web search ranking. In WWW, pages 387-396. ACM, 2011.
29. Z. Xu, M. K., M. Chen, and K. Q. Weinberger. Cost-sensitive tree of classifiers. In S. Dasgupta and D. Mcallester, editors, Proceedings of the 30th International Conference on Machine Learning (ICML-13), volume 28, pages 133-141. JMLR Workshop and Conference Proceedings, 2013.
30. Z. Xu, M. Kusner, G. Huang, and K. Q. Weinberger. Anytime representation learning. In Proceedings of the 30th International Conference on Machine Learning (ICML-13), pages 1076-1084, 2013.
31. Z. Xu, K. Weinberger, and O. Chapelle. The greedy miser: Learning under test-time budgets. In ICML, pages 1175-1182, 2012.
32. M. Yamada, W. Jitkrittum, L. Sigal, E. P. Xing, and M. Sugiyama. High-dimensional feature selection by feature-wise non-linear lasso. arXiv preprint arXiv:1202.0515, 2012.
33. T. Zhang. Multi-stage convex relaxation for learning with sparse regularization. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems 21, pages 1929-1936. 2008.
34. Z. Zhao, F. Morstatter, S. Sharma, S. Alelyani, A. Anand, and H. Liu. Advancing feature selection research. ASU Feature Selection Repository, 2010.