Grafting: Fast, incremental feature selection by gradient descent in function space

Journal of Machine Learning Research

Volumn 3, Issue , 2003, Pages 1333-1356

  Perkins, Simon ^a   Lacker, Kevin ^b   Theiler, James ^a  

^a LOS ALAMOS NATIONAL LABORATORY   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Boosting; Feature selection; Functional gradient descent; Loss functions; Margin space

Indexed keywords

BOOSTING; FUNCTIONAL GRADIENT DESCENT; ITERATIVE FASHION; LARGE-SCALE PROBLEM; LEARNING FRAMEWORKS; LOSS FUNCTIONS; MARGIN SPACE; NON-LINEAR MODEL;

FEATURE EXTRACTION; ITERATIVE METHODS;

GRAFTING (CHEMICAL);

EID: 1942418470     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (23)

1. C.L. Blake and C.J. Merz. UCI repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html, 1998. University of California, Irvine, Dept. of Information and Computer Science.
2. B. Boser, I. Guyon, and V. Vapnik. A training algorithm for optimal margin classifiers. In Proc. Fifth Annual Workshop on Computational Learning Theory, pages 144-152, Pittsburgh, ACM, 1992.
3. C. Chang and C. Lin. LIBSVM: A library for support vector machines. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm, 2001.
4. S. Das. Filters, wrappers and a boosting-based hybrid for feature selection. In Proc. ICML. Morgan Kauffmann, 2001.
5. Y. Freund and R.E. Schapire. Experiments with a new boosting algorithm. In Machine Learning: Proc. 13th Int. Conf., pages 148-156. Morgan Kaufmann, 1996.
6. J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: A statistical view of boosting. Annals of Statistics, 28:337-307, 2000.
7. J. Friedman and W. Stuetzle. Projection pursuit regression. Journal of the American Statistical Association, 76:817-823, 1981.
8. G. Fung and O.L. Mangasarian. A feature selection Newton method for support vector machine classification. Technical Report 02-03, Data Mining Institute, Dept. of Computer Sciences, University of Wisconsin: Madison, Sept 2002.
9. F. Girosi, M. Jones, and T. Poggio. Regularization theory and neural networks architectures. Neural Computation, 7:219-269, 1995.
10. T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning. Springer, 2001.
11. A.E. Hoerl and R. Kennard. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12:55-67, 1970.
12. T. Jebara and T. Jaakkola. Feature selection and dualities in maximum entropy discrimination. In Proc. Int. Conf. on Uncertainity in Artificial Intelligence, 2000.
13. K. Kira and L. Rendell. A practical approach to feature selection. In D. Sleeman and P. Edwards, editors, Proc. Int. Conf. on Machine Learning, pages 249-256. Morgan Kaufmann, 1992.
14. R. Kohavi and G.H. John. Wrappers for feature subset selection. Artificial Intelligence, 97:273-324, 1997.
15. S. Mallat and Z. Zhang. Matching pursuit with time-frequency dictionaries. IEEE Transactions on Signal Processing, 41(12):3397-3415, 1993.
16. L. Mason, J. Baxter, P.L. Bartlett, and M. Frean. Functional gradient techniques for combining hypotheses. In A.J. Smola, P.L Bartlett, B. Scölkopf, and D. Schuurmans, editors, Advances in Large Margin Classifiers, pages 221-246. MIT Press, 2000.
17. J.A. Nelder and R. Mead. A simplex method for function minimization. Computer Journal, 7: 308-313, 1965.
18. W.H. Press, S.A. Teukolsky,W.T. Vetterling, and B.P. Flannery. Numerical Recipes in C. Cambridge University Press, 2nd edition, 1992.
19. R.E. Schapire and Y. Singer. Improved boosting algorithms using confidence-rated predictions. Machine Learning, 37(3):297-336, 1999.
20. B. Schölkopf and A. Smola. Learning with Kernels. MIT Press, Cambridge, MA, 2002.
21. A.J. Smola, P.J. Bartlett, B. Schölkopf, and D. Schuurmans, editors. Advances in Large Margin Classifiers. MIT Press, 2000.
22. R. Tibshirani. Regression shrinkage and selection via the lasso. Technical report, Dept. of Statistics, University of Toronto, 1994.
23. J. Weston, S. Mukherjee, O. Chapelle, M. Pontil, T. Poggio, and V. Vapnik. Feature selection for SVMs. Advances in Neural Information Processing Systems, 13, 2000.