Feature selection via sensitivity analysis of SVM probabilistic outputs

Machine Learning

Volumn 70, Issue 1, 2008, Pages 1-20

  Shen, Kai Quan ^a   Ong, Chong Jin ^a   Li, Xiao Ping ^a   Wilder Smith, Einar P V ^b  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^b NATIONAL UNIVERSITY HOSPITAL   (Singapore)

Author keywords

Feature ranking; Feature selection; Sensitivity; Support vector machines

Indexed keywords

APPROXIMATION THEORY; PROBABILISTIC LOGICS; PROBLEM SOLVING; REAL TIME SYSTEMS; SENSITIVITY ANALYSIS; SUPPORT VECTOR MACHINES;

DATASETS; FEATURE RANKING; FEATURE-BASED SENSITIVITY OF POSTERIOR PROBABILITIES (FSPP);

FEATURE EXTRACTION;

EID: 36849082989     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-007-5025-7     Document Type: Article

References (39)

1. Boser, B., Guyon, I., & Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. In Proceedings of the fifth annual workshop on computational learning theory (pp. 144-152). San Mateo: Kaufmann.
2. Bradley, P. S., & Mangasarian, O. L. (1998). Feature selection via concave minimization and support vector machines. In Proceedings of the 15th international conference on machine learning (pp. 82-90). San Francisco: Kaufmann.
3. Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2), 123-140.
4. Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5-32.
5. Chang, C. C., & Lin, C. J. (2001). LIBSVM: a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
6. Chapelle, O., Vapnik, V., Bousquet, O., & Mukherjee, S. (2002). Choosing multiple parameters for support vector machines. Machine Learning, 46, 131-159.
7. Chu, W., Keerthi, S. S., & Ong, C. J. (2003). Bayesian trigonometric support vector classifier. Neural Computation, 15(9), 2227-2254.
8. Chu, W., Keerthi, S. S., & Ong, C. J. (2004). Bayesian support vector regression using a unified loss function. IEEE Transactions on Neural Networks, 15(1), 29-44.
9. Cortes, C., & Vapnik, V. N. (1995). Support vector networks. Machine Learning, 20(3), 273-297.
10. Cristianini, N., & Shawe-Taylor, J. (2000). Introduction to support vector machines. Cambridge: Cambridge University Press.
11. Duan, K. B., & Keerthi, S. S. (2005). Which is the best multiclass SVM method? An empirical study. Lecture Notes in Computer Science, 3541, 278-285.
12. Guyon, I., & Elisseeff, A. (2003). An introduction to variable and feature selection. Journal of Machine Learning Research, 3, 1157-1182.
13. Guyon, I., Weston, J., Barnhill, S., & Vapnik, V. N. (2002). Gene selection for cancer classification using support vector machines. Machine Learning, 46(1-3), 389-422.
14. Guyon, I., et al. (2003). NIPS 2003 feature selection competition. http://www.nipsfsc.ecs.soton.ac.uk/.
15. Guyon, I., Gunn, S., Nikravesh, M., & Zadeh, L. A. (2006a). Feature extraction, foundations and applications. Berlin: Springer.
16. Guyon, I., Gunn, S., Hur, A. B., & Dror, G. (2006b). Feature selection benchmark for classification problems. In I. Guyon, S. Gunn, M. Nikravesh, & L. A. Zadeh (Eds.), Feature extraction, foundations and applications. Berlin: Springer.
17. Günter, S., & Bunke, H. (2004). Feature selection algorithms for the generation of multiple classifier systems and their application to handwritten word recognition. Pattern Recognition Letters, 25(11), 1323-1336.
18. Hastie, T., & Tibshirani, R. (1998). Classification by pairwise coupling. The Annals of Statistics, 26(2), 451-471.
19. Joachims, T. (1999). Making large-scale SVM learning practical. In B. Schölkopf, C. Burges, & A. Smola (Eds.), Advances in kernel methods: support vector learning. Cambridge: MIT Press.
20. Keerthi, S. S. (2002). Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms. IEEE Transactions on Neural Networks, 13(5), 1225-1229.
21. Kohavi, R., & John, G. (1997). Wrappers for feature subset selection. Artificial Intelligence, 97, 273-324.
22. Lal, T. N., Chapelle, O., Weston, J., & Elisseeff, A. (2006). Embedded methods. In I. Guyon, S. Gunn, M. Nikravesh, & L. A. Zadeh (Eds.), Feature extraction, foundations and applications. Berlin: Springer.
23. Lee, J. H., & Lin, C. J. (2000). Automatic model selection for support vector machines (Technical Report). Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan. Online at http://www.csie.ntu.edu.tw/cjlin/papers/modelselect.ps.gz.
24. Lin, H. T., Lin, C. J., & Weng, R. C. (2003). A note on Platt's probabilistic outputs for support vector machines (Technical Report). Department of Computer Science, National Taiwan University. http://www.csie.ntu.edu.tw/ ~cjlin/papers/plattprob.ps.
25. Mika, S., Rätsch, G., Weston, J., Schölkopf, B., & Müller, K.-R. (1999). Fisher discriminant analysis with kernels. In Y.-H. Hu, J. Larsen, E. Wilson, & S. Douglas (Eds.), Neural networks for signal processing IX (pp. 41-48). New York: IEEE.
26. Neumann, J., Schnörr, C., & Steidl, G. (2005). Combined SVM-based feature selection and classification. Machine Learning, 61(1-3), 129-150.
27. Newman, D. J., Hettich, S., Blake, C. L., & Merz, C. J. (1998). UCI repository of machine learning databases. Irvine, CA: University of California, Department of Information and Computer Science. http://www.ics.uci.edu/~mlearn/ MLRepository.html.
28. Page, E. S. (1967). A note on generating random permutations. Applied Statistics, 16(3), 273-274.
29. Platt, J. (1999). Fast training of support vector machines using sequential minimal optimization. In B. Schölkopf, C. Burges, & A. Smola (Eds.), Advances in kernel methods: support vector learning. Cambridge: MIT Press.
30. Platt, J. (2000). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In A. Smola, P. Bartlett, B. Scholkopf, & D. Schuurmans (Eds.), Advances in large margin classifiers. Cambridge: MIT Press.
31. Rakotomamonjy, A. (2003). Variable selection using SVM-based criteria. Journal of Machine Learning Research, 3, 1357-1370.
32. Rätsch, G. (2005). Benchmark repository. http://ida.first.fhg.de/ projects/bench/benchmarks.htm.
33. Rätsch, G., Onoda, T., & Müller, K.-R. (2001). Soft margins for AdaBoost. Machine Learning, 42(3), 287-320.
34. Saon, G., & Padmanabhan, M. (2001). Minimum Bayes error feature selection for continuous speech recognition. In T. Leen, T. Dietterich, & V. Tresp (Eds.), Advances in neural information processing systems (Vol. 13, pp. 800-806).
35. Vapnik, V. N. (1995). The nature of statistical learning theory. New York: Springer.
36. Vapnik, V. N. (1998). Statistical learning theory. New York: Wiley.
37. Vapnik, V. N., & Chapelle, O. (2000). Bounds on error expectation for support vector machines. Neural Computation, 12(9), 2013-2036.
38. Weston, J., Mukherjee, S., Chapelle, O., Pontil, M., Poggio, T., & Vapnik, V. N. (2001). Feature selection for SVMs. In T. Leen, T. Dietterich, & V. Tresp (Eds.), Advances in neural information processing systems (Vol. 13, pp. 668-674).
39. Williams, C. K. I., & Rasmussen, C. E. (1996). Gaussian processes for regression. In D.S. Touretzky, M.C. Mozer, & M.E. Hasselmo (Eds.), Advances in neural information processing systems (Vol. 8, pp. 598-604).