Feature Selection for Ensemble Learning and Its Application

Machine Learning in Bioinformatics

Volumn , Issue , 2008, Pages 135-155

  Li, Guo Zheng ^a   Yang, Jack Y ^b  

^a SHANGHAI UNIVERSITY   (China)

^b HARVARD UNIVERSITY   (United States)

Author keywords

Ensemble learning such as bagging and boosting for single learning machines; Feature selection for ensemble learning and its application; Multitask learning

Indexed keywords

EID: 70350739430     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9780470397428.ch6     Document Type: Chapter

References (44)

1. Dietterich, T. Machine-learning research: four current directions. The AI Magazine, 18(4): 97-136, 1998.
2. Breiman, L. Bagging predictors. Machine Learning, 24(2): 123-140, 1996.
3. Bauer, E. and Kohavi, R. An empirical comparison of voting classification algorithms: bagging, boosting, and variants. Machine Learning, 36(2): 105-139, 1999.
4. Brown, G.,Wyatt, J. L., and Tino, P. Managing diversity in regression ensembles. Journal of Machine Learning Research, 3: 1621-1650, 2005.
5. Guyon, I. and Elisseeff, A. An introduction to variable and feature selection. Journal of Machine Learning Research, 3: 1157-1182, 2003.
6. Yu, L. and Liu, H. Efficient feature selection via analysis of relevance and redundancy. Journal of Machine Learning Research, 5(Oct): 1205-24, 2004.
7. Liu, H. and Yu, L. Toward integrating feature selection algorithms for classification and clustering. IEEE Transactions on Knowledge and Data Engineering, 17(3): 1-12, 2005.
8. Lal, T. N., Chapelle, O.,Weston, J., and Elisseeff, A. Embedded methods. In: I. Guyon, S. Gunn, and M. Nikravesh, Eds, Feature Extraction, Foundations and Applications, Springer, Physica-Verlag, 2006.
9. Ho, T. The random subspace method for construction decision forests. IEEE Transaction Pattern Analysis and Machine Intelligence, 20(8): 832-844, 1998.
10. Simon, G. and Horst, B. Feature selection algorithms for the generation of multiple classifier systems and their application to handwritten word recognition. Pattern Recognition Letters, 25(1): 1323-1336, 2004.
11. Optiz, D., Feature selection for ensembles. Proceedings of the 16th National Conference on Artificial Intelligence (AAAI), pp. 379-384, 1999.
12. Oliveira, L., Sabourin, R., Bortolozzi, F., and Suen, C. Feature selection using multiobjective genetic algorithms for handwritten digit recognition. The 16th International Conference on Pattern Recognition, pp. 568-571, 2002.
13. Brylla, R. Gutierrez-Osunab, R., and Queka, F. Attribute bagging: improving accuracy of classifier ensembles by using random feature subsets. Pattern Recognition, 36(6): 1291-1302, 2003.
14. Tsymbal, A. and Cunningham, P. Search strategies for ensemble feature selection in medical diagnostics. Proceedings of the 16th IEEE Symposium on Computer-Based Medical Systems (CBMS), pp. 124-129, 2003.
15. Tsymbal, A., Pechenizkiy, M., and Cunningham, P. Sequential genetic search for ensemble feature selection. Proceedings of the International Joint Conference on Artificial Intelligence 2005 (IJCAI2005), Edinburgh, Scotland, pp. 877-882, 2005.
16. Tahir, M. A. and Smith, J. Improving nearest neighbor classifier using tabu search and ensemble distance metrics. Sixth International Conference on Data Mining, (ICDM 06), IEEE Press, pp. 1086-1090, 2006.
17. Li, G.-Z., Liu, T.-Y., and Cheng, V. S. Classification of brain glioma by using SVM bagging with feature selection. Proceedings of Workshop on Data Mining for Biomedical Application 2006 (BioDM2006). Vol. 3916, Springer, pp. 124-130, 2006.
18. Liu, T.-Y., Li, G.-Z., Liu, Y., Wu, G., and Wang., W. Estimation of the future earthquake situation by using neural networks ensemble. Proceedings of the International Symposium on Neural Networks, Lecture Notes in Computer Science, Vol. 3973, Springer, pp. 1231-1236, 2006.
19. Li, G.-Z. and Liu, T.-Y. Feature selection for bagging of support vector machines. Proceedings of 9th Biennial Pacific Rim International Conference on Artificial Intelligence (PRICAI2006) Vol. 4099, Springer, pp. 271-277, 2006.
20. Zhou, Z.-H., Wu, J.-X., and Tang, W. Ensembling neural networks: many could be better than all. Artificial Intelligence, 137(1-2): 239-263, 2002.
21. Caruana, R., Niculescu-Mizil, A., Crew, G., and Ksikes, A. Ensemble selection from libraries of models. Proceedings of the 21st International Conference on Machine Learning (ICML 2004), ACM Press, pp. 137-144, 2004.
22. Caruana, R., Munson, A., and Niculescu-Mizil, A. Getting the most out of ensemble selection. Sixth International Conference on Data Mining (ICDM'06), IEEE Press, pp. 828-833, 2006.
23. Liu T.-Y. Study on feature selection issues in ensemble learning, Ph.D. Thesis, School of Computer Engineering and Science, Shanghai University, China, 2007.
24. Caruana, R. Multitask learning. Machine Learning, 28(1): 41-75, 1997.
25. Caruana, R. and de Sa, V. R. Benefiting from the variables that variable selection discards. Journal of Machine Learning Research, 3: 1245-1264, 2003.
26. Li, G.-Z., Yang, J. Lu, J. Lu,W.-C., and Chen, N.-Y. On multivariate calibration problems. Proceedings of the International Symposium on Neural Networks, Lecture Notes on Computer Science Vol. 3173, Springer, pp. 389-394, 2004.
27. Li, G.-Z. and Liu, T.-Y. Improving generalization ability of neural networks ensemble with multi-task learning. Journal of Computational Information Systems, 2(4): 1235-1239, 2006.
28. Goldberg, D. E. Genetic Algorithms in Search, Optimization, and Machine Learning, Addison Wesley, Boston, 1998.
29. Yang, J. and Honavar, V. Feature subset selection using a genetic algorithm. IEEE Intelligent Systems, 13: 44-49, 1998.
30. Valentini, G. and Dietterich, T. Bias-variance analysis of support vector machines for the development of SVM-based ensemble methods. Journal of Machine Learning Research, 5 (1): 725-775, 2004.
31. Li, G.,-Z., Yang, J., Liu, G. P., and Xue, L. Feature selection for multi-class problems using support vector machines. Proceedings of 8th Biennial Pacific Rim International Conference on Artificial Intelligence (PRICAI2004), Vol. 3173, Springer, pp. 292-300, 2004.
32. Moody, J. and Utans, J. Principled architecture selection for neural networks: application to corporate bond rating prediction. In: J. E. Moody, S. J. Hanson, and R. P. Lippmann, Eds, Advances in Neural Information Processing Systems, Morgan Kaufmann Publishers, Inc., pp. 683-690, 1992.
33. Blake, C., Keogh, E., and Merz., C. J. UCI repository of machine learning databases. Technical report, Department of Information and Computer Science, University of California, Irvine, CA, 1998. Available at http://www.ics.uci.edu/mlearn/MLRepository. htm.
34. Foresee, F. D. and Hagan, M. T. Gauss-Newton approximation to Bayesian regularization. Proceedings of the 1997 International Joint Conference on Neural Networks, pp. 1930-1935, 1997.
35. Kudo, M. and Sklansky, J. Comparison of algorithms that select features for pattern classifiers. Pattern Recognition, 33(1): 25-41, 2000.
36. Bredel, M. and Pollack, L. F. The p21-ras signal transduction pathway and growth regulation in human high-grade gliomas. Brain Research Reviews, 29: 232-249, 1999.
37. Wang, C., Zhang, J., Liu, A., Sun, B., and Zhao, Y. Surgical treatment of primary midbrain gliomas. Surgical Neurology, 53: 41-51, 2000.
38. Gonzalez, M. A. L. and Sotelo, J. Brain tumors in Mexico: characteristics and prognosis of glioblastoma. Surgical Neurology, 53: 157-162, 2000.
39. Chow, L. K., Gobin, Y. P., Cloughesy, T. F., Sayre, J.W., Villablanca, J. P., and Vinuela, F. Prognostic factors in recurrent glioblastoma multiforme and anaplastic astrocytoma, treated with selective intra-arteral chemotherapy. AJNR American Journal of Neuroradiology, 21: 471-478, 2000.
40. Ye, C.-Z.,Yang, J., Geng, D.-Y., Zhou,Y., and Chen., N.-Y. Fuzzy rules to predict degree of malignancy in brain glioma. Medical and Biological Engineering and Computing, 40: 145-152, 2002.
41. Li, G.-Z., Yang, J., Ye, C.-Z., and Geng, D. Degree prediction of malignancy in brain glioma using support vector machines. Computers in Biology and Medicine, 36(3): 313-325, 2006.
42. Kohavi, R. and George, J. H. Wrappers for feature subset selection. Artificial Intelligence, 97: 273-324, 1997.
43. Arle, J. E., Morriss, C., Wang, Z., Zimmerman, R. A., Phillips, P. G., and Sutton, L. N. Prediction of posterior fossa tumor type in children by means of magnetic resonance image properties, spectroscopy, and neural networks. Journal of Neurosurgery, 86: 755-761, 1997.
44. Yang, J. Y., Li, G.-Z., Liu, L.-X., and Yang, M. Q. Classification of brain glioma by using neural networks ensemble with multi-task learning. Proceedings of the 2007 International Conference on Bioinformatics and Computational Biology (BIOCOMP'07), CSREA Press, Nevada, pp. 515-522, 2007.