Evolutionary multiobjective ensemble learning based on Bayesian feature selection

2006 IEEE Congress on Evolutionary Computation, CEC 2006

Volumn , Issue , 2006, Pages 267-274

  Chen, Huanhuan ^b   Yao, Xin ^a,b  

^a IEEE   (United Kingdom)

^b UNIVERSITY OF BIRMINGHAM   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATIC RELEVANCE DETERMINATION (ARD); CLASSIFICATION PROBLEMS; MULTIOBJECTIVE ALGORITHMS;

BAYESIAN NETWORKS; ERROR ANALYSIS; LEARNING SYSTEMS; PARAMETER ESTIMATION; PROBLEM SOLVING;

EVOLUTIONARY ALGORITHMS;

EID: 34547287417     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (42)

1. L. K. Hansen and P. Salamon. Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(10):993-1001, 1990.
2. F. J. Huang, T. Chen, Z. Zhou, and H. Zhang. Pose invariant face recognition. In Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000, pages 245-250, Washington, DC, USA, 2000.
3. L. K. Hansen, L. Liisberg, and P. Salamon. Ensemble methods for handwritten digit recognition. In Proceedings of the IEEE Workshop on Neural Networks for Signal Processing, pages 333-342, Helsingoer, Denmark, 1992.
4. K. J. Cherkauer. Human expert level performance on a scientific image analysis task by a system using combined artificial neural networks. In Proceedings of AAAI-96 Workshop on Integrating Multiple Learned Models for Improving and Scaling Machine Learning Algorithms, pages 15-21, Menlo Park, CA, USA, 1996.
5. M. I. Jordan and R. A. Jacobs. Hierarchical mixtures of experts and the em algorithm. Neural Computation, 6(2): 181-214, 1994.
6. L. Breiman. Bagging predictors. Machine Learning, 24(2): 123-140, 1996.
7. R. E. Schapire. A brief introduction to boosting. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, pages 1401-1406, 1999.
8. A. Krogh and J. Vedelsby. Neural network ensembles, cross validation, and active learning. In Advances in Neural Information Processing Systems 7, pages 231-238, Denver, Colorado, USA, 1995.
9. S. Hashem. Optimal linear combinations of neural networks. Neural Networks, 10(4):599-614, 1997.
10. T. K. Ho. The random subspace method for constructing decision forests. IEEE Transaction on Pattern Analysis and Machine Intelligence, 20(8):832-844, .1998.
11. D. W. Optiz. Feature selection for ensembles. In Proceedings of the 16th International Conference on Artificial Intelligence, pages 379-384, 1999.
12. L. S. Oliveira, M. Morita, R. Sabourin, and F. Bortolozzi. Multiobjective genetic algorithms to create ensemble of classifiers. In Proceedings of the Third International Conference on Evolutionary Multi-Criterion Optimization, volume 87, pages 592-606, 2005.
13. J. O'Sullivan, J. Langford, R. Caruana, and A. Blum. Featureboost: A meta-leaming algorithm, that improves model robustness. In Proceedings of the Seventeenth International Conference on Machine. Learning, pages 703-710, San Francisco, CA, USA, 2000.
14. B. Efron and R. J. Tibshirani. An Introduction to the Bootstrap. Chapman & Hall, London, U.K., 1993.
15. J. H. Friedman. Bias, variance, 0-1 loss and the curse of dimensionality. Technical report, Stanford University, 1996.
16. G. Rätsch, T. Onoda, and K. R. Müller. Soft margins for AdaBoost. Machine Learning, 42(3):287-320, 2001.
17. T. G. Dietterich. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning, 40(2): 139-157, 2003.
18. Y. Liao and J. Moody. Constructing heterogeneous committees using input feature grouping: Application to economic forecasting. In Advances in Neural Information Processing Systems, pages 921-927, 1999.
19. H. Mamitsuka. Empirical evaluation of ensemble feature subset selection methods for learning from, a high-dimensional database in drug design. In Proceedings of Third IEEE Symposium on Bioinformatics and BioEngineering, pages 253-257, 2003.
20. A. Tsymbal, P. Cunningham, M. Pechenizkiy, and S. Puuronen. Search strategies for ensemble feature selection in medical diagnostics. In Proceedings of 16th IEEE Symposium on Computer-Based Medical Systems, pages 124-129. IEEE Computer Society, 2003.
21. Y. Liu and X. Yao. Ensemble learning via negative correlation. Neural Networks, 12(10):1399-1404, 1999.
22. Y. Liu and X. Yao. Simultaneous training of negatively correlated neural networks in an ensemble. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 29(6):716-725, 1999.
23. M. M. Islam, X. Yao, and K. Murase. A constructive algorithm for training cooperative neural network ensembles. IEEE Transaction on Neural Networks, 14(4):820-834, 2003.
24. G. Brown, J. Wyatt, and P. Tino. Managing diversity in regression ensembles. Journal of Machine Learning Research, 6:1621-1650, 2005.
25. A. Chandra and X. Yao. Divace: Diverse and accurate ensemble learning algorithm. In Proceedings of the Fifth International Conference on Intelligent Data Engineering and Automated Learning, volume 3177, pages 6.19-625, 2004.
26. H. A. Abbass. A. memetic pareto evolutionary approach to artificial neural networks. In Proceedings of the fourteenth Australian Joint Conference on Artificial Intelligence, volume 2256, pages 1-12, 2000.
27. N. García, C. Hervás, and D. Ortiz. Cooperative coevolution of artificial neural network ensembles for pattern classification. IEEE Transactions on Evolutionary Computation, 9(3):271-302, 2005.
28. L. S. Oliveira, R. Sabourin, F. Bortolozzi, and C. Y. Suen. A methodology for feature selection using multi-objective genetic algorithms for handwritten digit string recognition. International Journal of Pattern Recognition and Artificial Intelligence, 17(6):903-990, 2003.
29. Y. Liu, X. Yao, and T. Higuchi. Evolutionary ensembles with negative correlation learning. IEEE Transaction on Evolutionary Computation, 4(4):380-387, 2000.
30. W. Finnoff, F. Hergert, and H. G. Zimmermann. Improving model selection by nonconvergent methods. Neural Network, 6(6):771-783, 1993.
31. D. J. C. MacKay. The evidence framework applied to classification networks. Neural Computation, 4(3):720-736, 1992.
32. R. M. Neal. Bayesian learning for neural networks. 118. Springer, New York, 1996.
33. M. E. Tipping. Sparse bayesian learning and the relevance vector machine. Journal of Machine Learning Research, 1(3):211-244, 2001.
34. N. Srinivas and K. Deb. Multiobjective function optimization using nondominated sorting genetic algorithms. Evolutionary Computation, 2(3):221-248, 1995.
35. Z. Zhou, J. Wu, and W. Tang. Ensembling neural networks: many could be better than all. Artificial Intelligence, 137(l-2):239-263, 2002.
36. E. Zitzler, M. Laumanns, L. Thiele, C. M. Fonseca, and V. G. d. Fonseca. Performance assessment of multiobjective optimizers: An analysis and review. IEEE Transactions on Evolutionary Computation, 7(2): 117-132, 2003.
37. K. Deb and D. Kalyanmoy. Multi-Objective Optimization Using Evolutionary Algorithms. John Wiley & Sons, Inc., New York, NY, USA, 2001.
38. P. Darwen and X. Yao. Every niching method has its niche: fitness sharing and implicit sharing compared. In Proceedings of Parallel Problem Solving from Nature (PPSN) IV, volume 1141, pages 398-407, Berlin, Germany, 1996.
39. D. J. Newman, S. Hettich, C. L. Blake, and C. J. Merz. UCI repository of machine learning databases, 1998.
40. R. Kohavi. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the fifteenth International Conference on Artificial Intelligence, pages 1137-1143, Qubec, Canada, 1995.
41. B. Scholkopf and A. J. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge, MA, USA, 2001.
42. S. Geman, E. Bienenstock, and R. Doursat. Neural networks and the bias/variance dilemma. Neural Computation, 4(1): 1-58, 1992.