Error Correlation and Error Reduction in Ensemble Classifiers

Connection Science

Volumn 8, Issue 3-4, 1996, Pages 385-404

  Tumer, Kagan ^a   Ghosh, Joydeep ^a  

^a The University of Texas at Austin   (United States)

Author keywords

Bootstrapping; Combining; Cross validation; Ensemble classifiers; Error correlation; Error reduction; Resampling

Indexed keywords

EID: 0030365938     PISSN: 09540091     EISSN: None     Source Type: Journal    
DOI: 10.1080/095400996116839     Document Type: Article

References (53)

1. Al-Ghoneim, K. & Vijaya Kumar, B.V.K. (1995) Learning ranks with neural networks. In Applications and Science of Artificial Neural Networks, Proceedings of the SPIE, Vol. 2492, pp. 446-464.
2. Ali, K.M. & Pazzani, M.J. (1995) On the Link between Error Correlation and Error Reduction in Decision Tree Ensembles. Technical Report 95-38, Department of Information and Computer Science, University of California, Irvine, CA.
3. Battiti, R. & Colla, A.M. (1994) Democracy in neural nets: Voting schemes for classification. Neural Networks, 7, 691-709.
4. Baxt, W.G. (1992) Improving the accuracy of an artificial neural network using multiple differently trained networks. Neural Computation, 4, 772-780.
5. Benediktsson, J., Sveinsson, J., Ersoy, O. & Swain, P. (1994) Parallel consensual neural networks with optimally weighted outputs. In Proceedings of the World Congress on Neural Networks. INNS Press, pp. III:129-137.
6. Breiman, L. (1993) Stacked Regression. Technical Report 367, Department of Statistics, University of California, Berkeley, CA.
7. Breiman, L. (1994) Bagging Predictors. Technical Report 421, Department of Statistics, University of California, Berkeley, CA.
8. Drucker, H., Cortes, C., Jackel, L.D., LeCun, Y. & Vapnik, V. (1994) Boosting and other ensemble methods. Neural Computation, 6, 1289-1301.
9. Efron, B. (1982) The Jacknife, the Bootstrap and Other Resampling Plans. Philadelphia, PA: SIAM.
10. Efron, B. (1983) Estimating the error rate of a prediction rule. Journal of the American Statistical Association, 78, 316-333.
11. Friedman, J.H. (1994) An overview of predictive learning and function approximation. In V. Cherkassky, J. Friedman & H. Wechsler (Eds), From Statistics to Neural Networks, Proceedings NATO/ASI Workshop. Berlin: Springer Verlag, pp. 1-55.
12. Ghosh, J. & Tumer, K. (1994) Structural adaptation and generalization in supervised feedforward networks. Journal of Artificial Neural Networks, 1, 431-458.
13. Ghosh, J., Beck, S. & Chu, C. (1992) Evidence combination techniques for robust classification of short-duration oceanic signals. In SPIE Conference on Adaptive and Learning Systems, SPIE Proceedings, Vol. 1706, pp. 266-276.
14. Ghosh, J., Tumer K., Beck, S. & Deuser, L. (1996) Integration of neural classifiers for passive sonar signals. In C. Leondes (Ed.), Control and Dynamic Systems, Vol. 77. New York: Academic Press, pp. 301-338.
15. Hansen, L.K. & Salamon, P. (1990) Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 993-1000.
16. Hashem, S. & Schmeiser, B. (1993) Approximating a function and its derivatives using MSE-optimal linear combinations of trained feedforward neural networks. In Proceedings of the Joint Conference on Neural Networks, New Jersey, Vol. 87, pp. I:617-620.
17. Haykin, S. (1994) Neural Networks' A Comprehensive Foundation. New York: Macmillan.
18. Ho, T.K., Hull, J.J. & Srihari, S.N. (1994) Decision combination in multiple classifier systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16, 66-76.
19. Jacobs, R. (1995) Method for combining experts' probability assessments. Neural Computation, 7, 867-888.
20. Jacobs, R.A., Jordan, M.I., Nowlan, S.J. & Hinton, G.E. (1991) Adaptive mixtures of local experts. Neural Computation, 3, 78-88.
21. Jain, A., Dubes, R. & Chen, C. (1987) Bootstrap techniques for error estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9, 628-633.
22. Krogh, A. & Vedelsby, J. (1995) Neural network ensembles, cross validation and active learning. In G. Tesauro, D. S. Touretzky & T. K. Leen (Eds), Advances in Neural Information Processing Systems-7. Cambridge, MA: MIT Press, pp. 231-238.
23. Lee, J., Hwang, J.-N., Davis, D. & Nelson, A. (1991) Integration of neural networks and decision tree classifiers for automated cytology screening. In Proceedings of the International Joint Conference on Neural Networks, Seattle, pp. I:257-262.
24. Levin, E., Tishby, N. & Solla, S.A. (1990) A statistical approach to learning and generalization in layered neural networks. Proceedings of the IEEE, 78, 1568-1574.
25. Lincoln, W. & Skrzypek, J. (1990) Synergy of clustering multiple back propagation networks. In D. Touretzky (Ed.), Advances in Neural Information Processing Systems-2. San Mateo, CA: Morgan Kaufmann, pp. 650-657.
26. Lippmann, R.P. (1995) Personal communication.
27. Mangasarian, O.L., Setiono, R. & Wolberg, W.H. (1990) Pattern recognition via linear programming: Theory and application to medical diagnosis. In T. F. Coleman & Y. Li (Eds), Large-scale Numerical Optimization. Philadelphia, PA: SIAM, pp. 22-30.
28. Meir, R. (1995) Bias, variance, and the combination of estimators; the case of least linear squares. In G. Tesauro, D. S. Touretzky & T. K. Leen (Eds), Advances in Neural Information Processing Systems-7. Cambridge, MA: MIT Press, pp. 295-302.
29. Moody, J. (1994) Prediction risk and architecture selection for neural networks. In V. Cherkassky, J. Friedman & H. Wechsler (Eds), From Statistics to Neural Networks, Proceedings of NATO/ASI Workshop. Springer Verlag, pp. 143-156.
30. Noordewier, M.O., Towell, G.G. & Shavlik, J.W. (1991) Training knowledge-based neural networks to recognize genes in DNA sequences. In R. Lippmann, J. Moody & D. Touretzky (Eds), Advances in Neural Information Processing Systems-3. San Mateo, CA: Morgan Kaufmann, pp. 530-536.
31. Opitz, D.W. & Shavlik, J.W. (1996) Generating accurate and diverse members of a neural network ensemble. In D. S. Touretzky, M. C. Mozer & M. E. Hasselmo (Eds), Advances in Neural Information Processing Systems-8. Cambridge, MA: MIT Press, pp. 535-541.
32. Perrone, M. & Cooper, L.N. (1993a) Learning from what's been learned: supervised learning in multi-neural network systems. In Proceedings of the World Congress on Neural Networks. Portland, OR: INNS Press, pp. III:354-357.
33. Perrone, M. & Cooper, L.N. (1993b) When networks disagree: Ensemble methods for hybrid neural networks. In R. J. Mammone (Ed.), Neural Networks for Speech and Image Processing, Chapter 10. London: Chapman and Hall.
34. Prechelt, L. (1994) PROBEN1 - A Set of Benchmarks and Benchmarking Rules for Neural Network Training Algorithms. Technical Report 21/94, Fakultät für Informatik, Universität Karlsruhe, Karlsruhe, Germany.
35. Ramamurti, V. & Ghosh, J. (1996) Advances in using hierarchical mixture of experts for signal classification. In International Conference on Acoustics, Speech, and Signal Processing, Atlanta, pp. VI:3569-3572.
36. Richard, M. & Lippmann, R. (1991) Neural network classifiers estimate Bayesian a posteriori probabilities. Neural Computation, 3, 461-483.
37. Rogova, G. (1994) Combining the results of several neural network classifiers. Neural Networks, 7, 777-781.
38. Ruck, D.W., Rogers, S.K., Kabrisky, M.E., Oxley, M.E. & Suter, B.W. (1990) The multilayer perceptron as an approximation to a Bayes optimal discriminant function. IEEE Transactions on Neural Networks, 1, 296-298.
39. Sollich, P. & Krogh, A. (1996) Learning with ensembles: how overfitting can be useful. In D. S. Touretzky, M. C. Mozer & M. E. Hasselmo (Eds), Advances in Neural Information Processing Systems-8. Cambridge, MA: MIT Press, pp. 190-196.
40. Stone, M. (1974) Cross-validatory choice and assessment of statistical prediction. Journal of the Royal Statistical Society, 36, 111-147.
41. Towell, G.G. & Shavlik, J.W. (1992) Interpretation of artificial neural networks: Mapping knowledge-based neural networks into rules. In J. Moody, S. Hanson & R. Lippmann (Eds), Advances in Neural Information Processing Systems-4. San Mateo, CA: Morgan Kaufmann, pp. 977-984.
42. Turner, K. & Ghosh, J. (1995a) Bayes Error Rate Estimation through Classifier Combining. Technical Report 96-01-101, The Computer and Vision Research Center, University of Texas, Austin, TX.
43. Turner, K. & Ghosh, J. (1995b) Order statistics combiners for neural classifiers. In Proceedings of the World Congress on Neural Networks. Washington, DC: INNS Press, pp. I:31-34.
44. Tumer, K. & Ghosh, J. (1995c) Theoretical Foundations of Linear and Order Statistics Combiners for Neural Pattern Classifiers. Technical Report 95-02-98, The Computer and Vision Research Center, University of Texas, Austin, TX.
45. Turner, K. & Ghosh, J. (1996) Analysis of decision boundaries in linearly combined neural classifiers. Pattern Recognition, 29, 341-348.
46. Twomey, J.M. & Smith, A.E. (1995) Committee networks by resampling. In C. H. Dagli, M. Akay, C. L. P. Chen, B. R. Fernández & J. Ghosh (Eds), Intelligent Engineering Systems through Artificial Neural Networks, Vol. 5. New York: ASME Press, pp. 153-158.
47. Weiss, S.M. & Kulikowski, C. (1991) Computer Systems That Learn. San Mateo, CA: Morgan Kaufmann.
48. Wolberg, W.H. & Mangasarian, O. (1990) Multisurface method of pattern separation for medical diagnosis applied to breast cytology. In Proceedings of the National Academy of Sciences, Vol. 87, pp. 9193-9196.
49. Wolpert, D.H. (1990) A mathematical theory of generalization. Complex Systems, 4, 151-200.
50. Wolpert, D.H. (1992) Stacked generalization. Neural Networks, 5, 241-259.
51. Xu, L., Krzyzak, A. & Suen, C.Y. (1992) Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Transactions on Systems, Man and Cybernetics, 22, 418-435.
52. Xu, L., Jordan, M.I. & Hinton, G.E. (1995) An alternative method for mixtures of experts. In G. Tesauro, D. S. Touretzky & T. K. Leen (Eds), Advances in Neural Information Processing Systems - 7. Cambridge, MA: MIT Press, pp. 633-640.
53. Yang, J.-B. & Singh, M.G. (1994) An evidential reasoning approach for multiple-attribute decision making with uncertainty. IEEE Transactions on Systems, Man, and Cybernetics, 24, 1-19.