Robust combining of disparate classifiers through order statistics

Pattern Analysis and Applications

Volumn 5, Issue 2, 2002, Pages 189-200

  Tumer, Kagan ^a   Ghosh, Joydeep ^b  

^a NASA AMES RESEARCH CENTER   (United States)

^b The University of Texas at Austin   (United States)

Author keywords

Classification error; Ensembles; Order statistics; Robust statistics; Trimmed means

Indexed keywords

EID: 0036088195     PISSN: 14337541     EISSN: None     Source Type: Journal    
DOI: 10.1007/s100440200017     Document Type: Article

References (33)

1. Geman S, Bienenstock E, Doursat R. Neural networks and the bias/variance dilemma. Neural Computation 1992; 4(1):1-58
2. Ghosh J, Tumer K. Structural adaptation and generalization in supervised feedforward networks. J Artificial Neural Networks 1994; 1(4):431-458
3. Ghosh J, Deuser L, Beck S. A neural network based hybrid system for detection, characterization and classification of short-duration oceanic signals. IEEE J Ocean Engineering October 1992; 17(4):351-363
4. Hansen LK, Salamon P. Neural network ensembles. IEEE Trans Pattern Analysis and Machine Intelligence 1990; 12(10):993-1000
5. Ho TK, Hull JJ, Srihari SN. Decision combination in multiple classifier systems. IEEE Trans Pattern Analysis and Machine Intelligence 1994; 16(1):66-76
6. Perrone MP, Cooper LN. When networks disagree: Ensemble methods for hybrid neural networks. In: Mammone RJ (ed), Neural Networks for Speech and Image Processing. Chapman & Hall, 1993
7. Sharkey AJC (ed). Connection Science: Special Issue on Combining Artificial Neural Networks: Ensemble Approaches 1996; 8(3 & 4)
8. Breiman L. Combining predictors. In: Sharkey AJC (ed), Combining Artificial Neural Nets. Springer-Verlag, 1999; 31-50
9. Turner K, Ghosh J. Error correlation and error reduction in ensemble classifiers. Connection Science, Special Issue on Combining Artificial Neural Networks: Ensemble Approaches 1996; 8(3 & 4):385-404
10. Batti R, Colla AM. Democracy in neural nets: Voting schemes for classification. Neural Networks 1994; 7(4):691-709
11. Kittler J, Hatef M, Duin RPW, Matas J. On combining classifiers. IEEE Trans Pattern Analysis and Machine Intelligence 1998; 20(3):226-239
12. Turner K, Ghosh J. Analysis of decision boundaries in linearly combined neural classifiers. Pattern Recognition 1996; 29(2):341-348
13. Chan P, Stolfo S. On the accuracy of meta-learning for scalable data mining. J Intelligent Integration of Information 1997; 8(1):8-25
14. Wolpert DH. Stacked generalization. Neural Networks 1992; 5:241-259
15. Al-Ghoneim K, Vijaya Kumar BVK. Learning ranks with neural neworks (Invited paper). Applications and Science of Artificial Neural Networks, Proceedings of the SPIE April 1995; 2492:446-464
16. Jacobs R. Method for combining experts' probability assessments. Neural Computation 1995; 7(5):867-888
17. Dietterich TG. Ensemble methods in machine learning. In: Kittler J, Roli F (eds), Multiple Classifier Systems, LNCS. Springer-Verlag, 2000; 1857:1-15
18. Dasarathy B. Decision Fusion. IEEE CS Press, Los Alamitos, CA, 1994
19. Hashem S, Schmeiser B. Approximating a function and its derivatives using MSB-optimal linear combinations of trained feedforward neural networks. Proceedings Joint Conference on Neural Networks 1993; 87:I:617-620
20. Merz CJ, Pazzani MJ. Combining neural network regression estimates with regularized linear weights. In: Mozer MC, Jordan MI, Petsche T (eds), Advances in Neural Information Processing Systems-9. MIT Press, 1997; 564-570
21. Turner K, Ghosh J. Linear and order statistics combiners for pattern classification. In: Sharkey AJC (ed), Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems. Springer-Verlag, London, 1999; 127-162.
22. Blake C, Keogh E, Merz CJ. UCI repository of machine learning databases, 1998 (URL: http://www.ics.uci.edu/~mlearn/MLRepository.html)
23. Prechelt L. PROBEN1 - A set of benchmarks and benchmarking rules for neural network training algorithms. Technical Report 21/94, Fakultät für Informatik, Universität Karlsruhe, D-76128 Karlsruhe, Germany, September 1994. (Anonymous FTP:/pub/papers/techreports/1994/1994-21.ps.Z on ftp.ira.uka.de)
24. David HA. Order Statistics. Wiley, NY, 1970
25. Arnold BC, Balakrishnan N, Nagaraja HN. A First Course in Order Statistics. Wiley, NY, 1992
26. Sarhan AE, Greenberg BG. Estimation of location and scale parameters by order statistics from singly and doubly censored samples. Ann Math Statist Science 1956; 27:427-451
27. Friedman JH. On bias, variance, 0/1 loss and the curse-of-dimensionality. Data Mining and Knowledge Discovery 1997; 1(1):55-78
28. Schapire R, Freund Y, Bartlett P, Lee WS. Boosting the margin: A new explanation for the effectiveness of voting methods. Proceedings 14th International Conference on Machine Learning. Morgan Kaufmann, 1997
29. Kargupta H, Chan P (eds). Advances in Distributed and Parallel Knowledge Discovery. AAAI/MIT Press, Cambridge, MA, 2000
30. Turner K, Oza NC. Decimated input ensembles for improved generalization. Proceedings International Joint Conference on Neural Networks (IJCNN-99) 1999
31. Bradley PS, Fayyad UM. Refining initial points for K-means clustering. Proceedings International Conference on Machine Learning (ICML-98), July 1998; 91-99
32. Duin RPW, Tax DMJ. Experiments with classifier combining rules. In: Kittler J, Roli F (eds), Multiple Classifier Systems, LNCS. Springer-Verlag, 2000; 1857:16-29
33. Chen K. A connectionist method for pattern classification with diverse features. Pattern Recognition Letters 1998; 19:545-558