Trainable fusion rules. I. Large sample size case

Neural Networks

Volumn 19, Issue 10, 2006, Pages 1506-1516

  Raudys, Šarunas ^a  

^a VILNIUS UNIVERSITY   (Lithuania)

Author keywords

Classifier combination; Complexity; Fusion; Generalization error; Learning set; Multiple classifiers systems; Neural network ensembles

Indexed keywords

COMPUTATIONAL METHODS; COMPUTER OPERATING SYSTEMS; DECISION THEORY; MEASUREMENT ERRORS; NEURAL NETWORKS; PATTERN RECOGNITION; STATISTICAL METHODS;

FUSION RULES; MULTIPLE CLASSIFIERS SYSTEMS; ORACLE; STATISTICAL CLASSIFICATION RULES;

ALGORITHMS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; DISCRIMINANT ANALYSIS; LINEAR SYSTEM; MATHEMATICAL ANALYSIS; NONLINEAR SYSTEM; PERFORMANCE; PRIORITY JOURNAL; SAMPLE SIZE; STATISTICS;

ANIMALS; COMPUTER SIMULATION; FUZZY LOGIC; HUMANS; LEARNING; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); NEURONS; NONLINEAR DYNAMICS; SAMPLE SIZE;

EID: 33751205340     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2006.01.018     Document Type: Article

References (53)

1. Breiman L., Friedman J., Olshen R., and Stone C. Classification and regression trees (1984), Chapman & Hall, New York
2. Clemen R.T. Combining forecasts: a review and annotating bibliography. International Journal of Forecasting 5 (1989) 559-583
3. Cochran W.D., and Hopkins C. Some classification problems with multivariate qualitative data. Biometrics 17 (1961) 11-31
4. Diettrich T.G. Ensemble methods in machine learning. Lecture notes in computer science Vol. 1857 (2000) 1-15
5. Duda R.O., Hart P.E., and Stork D.G. Pattern classification and scene analysis. 2nd ed. (2000), John Wiley, New York
6. Friedman J.M. Regularized discriminant analysis. Journal of American Statistical Association 84 (1989) 165-175
7. Fukunaga K. Introduction to statistical pattern recognition. 2nd ed. (1990), Academic Press, New York
8. Fumera F., and Roli F. A theoretical and experimental analysis of linear combiners for multiple classifier systems. IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (2005) 942-956
9. Giacinto G., Roli F., and Fumera G. Selection of classifiers based on multiple classifier behaviour. Lecture notes in computer science Vol. 1876 (2000) 87-93
10. Giacinto G., and Roli F. Dynamic classifier selection based on multiple classifier behaviour. Pattern Recognition 34 9 (2001) 179-181
11. Gosh J. Multi-classifier systems: back to the future. Lecture notes in computer science Vol. 2364 (2002) 1-15
12. Güler, C., Sankur, B., Kahya, Y., Skurichina, M., & Raudys, S. (1996). Classification of respiratory sound patterns by means of cooperative neural networks. In G. Ramponi, G. L. Sicuranza, S. Carrato, S. Marsi (Eds.), Proceedings of 8th European signal processing conference (pp. 859-862)
13. Hashem S. Optimal linear combination of neural networks. Neural Networks 19 (1997) 599-614
14. Haykin S. Neural networks: A comprehensive foundation. 2nd ed. (1999), Prentice-Hall, Englewood Cliffs, NJ
15. Ho T.K. Data complexity analysis for classifier combination. Lecture notes in computer science Vol. 2096 (2001) 53-67
16. Huang Y.S., and Suen C.Y. A method of combining multiple experts for recognition of unconstrained handwritten numerals. IEEE Transactions on Pattern Analysis and Machine Intelligence 17 (1995) 90-94
17. Islam M.M., Yao X., and Murase K. A constructive algorithm for training cooperative neural network ensembles. IEEE Transactions on Neural Networks 14 (2003) 820-834
18. Janeliunas A., and Raudys S. Reduction of boasting bias' of linear expert. Lecture notes in computer science Vol. 2364 (2002) 242-251
19. Kang H.J. Combining multiple classifiers based on third-order dependency for handwritten numeral recognition. Pattern Recognition Letters 24 (2003) 3027-3036
20. Kittler J. Combining classifiers: A theoretical framework. Pattern Analysis and Applications 1 (1998) 18-27
21. Kittler J., Hatef M., Duin R.P.W., and Matas J. On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20 3 (1998) 226-239
22. Multiple classifier systems. In: Kittler J., and Roli F. (Eds). Lecture notes in computer science Vol. 1857 (2000)
23. Multiple classifier systems. In: Kittler J., and Roli F. (Eds). Lecture notes in computer science Vol. 2096 (2001)
24. Multiple classifier systems. In: Kittler J., and Roli F. (Eds). Lecture notes in computer science Vol. 2364 (2002)
25. Multiple classifier systems. In: Kittler J., and Roli F. (Eds). Lecture notes in computer science Vol. 2709 (2003)
26. Multiple classifier systems. In: Kittler J., and Roli F. (Eds). Lecture notes in computer science Vol. 3077 (2004)
27. Kuncheva L.I., Bezdek J.C., and Duin R.P.W. Decision templates for multiple classifier fusion: And experimental comparison. Pattern Recognition 34 (2001) 299-314
28. Kuncheva L.I. Combining pattern classifiers: Methods and Algorithms (2004), Wiley, Hoboken, NJ
29. Lachenbruch P.A., and Goldstein M. Discriminant analysis. Biometrics 5 3 (1979) 9-85
30. Lbov G.S. Methods of processing experimental data with mixed variables (1981), Nauka, Novosibirsk (in Russian)
31. Linhart G. Techniques of discriminant analysis with discrete variables. Metrika 2 116 (1959) 138-140
32. Nilsson N. Learning machines (1965), McGraw-Hill Book Company, New York
33. Quinlan J.R. C4.5: Programs for machine learning (1993), Morgan Kaufmann, San Mateo, CA
34. Rahman A.F.R., and Fairhurst M.C. Multiple classifier decision combination strategies for character recognition: A review. International Journal of Document Analysis and Recognition 5 4 (2003) 166-194
35. Ramachandran R.P., Farrell K.R., and Mammone R.J. Speaker recognition-general classifier approaches and data fusion methods. Pattern Recognition 35 (2002) 2801-2821
36. Rastrigin L.A., and Erenstein R.Ch. On decision making in a collective of decision rules. Priborostroenie 16 11 (1973) 31-35 (in Russian)
37. Rastrigin L.A., and Erenstein R.Ch. Method of collective recognition (1981), Energoizdat, Moscow (in Russian)
38. Raudys S. On the effectiveness of Parzen window classifier. Informatica 2 (1991) 434-454
39. Raudys S. Evolution and generalization of a single neurone. I. SLP as seven statistical classifiers. Neural Networks 11 (1998) 283-296
40. Raudys S. Combining the expert networks: a review. In: Sadykhov R. (Ed). Proc. of internat. conf. on neural networks and artificial intelligence (2001), Minsk 81-91
41. Raudys S. Statistical and neural classifiers: An integrated approach to design (2001), Springer-Verlag, London
42. Raudys S. Multiple classification systems in the context of feature extraction and selection. Lecture notes in computer science Vol. 2364 (2002) 27-41
43. Raudys S. Experts' boasting in trainable fusion rules. IEEE Transactions on Pattern Analysis and Machine Intelligence 25 (2003) 1178-1182
44. Raudys S. Trainable fusion rules. II. Small sample-size effects. Neural Networks (2006) 10.1016/j.neunet.2006.01.019
45. Raudys S., and Jain A.K. Small sample size effects in statistical pattern recognition: Recommendations for practitioners. IEEE Transactions on Pattern Analysis and Machine Intelligence 13 (1991) 252-264
46. Raudys S., and Roli F. The behavior knowledge space fusion method: analysis of generalization error and strategies for performance improvement. Lecture notes in computer science Vol. 2709 (2003) 55-64
47. Roli F., Raudys S., and Marcialis G.L. An experimental comparison of fixed and trained rules for crisp classifiers outputs. Lecture notes in computer science Vol. 2364 (2002) 232-241
48. Sabourin, M., Mitiche, A., Thomas, D., & Nagy, G. (1993). Classifier combination for handprinted digit recognition. In Proc. second int. conf. document analysis and recognition (pp. 163-166)
49. Telksnys L. Two-stage optimal recognition systems. Cybernetics 4 2 (1968) 89-92 Kiev
50. Ueda N.L. Optimal linear combination of neural networks for improving classification performance. IEEE Transactions on Pattern Analysis and Machine Inteligence 22 (2000) 207-215
51. Vapnik V.N. The nature of statistical learning theory (1995), Springer-Verlag, Berlin
52. Woods K., Kegelmeyer W.P., and Bowyer K. Combination of multiple classifiers using local accuracy estimates. IEEE Transactions on Pattern Analysis and Machine Intelligence 19 (1997) 405-410
53. Xu L., Krzyzak A., and Suen C.Y. Methods of combining multiple experts for the recognition of unconstrained handwritten numerals. IEEE Transactions Systems Man and Cybernetics 22 (1992) 418-435