Accuracy/diversity and ensemble MLP classifier design

IEEE Transactions on Neural Networks

Volumn 17, Issue 5, 2006, Pages 1194-1211

  Windeatt, Terry ^a  

^a UNIVERSITY OF SURREY   (United Kingdom)

Author keywords

Boolean; Diversity; Error correcting output coding (ECOC); Face identification; Multiple classifiers

Indexed keywords

BOOLEAN FUNCTIONS; CORRELATION THEORY; DATABASE SYSTEMS; ERROR ANALYSIS; FACE RECOGNITION; PATTERN RECOGNITION SYSTEMS;

ERROR-CORRECTING OUTPUT CODING (ECOC); FACE IDENTIFICATION; MULTIPLE CLASSIFIERS;

MULTILAYER NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; CLUSTER ANALYSIS; COMPUTER ANALYSIS; COMPUTER SIMULATION; METHODOLOGY; THEORETICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; CLUSTER ANALYSIS; COMPUTER SIMULATION; COMPUTING METHODOLOGIES; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED;

EID: 33750127404     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2006.875979     Document Type: Article

References (37)

1. L. K. Hansen and P. Salamon, "Neural network ensembles," IEEE Trans. Pattern Anal. Mach. Intell., vol. 12, no. 10, pp. 993-1001, Oct. 1990.
2. T. Bylander, "Estimating generalisation error on two-class datasets using out-of-bag estimates," Mach. Leani., vol. 48, pp. 287-297, 2002.
3. M. J. Er, S. Wu, and H. L. Toh, "Face recognition with RBF neural networks," IEEE Trans. Neural Netw., vol. 13, no. 3, pp. 697-710, May 2002.
4. L. Breiman, "Bagging predictors," Mach. Learn., vol. 24, no. 2, pp. 123-140, 1997.
5. T. K. Ho, "The random subspace method for constructing decision forests," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 8, pp. 832-844, Aug. 1998.
6. L. Breiman, "Randomizing outputs to increase prediction accuracy," Mach. Learn., vol. 40, no. 3, pp. 229-242, 2000.
7. L. I. Kuncheva and C. J. Whitaker, "Measures of diversity in classifier ensembles," Mach. Learn., vol. 51, pp. 181-207, 2003.
8. A. Narasimhamurthy, "Evaluation of diversity measures for binary classifier ensembles," in Proc. 6th Int. Workshop Multiple Classifier Systems, N. C. Oza, R. Polikar, F. Roli, and J. Kittler, Eds., Seaside, CA, Jun. 2005, Lecture Notes in Computer Science, pp. 267-277.
9. T. Windeatt and R. Tebbs, "Spectral technique for hidden layer neural network training," Pattern Recognit. Lett., vol. 18, no. 8, pp. 723-731, 1997.
10. T. Windeatt, "Recursive Partitioning for combining multiple classifiers," Neural Process. Lett., vol. 13, no. 3, pp. 221-236, Jun. 2001.
11. T. Windeatt, "Vote counting measures for ensemble classifiers," Pattern Recognit., vol. 36, no. 12, pp. 2743-2756, 2003.
12. A. N. Tikhonov and V. A. Arsenin, Solutions of Ill-Posed Problems. Washington, DC: Winston, 1977.
13. S. L. Hurst, D. M. Miller, and J. Muzio, Spectral Techniques in Digital Logic. New York: Academic, 1985.
14. R. E. Schapire, Y. Freund, and P. Bartlett, "Boosting the margin: a new explanation for the effectiveness of voting methods," Ann. Statistics, vol. 26, no. 5, pp. 1651-1686, 1998.
15. L. I. Kuncheva, "That elusive diversity in classifier ensembles," in Proc. Iberian Conf. Pattern Recognition and Image Analysis, Mallorca, Spain, 2003, Lecture Notes in Computer Science, pp. 1126-1138.
16. G. James, "Variance and bias for general loss functions," Mach. Learn., vol. 51, no. 2, pp. 115-135, 2003.
17. L. Breiman, "Arcing classifiers," Ann. Statistics, vol. 26, no. 3, pp. 801-849, 1998.
18. B. Efron and R. J. Tibshirani, An Introduction to the Bootstrap. London, U.K.: Chapman & Hall, 1993.
19. T. G. Dietterich and G. Bakiri, "Solving multi-class learning problems via error-correcting output codes," J. Artif. Intell. Res., vol. 2, pp. 263-286, 1995.
20. T. Windeatt, "Spectral measure for multi-class problems," in Proc. 4th Int. Workshop Multiple Classifier Systems, F. Roli, J. Kittler, and T. Windeatt, Eds., Cagliari, Italy, Jun. 2004, Lecture Notes in Computer Science, pp. 184-193.
21. T. Windeatt and R. Ghaderi, "Coding and decoding strategies for multi-class learning problems," Inf. Fusion, vol. 4, no. 1, pp. 11-21, 2003.
22. E. L. Allwein, R. E. Schapire, and Y. Singer, "Reducing multi-class to binary: a unifying approach for margin classifiers," J. Mach. Learn. Res., vol. 1, pp. 113-141, 2000.
23. V. Guruswami and A. Sahai, "Multi-class learning, boosting, and error-correcting codes," in Proc. 12th Conf. Computational Learning Theory, Santa Cruz, CA, July 1999.
24. K. Crammer and Y. Singer, "Improved output coding for classification using continuous relaxation," in Advances in Neural Information Processing Systems, T. G. Dietterich, S. Becker, and Z. Ghahramani, Eds. Cambridge, MA: MIT Press, 2002, vol. 14.
25. K. Crammer, "On the leamability and design of output codes for multi-class problems," Mach. Learn., vol. 47, no. 2, pp. 201-233, 2002.
26. R. E. Schapire, "Using output codes to boost multi-class learning problems," in 14th Int. Conf. Machine Learning, 1997, pp. 313-321.
27. J. Kittler, R. Ghaderi, T. Windeatt, and J. Matas, "Face verification via error correcting output codes," Image Vis. Comput., vol. 21, no. 13-14, pp. 1163-1169, 2003.
28. L. Prechelt, Proben1: A set of neural network benchmark problems and benchmarking rules. Univ. Karlsruhe, Karlsruhe, Germany, Tech. Rep. 21/94, 1994.
29. C. J. Merz and P. M. Murphy, UCI Repository of Machine Learning Databases 1998 [Online]. Available: http://www.ics.uci.edu/~mlearn/MLRepository.html
30. M. Riedmiller and H. Braun, "A direct adaptive method for faster backpropagation learning: the {RPROP} algorithm," in Proc. Int. Conf. Neural Networks, San Francisco, CA, 1993, pp. 586-591.
31. T. Windeatt, "Diversity measures for multiple classifier system analysis and design," Inf. Fusion, vol. 6, no. 1, pp. 21-36, 2004.
32. R. Rifkin and A. Klautau, "In defense of one-vs-all classification," J. Mach. Learn. Res., vol. 5, pp. 101-141, 2004.
33. G. L. Marcialis and F. Roli, "Fusion of appearance-based face recognition algorithms," Pattern Anal. Appl., vol. 7, no. 2, pp. 151-163, 2004.
34. M. Wang and S. Chen, "Enhanced FMAM based on empirical Kernel map," IEEE Trans. Neural Netw., vol. 16, no. 3, pp. 557-564, May 2005.
35. M. E. Er, W. Chen, and S. Wu, "High speed recognition based on discrete cosine transform and RBF neural networks," IEEE Trans. Neural Netw., vol. 16, no. 3, pp. 679-691, May 2005.
36. H. Zhang, B. Zhang, W. Huang, and Q. Tian, "Gabor wavelet associative memory for face recognition," IEEE Trans. Neural Netw., vol. 16, no. 1, pp. 275-278, Jan. 2005.
37. R. Caruna, S. Lawrence, and L. Giles, "Overfitting in neural nets: back-propagation, conjugate gradient, and early stopping," in Proc. Neural Information Processing Systems, Denver, CO, Nov. 28-30, 2000.