Evolving output codes for multiclass problems

IEEE Transactions on Evolutionary Computation

Volumn 12, Issue 1, 2008, Pages 93-106

  Garcia Pedrajas, Nicolás ^a   Fyfe, Fyfe ^b  

^a UNIVERSITY OF CÓRDOBA   (Spain)

^b UNIVERSITY OF THE WEST OF SCOTLAND   (United Kingdom)

Author keywords

Evolutionary computation; Multiclass; Output coding; Pattern recognition

Indexed keywords

CODES (SYMBOLS); ENCODING (SYMBOLS); EVOLUTIONARY ALGORITHMS; NEURAL NETWORKS; SUPPORT VECTOR MACHINES;

CODEWORDS; DATASETS; OUTPUT CODES; OUTPUT CODING;

PATTERN RECOGNITION;

EID: 40249096509     PISSN: 1089778X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TEVC.2007.894201     Document Type: Article

References (51)

1. L. J. Eshelman, The CHC Adaptive Search Algorithm: How to Have Safe Search When Engaging in Nontraditional Genetic Recombination. San Mateo, CA: Morgan Kauffman, 1990.
2. B. E. Boser, I. M. Guyon, and V. N. Vapnik, "A training algorithm for optimal margin classifiers," in Proc. 5th Annu. ACM Workshop on COLT D. Haussler, Ed., 1992, pp. 144-152.
3. A. G. Ivakhnenko, G. A. Ivakhnenko, and J.-A. Müller, "Self-organization of neural networks with active neurons," Pattern Recogn. Image Anal., vol. 4, no. 2, pp. 185-196, 1994.
4. T. G. Dietterich and G. Bakiri, "Solving multiclass learning problems via error-correcting output codes," J. Artif. Intell. Res., vol. 2, pp. 263-286, 1995.
5. T. Windeatt and R. Ghaderi, "Coding and decoding strategies for multi-class learning problems," Information Fusion, vol. 4, pp. 11-21, 2003.
6. E. B. Kong and T. G. Dietterich, Dept. Comput. Sci., Oregon State Univ., "Why error-correcting output coding works with decision trees," Corvallis, OR, Tech. Rep., 1995.
7. R. Rifkin and A. Klautau, "In defense of one-vs.-all classification," J. Mach. Learn. Res., vol. 5, pp. 101-141, 2004.
8. E. Alpaydin and E. Mayoraz, "Learning error-correcting output codes from data," in Proc. Int. Conf. Neural Networks ICANN'99, Edinburgh, U.K., Sep. 1999.
9. K. Crammer and Y. Singer, "On the learnability and design of output codes for multiclass problems," Mach. Learn., vol. 47, pp. 201-233, 2002.
10. F. Masulli and G. Valentini, "Effectiveness of error correcting output codes in multiclass learning problems," in Proc. Multiple Classifier Systems, MCS2000, J. Kittler and F. Roli, Eds., Cagliary, Italy, 2000, Lecture Notes in Computer Science, pp. 107-116.
11. E. L. Allwein, R. E. Schapire, and Y. Singer, "Reducing multiclass to binary: A unifying approach for margin classifiers," J. Mach. Learn. Res., vol. 1, pp. 113-141, 2000.
12. G. F. Miller, P. M. Todd, and S. U. Hedge, "Desiong neural networks," Neural Netw., vol. 4, pp. 53-60, 1991.
13. R. C. Bose and D. K. Ray-Chaudhuri, "On a class of error-correcting binary group codes," Inf. Control, vol. 3, pp. 68-79, 1960.
14. W. Utschick and W. Weichselberger, "Stochastic organization of output codes in multiclass learning problems," Neural Comput., vol. 13, no. 5, pp. 1065-1102, May 2001.
15. L. I. Kuncheva, "Using diversity measures for generating error-correcting output codes," Pattern Recogn. Lett., vol. 26, pp. 83-90, 2005.
16. L. Kuncheva and C. J. Whitaker, "Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy," Mach. Learn., vol. 51, no. 2, pp. 181-207, May 2003.
17. A. Passerini, M. Pontil, and P. Frasconi, "New results on error correcting output codes of kernel machines," IEEE Trans. Neural Netw., vol. 15, no. 1, pp. 45-54, Jan. 2004.
18. R. E. Shapire, "Using output codes to boost multiclass learning problems," in Proc. 14th Int. Conf. Mach. Learn., D. H. Fisher, Ed., Jul. 1997, pp. 313-321.
19. G. M. James and T. Hastie, "The error coding method and PICT'S," Computational and Graphical Statistics, vol. 7, pp. 377-387, 1998.
20. T. Windeatt and R. Ghaderi, "Binary labelling and decision-level fusion," Inf. Fusion, vol. 2, pp. 103-112, 2001.
21. R. E. Schapire, "Using output codes to boost multiclass learning problems," in Proc. 14th Int. Conf. Mach. Learn., 1997, pp. 313-321.
22. Y. Freund and R. E. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting," J. Comput. Syst. Sci., vol. 55, no. 1, pp. 119-139, 1997.
23. O. Pujol, P. Radeva, and J. Vitrià, "Discrimnant ECOC: A heuristic method for application dependent design of error correcting output codes," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 6, pp. 1007-1012, Jun.2006.
24. D. Aha and R. Bankert, "Cloud classification using error-correcting output codes," Artificial Intelligence Applications: Natural Resources, Agriculture, and Environmental Science, vol. 11, no. 1, pp. 13-28, 1997.
25. D. E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning. Reading, MA: Addison-Wesley, 1989.
26. Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs. New York: Springer-Verlag, 1994.
27. S. J. Louis and G. Li, "Combining robot control strategies using genetic algorithms with memory," Lecture Notes in Computer Science, Evolutionary Programming VI, vol. 1213, pp. 431-442, 1997.
28. J. R. Cano, F. Herrera, and M. Lozano, "Using evolutionary algorithms as instance selection for data reduction in KDD: An experimental study," IEEE Trans. Evol. Comput., vol. 7, no. 6, pp. 561-575, Dec. 2003.
29. L. Mason, P. L. Bartlett, and J. Baxter, "Improved generalization through explicit optimization of margins," Mach. Learn., vol. 38, pp. 243-255, 2000.
30. P. L. Bartlett, "The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the networks," IEEE Trans. Inf. Theory, vol. 44, no. 2, pp. 525-536, 1998.
31. R. E. Schapire, Y. Freund, P. L. Bartlett, and W. S. Lee, "Boosting the margin: A new explanation for the effectiveness of voting methods," Ann. Statistics, vol. 26, no. 5, pp. 1651-1686, 1998.
32. A. Afifi and S. Azen, Statistical Analysis. A Computer Oriented Approach. New York: Academic, 1979.
33. G. Yule, "On the association of attributes in statistics," Phil. Trans., vol. 194, pp. 257-319, 1900.
34. A. S. Weigend, D. E. Rumelhart, and B. A. Huberman, "Generalization by weight-elimination applied to currency exchange rate prediction," in Proc. Int. Joint Conf. Neural Netw., Seattle, WA, 1991, vol. 1, pp. 837-841.
35. G. Bebis, M. Georgiopoulos, and T. Kasparis, "Coupling weight elimination with genetic algorithms to reduce network size and preserve generalization," Neurocomputing, vol. 17, pp. 167-194, 1997.
36. S. Hettich, C. L. Blake, and C. J. Merz, "UCI repository of machine learning databases," 1998. Online. Available: http://www.ics.uci.edu/~mlearn/MLRepository.html
37. T. G. Dietterich, "Approximate statistical tests for comparing supervised classification learning algorithms," Neural Comput., vol. 10, no. 7, pp. 1895-1923, 1998.
38. T. W. Anderson, An Introduction to Multivariate Statistical Analysis ser. Wiley Series in Probability and Mathematical Statistics, 2nd ed. New York: Wiley, 1984.
39. J. Demšar, "Statistical comparisons of classifiers over multiple data sets," J. Mach. Learn. Res., vol. 7, pp. 1-30, 2006.
40. F. Wilcoxon, "Individual comparisons by ranking methods," Biometrics, vol. 1, pp. 80-83, 1945.
41. D. Rumelhart, G. Hinton, and R. J. Williams, "Learning internal representations by error propagation," in Parallel Distrib. Process. D. Rumelhart and J. McClelland, Eds. Cambridge, MA: MIT Press, 1986, pp. 318-362.
42. J. R. Quinlan, C4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kaufmann, 1993.
43. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines. Cambridge, U.K.: Cambridge Univ. Press, 2000.
44. Ch.-Ch. Chang and Ch.-J. Lin, "LIBSVM: A library for support vector machines," 2001. Online. Available: http://www.csie.ntu.edu.tw/ ~cjlin/libsvm
45. S. S. Keerthi and C.-J. Lin, "Asymptotic behaviors of support vector machines with a Gaussian kernel," Neural Comput., vol. 15, pp. 1667-1689, 2003.
46. S. Knerr, L. Personnaz, and G. Dreyfus, "Single-layer learning revisited: A stepwise procedure for building and training a neural network," in Neurocomputing: Algorithms, Architectures and Applications, J. Fogelman, Ed. New York: Springer-Verlag, 1990.
47. T. Hastie and R. Tibshirani, "Classification by pairwise coupling," The Annals of Statistics, vol. 26, no. 2, pp. 451-471, 1998.
48. J. Friedman, Department of Statistics, Standford University, "Another approach to polychotomous classification." Tech. Rep., 1996.
49. P. Clark and R. Boswell, "Rule induction with CN2: Some recent improvements," in Proc. 5th Eur. Working Session on Learning (EWSL-91), Porto, Portugal, 1991, pp. 151-163.
50. R. Anand, K. G. Mehrotra, C. K. Mohan, and S. Ranka, "Efficient classification for multiclass problems using modular neural networks," IEEE Trans. Neural Netw., vol. 6, pp. 117-124, 1995.
51. D. H. Wolpert and W. G. Macready, "No free lunch theorems for optimization," IEEE Trans. Evol. Comput., vol. 1, no. 1, pp. 67-82, 1997.