Analysis of CCME: Coevolutionary dynamics, automatic problem decomposition, and regularization

IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews

Volumn 38, Issue 1, 2008, Pages 100-109

  Nguyen, Minh Ha ^a   Abbass, Hussein A ^a   McKay, Robert I ^b  

^a UNIVERSITY OF NEW SOUTH WALES   (Australia)

^b Seoul National University   (South Korea)

Author keywords

Automatic problem decomposition (APD); Classification; Cooperative coevolution; Evolution strategies; Learning by forgetting (LF); Mixture of experts (ME); Regularization; Visualization

Indexed keywords

CLASSIFICATION (OF INFORMATION); DATA HANDLING; FLOW VISUALIZATION; GLOBAL OPTIMIZATION; MIXTURES; VISUALIZATION;

AUTOMATIC PROBLEM DECOMPOSITION; COOPERATIVE CO-EVOLUTION; EVOLUTION STRATEGIES; LEARNING BY FORGETTING (LF); MIXTURE OF EXPERTS; REGULARIZATION;

PRINCIPAL COMPONENT ANALYSIS;

EID: 63049114961     PISSN: 10946977     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCC.2007.905847     Document Type: Article

References (35)

1. X. Yao, "Evolving artificial neural networks", Proc. IEEE, vol. 87, no. 9, pp. 1423-1447, Sep. 1999.
2. H. A. Abbass, "Pareto neuro ensembles", in Proc. Aust. Conf. Artif. Intell., 2003, pp. 554-566.
3. D. Jimenez and N. Walsh, "Dynamically weighted ensemble neural networks for classification", in Proc. 1998 Int. Joint Conf. Neural Netw., pp. 753-756.
4. B. E. Rosen, "Ensemble learning using decorrelated neural networks", Connection Sci., vol. 8, no. 3/4, pp. 373-384, 1996.
5. A. Sharkey, Combining Artificial Neural Nets. Ensemble and Modular Multi-Net Systems. New York: Springer-Verlag, 1998.
6. X. Yao and Y. Liu, "Making use of population information in evolutionary artificial neural networks", IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 28, no. 3, pp. 417-425, Jun. 1998.
7. R. A. Jacobs, M. J. Jordan, S. J. Nowlan, and G. E. Hinton, "Adaptive mixtures of local experts", Neural Comput., vol. 3, pp. 79-87, 1991.
8. M. I. Jordan and R. A. Jacobs, "Hierachical mixtures of experts and the EM algorithm", Neural Comput., vol. 6, no. 2, pp. 181-214, 1994.
9. E. Wan and D. Bone, "Interpolating earth-science data using RBF networks and mixtures of experts", in Proc. NIPS, 1996, pp. 988-994.
10. R. Avnimelech and N. Intrator, "Boosted mixture of experts: An ensemble learning scheme", Neural Comput., vol. 11, no. 2, pp. 483-497, 1999.
11. B. Tang, M. I. Heywood, and M. Shepherd, "Input partitioning to mixture of experts", in Proc. Int. Joint Conf. Neural Netw., 2002, pp. 227-232.
12. M. I. Jordan and L. Xu, "Convergence results for the EM approach to mixtures of experts architectures", Aritif. Intell. Lab., MIT, Cambridge, MA, Nov. 1993.
13. K. Kang and J.-H. Oh. (1997). Statistical mechanics of the mixture of experts. Adv. Neural Inf. Process. Syst., M. C. Mozer, M. I. Jordan, and T. Petsche, Eds., vol. 9. MIT Press, Cambridge, MA.
14. L. A. Alexandre, A. Campilho, and M. Kamel, "Bounds for the average generalization error of the mixture of experts neural network", in SSPR/SPR, A. Fred, Ed. New York: Springer-Verlag, 2004, pp. 618-625.
15. S. R. Waterhouse, "Divide and conquer: Pattern recognition using mixtures of experts", Ph. D. dissertation, Univ. Cambridge, Cambridge, U. K., Feb. 1997.
16. V. Khare, X. Yao, B. Sendhoff, Y. Jin, and H. Wersing, "Co-evolutionary modular neural networks for automatic problem decomposition", in Proc. 2005 IEEE Congr. Evol. Comput., Sep. 2005, pp. 2691-2698.
17. N. Garcia-Pedrajas, C. Hervas-Martinez, and D. Ortiz-Boyer, "Cooperative coevolution of artificial neural network ensembles for pattern recognition", IEEE Trans. Evol. Comput., vol. 9, no. 3, pp. 271-302, Jun. 2005.
18. Y. Jin, T. Okabe, and B. Sendohff, "Neural network regularization and ensembling using multi-objective evolutionary algorithms", in Proc. Congr. Evol. Comput., Jun. 2004, pp. 1-8.
19. G. Brown and J. Wyatt, "Negative correlation learning and the ambiguity family of ensemble methods", in Proc. 4th Int. Workshop Multiple Classifier Syst. Lecture Notes in Computer Science, vol. 2709, T. Windeatt and F. Roli, Eds. Berlin, Germany: Springer-Verlag, Jun. 2003, pp. 266-275.
20. J. V. Hansen, "Combining predictors: Meta machine learning methods and bias/variance and ambiguity decompositions", Ph. D. dissertation, Dept. Comput. Sci. Univ. Aarhus, Aarhus, Denmark, 2000.
21. Y. Liu, X. Yao, and T. Higuchi, "Evolutionary ensembles with negative correlation learning", IEEE Trans. Evol. Comput., vol. 4, no. 4, pp. 380-387, Nov. 2000.
22. M. H. Nguyen, H. A. Abbass, and R. I. Mckay, "A novel mixture of experts model based on cooperative coevolution", Neurocomputing, vol. 70, pp. 155-163, 2006.
23. M. A. Potter, "The design and analysis of a computational model of cooperative coevolution", Ph. D. dissertation, George Mason Univ., Fairfax, VA, 1997.
24. D. E. Moriarty and R. Miikkulainen. (1997). Forming neural networks through efficient and adaptive coevolution. Evol. Comput., vol. 5, no. 4, pp. 373-399.
25. M. Ishikawa and K. Yoshino, "Automatic task decomposition in modular networks by structural learning with forgetting", in Proc. 1993 Int. Joint Conf. Neural Netw., pp. 1345-1348.
26. E. Cantu-Paz and C. Kamath, "An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems", IEEE Trans. Syst., Man Cybern., B, Cybern., vol. 35, no. 5, pp. 915-927, Oct. 2005.
27. I. T. Jolliffe, Principle Component Analysis. New York: Springer-Verlag, 1986.
28. C. M. Bishop, Neural Networks for Pattern Recognition. New York: Oxford Univ. Press, 1995.
29. D. J. Newman, S. Hettich, C. L. Blake, and C. J. Merz. (1998). UCI repository of machine learning databases. [Online]. Available: http://www.ics.uci.edu/~mlearn/MLRepository.html
30. R. King, C. Feng, and A. Shutherland, "Statlog: Comparison of classification algorithms on large real-world problems", Appl. Artif. Intell., vol. 9, no. 3, pp. 259-287, May/Jun. 1995.
31. I. Rechenberg, Evolutionsstrategie'94 (Werkstatt Bionik und Evolutionstechnik). Stuttgart, Germany: Frommann-Holzboog, 1994, vol. 1.
32. H.-G. Beyer and H.-P. Schwefel, "Evolution strategies: A comprehensive introduction", J. Natural Comput., vol. 1, no. 1, pp. 3-52, 2002.
33. E. Popovici and K. A. D. Jong, "Understanding EA dynamics via population fitness distributions", in Proc. GECCO, 2003, pp. 1604-1605.
34. B. C. Barber, D. P. Dobkin, and H. Huhdanpaa. The quickhull algorithm for convex hulls. ACM Trans. Math. Softw. vol. 22, no. 4, Dec. 1996, pp. 469-483.
35. A. S. Weigend, "On overfitting and the effective number of hidden units", in Proc. 1993 Connectionist Models Summer School, M. Mozer, P. Smolensky, D. Touretzky, J. Elman, and A. Weigend, Eds., pp. 335-342.