Evolving Artificial Neural Network Ensembles

IEEE Computational Intelligence Magazine

Volumn 3, Issue 1, 2008, Pages 31-42

  Yao, Xin ^a   Islam, Md Monirul ^b  

^a UNIVERSITY OF BIRMINGHAM   (United Kingdom)

^b BANGLADESH UNIVERSITY OF ENGINEERING AND TECHNOLOGY   (Bangladesh)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85008014942     PISSN: 1556603X     EISSN: None     Source Type: Journal    
DOI: 10.1109/MCI.2007.913386     Document Type: Article

References (64)

1. H.A. Abbass, and R. Sarker, and C. Newton, “PDE: A Pareto-frontier differential evolution approach for multi-objective optimization problems,” Proc. of the IEEE Congress on Evolutionary Computation (CEC2001), IEEE Press, pp. 971–978, 2001.
2. H.A. Abbass, “The self-adaptive pareto differential evolution algorithm,” Proc. of the 2002 Congress on Evolutionary Computation CEC2002, IEEE Press, pp. 831–836, 2002.
3. H.A. Abbass, “Speeding up backpropagation using multiobjective evolutionary algorithms,” Neural Computation, vol. 15, no. 11, pp. 2705–2726, 2003.
4. H.A. Abbass, “Pareto neuro-evolution: constructing ensemble of neural networks using multi-objective optimization,” Proc. of the 2003 Conference on Evolutionary Computation, IEEE Press, pp. 2074–2080, 2003.
5. H.A. Abbass, “Pareto neuro-ensemble,” Proc. of the 16th Australian Joint Conference on Artificial Intelligence, Springer, pp. 554–566, 2003.
6. P.J. Angeline, G.M. Sauders, and J.B. Pollack, “An evolutionary algorithm that constructs recurrent neural networks,” IEEE Trans. Neural Networks, vol. 5, no. 1, pp. 54–65, 1994.
7. P.F. Baldi and K. Hornik, “Learning in linear neural networks: A survey,” IEEE Trans. Neural Networks, vol. 6, no. 4, pp. 837–858, 1995.
8. C. Blake and C. Merz, UCI repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html.
9. R.K. Belew, J. McInerney, and N.N. Schraudolph, “Evolving networks: Using genetic algorithm with connectionist learning,” Tech. Rep. CS90- 174 (revised), Comput. Sci. Eng. Dept. (C-014), Univ. Calif., San Diego, Feb. 1991.
10. D. Bollé, D.R.C. Dominguez, and S. Amari, “Mutual information of sparsely coded associative memory with self-control and tenary neurons,” Neural Networks, vol. 13, pp. 452–462, 2000.
11. G. Brown and J.L. Wyatt, “Negative correlation learning and the ambiguity family of ensemble methods,” Proc. Int. Workshop on Multiple Classifier Systems, Springer, pp. 266–275, 2003.
12. G. Brown, J. Wyatt, R. Harris, and X. Yao, “Diversity creation methods: A survey and categorisation,” J. Inf. Fusion, vol. 6, pp. 5–20, 2005.
13. A. Chandra and X. Yao, “Ensemble learning using multi-objective evolutionary algorithms,” Journal of Mathematical Modelling and Algorithms, vol. 5, no. 4, pp. 417–445, 2006.
14. P. Darwen and X. Yao, “Every niching method has its niche: Fitness sharing and implicit sharing compared,” in Proc. of Parallel Problem Solving from Nature (PPSN IV), vol. 1141 of Lecture Notes in Computer Science, Berlin, Germany: Springer-Verlag, pp. 398–407, 1996.
15. P.J. Darwen and X. Yao, “Speciation as automatic categorical modularization,” IEEE Transactions on Evolutionary Computation, vol. 1, pp. 101–108, 1997.
16. T.G. Dietterich, “Machine-learning research: four current directions,” AI Mag., vol. 18, no. 4, pp. 97–136, 1998.
17. W. Finnoff, F. Hergent, and H.G. Zimmermann, “Improving model selection by nonconvergent methods,” Neural Networks, vol. 6, pp. 771–783, 1993.
18. L.J. Fogel, A.J. Owens, and M.J. Walsh, Artificial Intelligence Through Simulated Evolution. New York: Wiley, 1966.
19. G.B. Fogel and D.B. Fogel, “Continuous evolutionary programming: Analysis and experiments,” Cybern. Syst., vol. 26, pp. 79–90, 1995.
20. D.B. Fogel, Evolutionary Computation: Toward a New Philosophy of Machine Intelligence. New York: IEEE Press, 1995.
21. D.E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, MA: Addison-Wesley, 1989.
22. P.J.B. Hancock, “Genetic algorithms and permutation problems: a comparison of recombination operators for neural net structure specification,” Proc. Int. Workshop Combinations Genetic Algorithms Neural Networks (COGANN-92), D. Whitley and J. D. Schaffer, Eds., Los Alamitos, CA: IEEE Comput. Soc. Press, pp. 108–122, 1992.
23. L.K. Hansen and P. Salamon, “Neural network ensembles,” IEEE Trans. Pattern Anal. Mach. Intell, vol. 12, no. 10, pp. 993–1001, 1990.
24. S. Hashem, “Optimal linear combinations of neural networks,” Ph.D. dissertation, School Ind. Eng., Purdue Univ., West Lafayette, IN, Dec. 1993.
25. Kalyanmoy Deb, Multi-Objective Optimization Using Evolutionary Algorithms, Chichester, UK: Wiley, 2001.
26. V. Khare, X. Yao, and B. Sendhoff, “Multi-network evolutionary systems and automatic problem decomposition,” International Journal of General Systems, vol. 35, no. 3, pp. 259–274, 2006.
27. A. Krogh and J. Vedelsby, “Neural network ensembles, cross validation, and active learning,” Neural Inf. Process. Syst., vol. 7, pp. 231–238, 1995.
28. A. Krogh and P. Sollich, “Statistical mechanics of ensemble learning,” Phys. Rev. E, vol. 55, pp. 811–825, 1997.
29. T.Y. Kwok and D.Y. Yeung, “Constructive algorithms for structure learning in feed-forward neural networks for regression problems,” IEEE Trans. Neural Networks, vol. 8, pp. 630–645, May 1997.
30. T.Y. Kwok and D.Y. Yeung, “Objective functions for training new hidden units in constructive neural networks,” IEEE Trans. Neural Networks, vol. 8, pp. 1131–1148, Sept. 1997.
31. M. Lehtokangas, “Modeling with constructive backpropagation,” Neural Networks, vol. 12, pp. 707–716, 1999.
32. C.Y. Lee and X. Yao, “Evolutionary programming using the mutations based on the Lévy probability distribution,” IEEE Transactions on Evolutionary Computation, vol. 8, no. 1, pp. 1–13, 2004.
33. Y. Liu and X. Yao, “Ensemble learning via negative correlation,” Neural Networks, vol. 12, pp. 1399–1404, 1999.
34. Y. Liu, X. Yao, and T. Higuchi, “Evolutionary ensembles with negative correlation learning,” IEEE Transactions on Evolutionary Computation, vol. 4, no. 4, pp. 380–387, 2000.
35. J. MacQueen, “Some methods for classification and analysis of multivariate observation,” in Proc. 5th Berkely Symp. Mathematical Statistics and Probability, Berkely, CA: University of California Press, vol. 1, pp. 281–297, 1967.
36. S.W. Mahfoud, “Niching methods for genetic algorithms,” Ph.D.thesis, Dept. General Engineering, Univ. Illinois, Urbana-Champaign, IL, 1995.
37. Md. Monirul Islam, X. Yao, and K. Murase, “A constructive algorithm for training cooperative neural network ensembles,” IEEE Transactions on Neural Networks, vol. 14, pp. 820–834, 2003.
38. B. Mulgrew and C.F.N. Cowan, Adaptive Filters and Equalizers. Boston, MA: Kluwer, 1988.
39. S.V. Odri, D.P. Petrovacki, and G.A. Krstonosic, “Evolutional development of a multilevel neural network,” Neural Networks, vol. 6, no. 4, pp. 583–595, 1993.
40. D.W. Opitz and J.W. Shavlik, “Generating accurate and diverse members of a neural-network ensemble,” Neural Inf. Process. Syst., vol. 8, pp. 535–541, 1996.
41. D. Opitz and R. Maclin, “Popular ensemble methods: an empirical study,” J. Artif. Intell. Res., vol. 11, pp. 169–198, 1999.
42. M.P. Perrone, “Improving regression estimation: Averaging methods for variance reduction with extensions to general convex measure optimization,” Ph.D. dissertation, Dept. Physics, Brown Univ., Providence, RI, May 1993.
43. L. Prechelt, “Proben1-A set of neural network benchmark problems and benchmarking rules,” Tech. Rep. 21/94, Fakultät für Informatik, Univ. Karlsruhe, Karlsruhe, Germany, Sept. 1994.
44. L. Prechelt, “Some notes on neural learning algorithm benchmarking,” Neurocomputing, vol. 9, no. 3, pp. 343–347, 1995.
45. J. Rissanen, “Modeling by shortest data description,” Automatica, vol. 14, pp. 465–471, 1978.
46. D.E. Rumelhart, G.E. Hinton, and R.J. Williams, “Learning internal representations by error propagation,” in D. E. Rumelhart & J. L. McClelland (ed.), Parallel Distributed Processing: Explorations in the Microstructures of Cognition, I, Cambridge, MA: MIT Press, pp. 318–362, 1986.
47. A.J.C. Sharkey, “On combining artificial neural nets,” Connect. Sci., vol. 8, no. 3/4, pp. 299–313, 1996.
48. J.D. Schaffer, D. Whitley, and L.J. Eshelman, “Combinations of genetic algorithms and neural networks: A survey of the state of the art,” in Proc. Int. Workshop Combinations Genetic Algorithms Neural Networks (COGANN-92), D. Whitley and J. D. Schaffer (Eds.), Los Alamitos, CA: IEEE Comput. Soc. Press, pp. 1–37, 1992.
49. N. Srinivas and K. Deb, “Multi-objective function optimization using non-dominated sorting genetic algorithms,” Evolutionary Computation, vol. 2, no. 3, pp. 221–248, 1994.
50. R. Storn and K. Price, “Minimizing the real functions of the ICEC'96 contest by differential evolution,” Proc. of the IEEE International Conference on Evolutionary Computation, Nagoya, pp. 842–844, May 1996.
51. E.K. Tang, P.N. Suganthan, and X. Yao, “An Analysis of Diversity Measures,” Machine Learning, vol. 65, pp. 247–271, 2006.
52. X. Yao, “Evolution of connectionist networks,” Proc. of the Int. Symp. AI, Reasoning & Creativity, Queensland, Australia, Griffith Univ., pp. 49–52, 1991.
53. X. Yao, “An empirical study of genetic operators in genetic algorithms,” Microprocess. Microprogr., vol. 38, pp. 707–714, 1993.
54. X. Yao, “A review of evolutionary artificial neural networks,” Int. J. Intell. Syst., vol. 8, no. 4, pp. 539–567, 1993.
55. X. Yao, “Evolutionary artificial neural networks,” Int. J. Neural Syst., vol. 4, no. 3, pp. 203–222, 1993.
56. X. Yao, “The evolution of connectionist networks,” in Artificial Intelligence and Creativity, T. Dartnall (Ed.), Dordrecht, The Netherlands: Kluwer, pp. 233–243, 1994.
57. X. Yao, “Evolutionary artificial neural networks,” in Encyclopedia of Computer Science and Technology, vol. 33, A. Kent and J. G. Williams (Eds.), New York: Marcel Dekker, pp. 137–170, 1995.
58. X. Yao and Y. Shi, “A preliminary study on designing artificial neural networks using co-evolution,” in Proc. IEEE Singapore Int. Conf. Intelligent Control Instrumentation, pp. 149–154, June 1995.
59. X. Yao, “Evolving artificial neural networks,” Proceedings of the IEEE, vol. 87, pp. 1423–1447, 1999.
60. X. Yao and Y. Liu, “Ensemble structure of evolutionary artificial neural networks,” in Proc. of 1996 IEEE Int. Conf. Evolutionary Computation (ICEC'96), Nagoya, Japan, pp. 659–664, 1996.
61. X. Yao and Y. Liu, “A new evolutionary system for evolving artificial neural networks,” IEEE Trans. Neural Networks, vol. 8, no. 3, pp. 694–713, 1997.
62. X. Yao and Y. Liu, “Making use of population information in evolutionary artificial neural networks,” IEEE Trans. Syst., Man, Cybern., B., vol. 28, no. 3, pp. 417–425, 1998.
63. X. Yao, Y. Liu and P. Darwen, “How to make best use of evolutionary learning,” in Complex Systems: From Local Interactions to Global Phenomena, R. Stocker, H. Jelinek, and B. Durnota (Eds.), Amsterdam, Netherlands: IOS Press, pp. 229–242, 1996.
64. X. Yao, Y. Liu and G. Lin, “Evolutionary Programming Made Faster,” IEEE Transactions on Evolutionary Computation, vol. 3, no. 2, pp. 82–102, 1999.