A unified evolutionary training scheme for single and ensemble of feedforward neural network

Neurocomputing

Volumn 143, Issue , 2014, Pages 347-361

  Chen, Wen Ching ^a   Tseng, Lin Yu ^b   Wu, Chih Sheng ^c  

^a HSIUPING INSTITUTE OF TECHNOLOGY   (Taiwan)

^b PROVIDENCE UNIVERSITY   (Taiwan)

^c NATIONAL CHUNG HSING UNIVERSITY   (Taiwan)

Author keywords

Artificial neural network; Ensemble; Evolutionary algorithm; Orthogonal array

Indexed keywords

CLASSIFICATION (OF INFORMATION); EVOLUTIONARY ALGORITHMS; NEURAL NETWORKS;

ARTIFICIAL NEURAL NETWORK ENSEMBLES; CLASSIFICATION ABILITY; CONNECTION WEIGHTS; ENSEMBLE; EVOLUTIONARY TRAINING; GENERALIZATION ABILITY; ORTHOGONAL ARRAY; UCI MACHINE LEARNING REPOSITORY;

FEEDFORWARD NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; ARTIFICIAL NEURAL NETWORK ENSEMBLE CLASSIFIER; ARTIFICIAL NEURAL NETWORK SINGLE CLASSIFIER; CLASSIFICATION ALGORITHM; CLASSIFIER; EVOLUTIONARY ALGORITHM; FEED FORWARD NEURAL NETWORK; KERNEL METHOD; MATHEMATICAL COMPUTING; ORTHOGONAL ARRAY; PRIORITY JOURNAL; PROBLEM SOLVING; RANDOM ARRAY; RANDOM ARRAY LOCAL SEARCH; RANDOMIZATION; STATISTICAL ANALYSIS; STATISTICAL CONCEPTS; UNIFIED EVOLUTIONARY TRAINING SCHEME;

EID: 84904798148     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2014.05.057     Document Type: Article

References (69)

1. Bishop C.M. Neural Networks for Pattern Recognition 1995, Clarendon Press, Oxford.
2. Haykin S. Neural Networks: A Comprehensive Foundation 1999, Prentice-Hall, Upper Saddle River, NJ. 2nd ed.
3. Cai Y.D., Chou K.C. Artificial neural network for predicting alpha-turn types. Anal. Biochem. 1999, 268:407-409.
4. Fahlman S.E. An empirical study of learning speed in back-propagation networks. Technical Report CMU-CS-88-162 1988, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA.
5. Schiffmann W., Joost M., Werner R. Optimization of the Backpropagation Algorithm for Training Multilayer Perceptrons, Technical Report 1993, University of Koblenz, Institute of Physics, Germany.
6. M. Riedmiller, H. Braun, A direct adaptive method for faster backpropagation learning: the RPROP algorithm, in: Proceedings of the IEEE[U+05F3]93 International Conference on Neural Networks, 1993, pp. 586-591.
7. Yao X. Evolving artificial neural networks. Proc. IEEE 1999, 87(9):1423-1447.
8. Gen M., Cheng R. Genetic Algorithms and Engineering Design 1997, John Wiley & Sons.
9. Hansen L.K., Salamon P. Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell. 1990, 12(10):993-1001.
10. Yao X., Liu Y. Making use of population information in evolutionary artificial neural networks. IEEE Trans. Syst., Man, Cybern., Part B: Cybern. 1998, 28(3):417-425.
11. Yao X., Liu Y. A new evolutionary system for evolving artificial neural networks. IEEE Trans. Neural Netw. 1997, 8:694-713.
12. R. Mendes, P. Cortez, M. Rocha, J. Neves, Particle swarms for feedforward neural network training, in: Proceedings of the IEEE International Joint Conference on Neural Networks, 2002, pp. 1895-1899.
13. Nikolaev N., Iba H. Learning polynomial feedforward neural networks by genetic programming and backpropagation. IEEE Trans. Neural Netw. 2003, 14(2):337-350.
14. Su T., Jhang J., Hou C. A hybrid artificial neural networks and particle swarm optimization for function approximation. Int. J. Innov. Comput., Inf. Control 2008, 4(9):2363-2374.
15. Leung F., Lam H., Ling S., Tam P. Tuning of the structure and parameters of a neural network using an improved genetic algorithm. IEEE Trans. Neural Netw. 2003, 14(1):79-88.
16. Yu J., Wang S., Xi L. Evolving artificial neural networks using an improved PSO and DPSO. Neurocomputing 2008, 71(4-6):1054-1060.
17. He S., Wu Q.H., Saunders J.R. Group search optimizer: an optimization algorithm inspired by animal searching behavior. IEEE Trans. Evol. Comput. 2009, 13(5):973-990.
18. J.J.Q. Yu, A.Y.S. Lam, V.O.K. Li, Evolutionary artificial neural network based on chemical reaction optimization, in: Proceedings of the IEEE Congress on Evolutionary Computation, 2011, pp. 2083-2090.
19. Juang C.-F. A hybrid of genetic algorithm and particle swarm optimization for recurrent network design. IEEE Trans. Syst., Man, Cybern., Part B: Cybern. 2004, 34(2):997-1006.
20. Tsai J.-T., Liu T.-K., Chou J.-H. Hybrid Taguchi-genetic algorithm for global numerical optimization. IEEE Trans. Evol. Comput. 2004, 8(4):365-377.
21. Tsai J.-T., Chou J.-H., Liu T.-K. Tuning the structure and parameters of a neural network by using hybrid Taguchi-genetic algorithm. IEEE Trans. Neural Netw. 2006, 17(1):69-80.
22. A. Georgieva, I. Jordanov, Supervised neural network training with a hybrid global optimization technique, in: Proceedings of the IEEE International Joint Conference on Neural Networks, 2006, pp. 6433-6440.
23. Ludermir T.B., Yamazaki A., Zanchettin C. An optimization methodology for neural network weights and architectures. IEEE Trans. Neural Netw. 2006, 17(6):1452-1459.
24. Jordanov I., Georgieva A. Neural network learning with global heuristic search. IEEE Trans. Neural Netw. 2007, 18(3):937-942.
25. L.-Y. Tseng, W.-C. Chen, A two-phase genetic local search algorithm for feedforward neural network training, in: Proceedings of the IEEE International Joint Conference on Neural Networks, 2006, pp. 2914-2918.
26. Chen W.-C., Tseng L.-Y. The systematic trajectory search algorithm for the generalized feedforward neural network training. Int. J. Innov. Comput., Inf. Control 2011, 7(5A):2457-2471.
27. Palmes P.P., Hayasaka T., Usui S. Mutation-based genetic neural network. IEEE Trans. Neural Netw. 2005, 16(3):587-600.
28. Prechelt L. Proben1 - A Set of Neural Network Benchmark Problems and Benchmarking Rules, Technique Report 21/94 1994, Fakultat fur Informatik, University of Karlsruhe, Karlsruhe, Germany.
29. Alhamdoosh M., Wang D. Fast decorrelated neural network ensembles with random weights. Inf. Sci. 2014, 264:104-117.
30. Wang D., Alhamdoosh M. Evolutionary extreme learning machine ensembles with size control. Neurocomputing 2013, 102(15):98-110.
31. Tan C.-J., Lim C.-P., Cheah Y.-N. A multi-objective evolutionary algorithm-based ensemble optimizer for feature selection and classification with neural network models. Neurocomputing 2014, 125(11):217-228.
32. Tian J., Li M., Chen F., Kou J. Coevolutionary learning of neural network ensemble for complex classification tasks. Pattern Recognit. 2012, 45(4):1373-1385.
33. Tsakonas A., Gabrys B. A fuzzy evolutionary framework for combining ensembles. Appl. Soft Comput. 2013, 13(4):1800-1812.
34. Kumar P.A.R., Selvakumar S. Detection of distributed denial of service attacks using an ensemble of adaptive and hybrid neuro-fuzzy systems. Comput. Commun. 2013, 36(3):303-319.
35. Chou K.C., Shen H.B. Hum-PLoc: a novel ensemble classifier for predicting human protein subcellular localization. Biochem. Biophys. Res. Commun. 2006, 347:150-157.
36. Shen H.B., Chou K.C. Using ensemble classifier to identify membrane protein types. Amino Acids 2007, 32:483-488.
37. Sharkey A.J.C. On combining artificial neural nets. Connect. Sci. 1996, 8(3-4):299-313.
38. Breiman L. Bagging predictors. Mach. Learn. 1996, 24(2):123-140.
39. Y. Freund, R. Schapire, Experiments with a new boosting algorithm, in: Proceedings of the Thirteenth International Conference on Machine Learning, 1996, pp. 148-156.
40. Opitz D., Maclin R. Popular ensemble methods: an empirical study. J. Artif. Intell. Res. 1999, 11:169-198.
41. Yao X., Islam M.M. Evolving artificial neural network ensembles. IEEE Comput. Intell. Mag. 2008, 3(1):31-42.
42. Zhou Z.-H., Wu J., Tang W. Ensembling neural networks: many could be better than all. Artif. Intell. 2002, 137(1-2):239-253.
43. Dai Q., Liu Z. ModEnPBT: a modified backtracking ensemble pruning algorithm. Appl. Soft Comput. 2013, 13:4292-4302.
44. Dai Q. A novel ensemble pruning algorithm based on randomized greedy selective strategy and ballot. Neurocomputing 2013, 122:258-265.
45. Lin C., Chen W., Qiu C., Wu Y., Krishnan S., Zou Q. LibD3C: ensemble classifiers with a clustering and dynamic selection strategy. Neurocomputing 2014, 123:424-435.
46. Sharkey A.J.C. Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems 1999, Springer-Verlag, London.
47. Minku F.L., Ludermir T.B. Clustering and co-evolution to construct neural network ensembles: an experimental study. Neural Netw. 2008, 21:1363-1379.
48. Kim K.J., Cho S.B. Evolutionary ensemble of diverse artificial neural networks using speciation. Neurocomputing 2008, 71:1604-1618.
49. Brown G., Wyatt J., Harris R., Yao X. Diversity creation methods: a survey and categorisation. J. Inf. Fusion 2005, 6:5-20.
50. Yang F., Li X., Li Q., Li T. Exploring the diversity in cluster ensemble generation: random sampling and random projection. Expert Syst. Appl. 2014, 41(10):4844-4866.
51. Liu Y., Yao X. Ensemble learning via negative correlation. Neural Netw. 1999, 12:1399-1404.
52. Liu Y., Yao X., Higuchi T. Evolutionary ensembles with negative correlation learning. IEEE Trans. Evol. Comput. 2000, 4(4):380-387.
53. Chan Z.S.H., Kasabov N. Fast neural network ensemble learning via negative-correlation data correction. IEEE Trans. Neural Netw. 2005, 16(6):1707-1710.
54. Chan Z.S.H., Kasabov N. A preliminary study on negative correlation learning via correlation-corrected data (NCCD). Neural Process. Lett. 2005, 21(3):207-214.
55. H. Chen, X. Yao, Evolutionary random neural ensembles based on negative correlation learning, in: Proceedings of the 2007 IEEE Congress on Evolutionary Computation, 2007, pp. 1468-1474.
56. Chen H., Yao X. Multiobjective neural network ensembles based on regularized negative correlation learning. IEEE Trans. Knowl. Data Eng. 2010, 22(12):1738-1751.
57. M. Korytkowski, R. Scherer, Negative correlation learning of neuro-fuzzy system ensembles, in: Proceedings of the 10th International Conference on Artificial Intelligence and Soft Computing, part I, vol. LNAI 6113, 2010, pp. 114-119.
58. Y. Liu, Generate different neural networks by negative correlation learning, in: Proceedings of the First International Conference on Natural Computation, vol. LNCS 3610, 2005, 149-156.
59. Liu Y., Yao X. Simultaneous training of negatively correlated neural networks in an ensemble. IEEE Trans. Syst., Man, Cybern., Part B: Cybern. 1999, 29(6):716-725.
60. L.-Y. Tseng and W.-C. Chen, An evolutionary algorithm for variable-sized ANN ensemble construction, in: Proceedings of the 2010 IEEE International Conference on Systems, Man and Cybernetics (SMC), 2010, pp. 2381-2386.
61. Kittler J., Hatef M., Duin R.P.W., Matas J. On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 1998, 20(3):226-239.
62. Hedayat A.S., Sloane N.J.A., Stufken J. Orthogonal Arrays: Theory and Applications 1999, Springer-Verlag, New York.
63. Montgomery D.C. Design and Analysis of Experiments 1991, Wiley, New York. 3rd edition.
64. Leung Y.-W., Wang Y. An orthogonal genetic algorithm with quantization for global numerical optimization. IEEE Trans. Evol. Comput. 2001, 5(1):41-53.
65. C.L. Blake, C.J. Merz, UCI Repository of Machine Learning Database, Department of Information and Computer Science, University of California, Irvine, Irvine, CA, 1998. Available at: http://www.ics.uci.edu/~mlearn.
66. Setiono R., Hui L.C.K. Use of a quasi-Newton method in a feedforward neural-network construction algorithm. IEEE Trans. Neural Netw. 1995, 6(1):273-277.
67. Michie D., Spiegelhalter D.J., Taylor C.C. Machine Learning, Neural and Statistical Classification 1994, Ellis Horwood, London.
68. Chou K.C., Zhang C.T. Review: prediction of protein structural classes. Crit. Rev. Biochem. Mol. Biol. 1995, 30:275-349.
69. Chou K.C., Shen H.B. Cell-PLoc: a package of Web servers for predicting subcellular localization of proteins in various organisms. Nat. Protoc. 2008, 3:153-162.