Designing multilayer perceptrons using a guided saw-tooth evolutionary programming algorithm

Soft Computing

Volumn 14, Issue 6, 2010, Pages 599-613

  Gutiérrez, Pedro Antonio ^a   Hervás, César ^a   Lozano, Manuel ^b  

^a UNIVERSITY OF CÓRDOBA   (Spain)

^b UNIVERSITY OF GRANADA   (Spain)

Author keywords

Artificial neural networks; Evolutionary algorithms; Evolutionary programming; Multilayer perceptrons; Population reinitializations; Saw tooth algorithm

Indexed keywords

ARTIFICIAL NEURAL NETWORK; ARTIFICIAL NEURAL NETWORKS; EVOLUTIONARY PROGRAMMING; MULTI-LAYER PERCEPTRONS; POPULATION REINITIALIZATIONS; SAW-TOOTH;

COMPUTER PROGRAMMING; COST REDUCTION; CYBERNETICS; DENTISTRY; ELECTRIC LOADS; MULTILAYER NEURAL NETWORKS; MULTILAYERS; PATTERN RECOGNITION SYSTEMS; POPULATION STATISTICS;

EVOLUTIONARY ALGORITHMS;

EID: 77952880576     PISSN: 14327643     EISSN: 14337479     Source Type: Journal    
DOI: 10.1007/s00500-009-0429-x     Document Type: Article

References (63)

1. Alcala-Fdez J, Sánchez L, García S, del Jesús MJ, Ventura S, Garrell JM, Otero J, Romero C, Bacardit J, Rivas VM, Fernández JC, Herrera F (2008) KEEL: a software tool to assess evolutionary algorithms for data mining problems. Soft Comput 13(3): 307-318.
2. Angeline PJ, Sauders GM, Pollack JB (1994) An evolutionary algorithm that constructs recurrent neural networks. IEEE Trans Neural Netw 5: 54-65.
3. Asuncion A, Newman D (2007) UCI machine learning repository. http://www. ics. uci. edu/mlearn/MLRepository. html.
4. Back T, Fogel DB, Michalewicz Z (1997) Handbook of evolutionary computation. IOP Publishing Ltd, Bristol.
5. Bishop CM (2006) Pattern recognition and machine learning. Springer, Berlin.
6. Bornholdt S, Graudenz D (1992) General asymmetric neural networks and structure design by genetic algorithms. Neural Netw 5(2): 327-334. doi: 10. 1016/S0893-6080(05)80030-9.
7. Bramlette MF (1991) Initialization, mutation and selection methods in genetic algorithms for function optimization. In: Belew RK, Booker LB (eds) Proceedings of the 4th international conference on genetic algorithms (ICGA'91), pp 100-107.
8. Chaiyaratana N, Piroonratana T, Sangkawelert N (2007) Effects of diversity control in single-objective and multi-objective genetic algorithms. J Heuristics 13(1): 1-34.
9. Cobb HG, Grefenstette JJ (1993) Genetic algorithms for tracking changing environments. In: Proceedings of the 5th international conference on genetic algorithms, pp 523-530.
10. De Jong ED, Watson RA, Pollack JB (2001) Reducing bloat and promoting diversity using multi-objective methods. In: Proceedings of the 2001 genetic and evolutionary computation conference, Morgan Kaufmann, San Francisco, CA, pp 11-18.
11. De Jong KA (1975) An analysis of the behavior of a class of genetic adaptive systems. PhD thesis, University of Michigan, MI, USA.
12. Eiben AE, Smith JE (2003) Introduction to evolutionary computing (Natural Computing Series). Springer, Berlin.
13. Eshelman LJ (1991) The CHC adaptive search algorithm: How to have safe search when engaging in nontraditional genetic recombination. In: Proceedings of the 1st workshop on foundations on genetic algorithms, pp 257-266.
14. Eshelman LJ, Schaffer JD (1991) Preventing premature convergence in genetic algorithms by preventing incest. In: Belew R, Booker L (eds) Proceedings of the fourth international conference on genetic algorithms, Morgan Kaufman, San Mateo, CA, pp 115-122.
15. Fogel D (1995) Evolutionary computation: toward a new philosophy of machine intelligence. IEEE Press, New York.
16. Fogel D, Owens A, Wals MJ (1966) Artificial intelligence through simulated evolution. Wiley, New York.
17. Fogel DB, Fogel LJ, Porto VW (1990) Evolving neural networks. Biol Cybern 63(6): 487-493. 10. 1007/BF00199581.
18. Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Reading.
19. Goldberg DE, Deb K (1990) A comparative analysis of selection schemes used in genetic algorithms. In: Proceedings of the first workshop on foundations of genetic algorithms, Bloomington Campus, Indiana, USA, pp 69-93.
20. Goldberg DE, Richardson J (1987) Genetic algorithms with sharing for multi-modal function optimization. In: Proceedings of the 2nd international conference on genetic algorithms, Lawrence Erlbaum Associates, pp 41-49.
21. Goldberg DE, Deb K, Clark JH (1992) Genetic algorithms, noise, and the sizing of populations. Complex Syst 6(4): 333-362.
22. Greewood GW, Fogel GB, Ciobanu M (1999) Emphasizing extinction in evolutionary programming. In: Angeline P, Michalewicz Z, Schoenauer M, Yao X, Zalzala A (eds) Proceedings of the 1999 congress on evolutionary computation (CEC99), pp 666-671.
23. Guillén A, Rojas I, González J, Pomares H, Herrera L, Valenzuela O, Rojas F (2007) Output value-based initialization for radial basis function neural networks. Neural Process Lett 25(3): 209-225.
24. Gutiérrez PA, Hervás-Martínez C, Lozano M (2007) Saw-tooth algorithm guided by the variance of best individual distributions for designing evolutionary neural networks. In: Proceedings of the 8th international conference on intelligent data engineering and automated learning (IDEAL'07), Springer, Birmingham, Lecture Notes in Computer Science, vol 4881, pp 1131-1140.
25. Herrera F, Lozano M (1996) Adaptation of genetic algorithm parameters based on fuzzy logic controllers. In: Herrera F, Verdegay JL (eds) Genetic algorithms and soft computing, Physica-Verlag, Heidelberg, pp 95-125.
26. Herrera F, Lozano M, Verdegay JL (1996) Dynamic and heuristic fuzzy connectives-based crossover operators for controlling the diversity and convergence of real-coded genetic algorithms. Int J Intell Syst 11(12): 1013-1040.
27. Hervás C, Martínez-Estudillo FJ, Gutiérrez PA (2006) Classification by means of evolutionary product-unit neural networks. In: Proceedings of the 2006 international joint conference on neural networks, Vancouver, Canada, pp 2834-2842.
28. Hutter M, Legg S (2006) Fitness uniform optimization. IEEE Trans Evol Comput 10(5): 568-589.
29. Koumousis VK, Dimou CK (2002) The effect of oscillating population size on the performance of genetic algorithms. In: Proceedings of the 4th GRACM congress on computational mechanics.
30. Koumousis VK, Katsaras CP (2006) A saw-tooth genetic algorithm combining the effects of variable population size and reinitialization to enhance performance. IEEE Trans Evol Comput 10(1): 19-28.
31. Krishnakumar K (1989) Micro-genetic algorithms for stationary and non-stationary function optimization. In: Proceedings of the SPIE: intelligent control and adaptive systems, vol 1196, pp 289-296.
32. Landwehr N, Hall M, Frank E (2005) Logistic model trees. Mach Learn 59(1-2): 161-205.
33. Leung FHF, Lam HK, Ling SH, Tam PKS (2003) Tuning of the structure and parameters of a neural network using an improved genetic algorithm. IEEE Trans Neural Netw 14(1): 79-88. doi: 10. 1109/TNN. 2002. 804317.
34. Levene H (1960) Essays in Honor of Harold Hotelling. In: Contributions to probability and statistics. Stanford University Press, London.
35. Lozano M, Herrera F, Cano JR (2008) Replacement strategies to preserve useful diversity in steady-state genetic algorithms. Inf Sci 178(23): 4421-4433.
36. Mahfoud SW (1992) Crowding and preselection revisited. In: Männer R, Manderick B (eds) Parallel problem solving from nature 2, North-Holland, Amsterdam, pp 27-36.
37. Maniezzo V (1993) Searching among search spaces: hastening the genetic evolution of feedforward networks. In: Proceedings of the conference on artificial neural nets and genetic algorithms.
38. Martínez-Estudillo AC, Hervás-Martínez C, Martínez-Estudillo FJ, García-Pedrajas N (2006a) Hybridization of evolutionary algorithms and local search by means of a clustering method. IEEE Trans Syst Man Cybern B 36(3): 534-545.
39. Martínez-Estudillo AC, Martínez-Estudillo FJ, Hervás-Martínez C, García N (2006b) Evolutionary product unit based neural networks for regression. Neural Netw 19(4): 477-486.
40. Martínez-Estudillo FJ, Hervás-Martínez C, Gutiérrez PA, Martínez-Estudillo AC (2008) Evolutionary product-unit neural networks classifiers. Neurocomputing 72(1-2): 548-561.
41. McDonnell JR, Waagen DE (1993) Evolving recurrent perceptrons. In: Ruck DW (ed) Proceedings of the SPIE conference, Science of Artificial Neural Networks II, vol 1966, pp 350-361.
42. Miller RG (1986) Beyond ANOVA, basics of applied statistics. Wiley, London.
43. Oppacher F, Wineberg M (1999) The shifting balance genetic algorithm: improving the GA in a dynamic environment. In: Proceedings of the genetic and evolutionary computation conference, Morgan Kaufmann, San Francisco, vol 1, pp 504-510.
44. Palmes PP, Hayasaka T, Usui S (2005) Mutation-based genetic neural network. IEEE Trans Neural Netw 16(3): 587-600.
45. Prechelt L (1994) PROBEN1: a set of neural network benchmark problems and benchmarking rules. Tech. Rep. 21/94, Fakultät für Informatik (Universität Karlsruhe).
46. Prügel-Bennett A, Shapiro JL (1994) An analysis of genetic algorithms using statistical mechanics. Phys Rev Lett 72(9): 1305-1309.
47. Prügel-Bennett A, Shapiro JL (1997) The dynamics of a genetic algorithm for simple Ising systems. Physica D 104: 75-114.
48. Rattray M (1996) Modeling the dynamics of genetic algorithms using statistical mechanics. PhD thesis, Manchester University, Manchester, UK.
49. Rechenberg I (1973) Evolutionsstrategie: optimierung technischer systeme nach prinzipien der biologischen evolution. Frommann-Holzboog, Stuttgart.
50. Rogers A, Pruegel-Bennett A (1999) Genetic drift in genetic algorithm selection schemes. IEEE Trans Evol Comput 3(4): 298.
51. Shimodaira H (2001) A diversity-control-oriented genetic algorithm DCGA: performance in function optimization. In: Proceedings of the 2001 congress on evolutionary computation, 2001, vol 1, pp 44-51. doi: 10. 1109/CEC. 2001. 934369.
52. Smith RE (1993) Adaptively resizing populations: an algorithm and analysis. In: Proceedings of the fifth international conference on genetic algorithms, pp 653-653.
53. Snedecor GW, Cochran WG (1989) Statistical methods, 6th edn. Iowa State University Press.
54. Tsai JT, Chou JH, Liu TK (2006) Tuning the structure and parameters of a neural network by using hybrid taguchi-genetic algorithm. IEEE Trans Neural Netw 17(1): 69-80.
55. Tsutsui S, Fujimoto Y, Ghosh A (1997) Forking genetic algorithms: GAs with search space division schemes. Evol Comput 5(1): 61-80.
56. Ursem RK (2002) Diversity-guided evolutionary algorithms. In: Proceedings of parallel problem solving from nature VII (PPSN-2002), Springer, Berlin, pp 462-471.
57. Ventura S, Romero C, Zafra A, Delgado JA, Hervás C (2008) JCLEC: a Java framework for evolutionary computation. Soft Comput 12(4): 381-392.
58. Wang L, Zheng DZ, Tang F (2002) An improved evolutionary programming for optimization. In: Proceedings of the 4th World Congress on Intelligent Control and Automation, vol 3, pp 1769-1773.
59. Whitley D (1989) The GENITOR algorithm and selection pressure: why rank-based allocation of reproductive trials is best. In: Schaffer JD (ed) Proceedings of the third international conference on genetic algorithms, Morgan Kaufman, San Mateo, CA.
60. Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1(1): 67-82.
61. Yao X (1997) Global optimization by evolutionary algorithms. In: Proceedings of the 2nd Aizu international symposium on parallel algorithms/architecutre synthesis, (pAs'97), Aizu-Wakamatsu, Japan, pp 282-291.
62. Yao X (1999) Evolving artificial neural networks. Proc IEEE 87(9): 1423-1447.
63. Yao X, Liu Y (1997) A new evolutionary system for evolving artificial neural networks. IEEE Trans Neural Netw 8(3): 694-713.