Genetic programming discovers efficient learning rules for the hidden and output layers of feedforward neural networks

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 1598, Issue , 1999, Pages 120-134

  Radi, Amr ^a   Poli, Riccardo ^a  

^a UNIVERSITY OF BIRMINGHAM   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

BACKPROPAGATION ALGORITHMS; FEEDFORWARD NEURAL NETWORKS; GENETIC ALGORITHMS; GENETIC PROGRAMMING; MACHINE LEARNING; MULTILAYER NEURAL NETWORKS;

EFFICIENT LEARNING; GENERALISATION; HIDDEN LAYERS; INITIAL CONDITIONS; LEARNING METHODS; LIMITED TRAINING DATA; SLOW CONVERGENCES; TRAINING ALGORITHMS;

LEARNING ALGORITHMS;

EID: 84956857243     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/3-540-48885-5_10     Document Type: Conference Paper

References (57)

1. R. Ahmad and F. M. A. Salam. Error back propagation learning using the polynomial energy function. In Proceedings of the International Joint Conference on Neural Networks, Iizuka, Japan, 1992.
2. Pierre Baldi. Gradient learning algorithm overview: A general dynamical systems perspective. IEEE Transactions on Neural Networks, 6(1):182-195, Jan 1995.
3. S. Bengio, Y. Bengio, and J. Cloutier. Use of genetic programming for the search of a learning rule for neural networks. In Proceedings of the First Conference on Evolutionary Computation,IEEE World Congress on Computational Intelligence, Orlando-Florida, USA, pages 324-327, 1994.
4. D. J. Chalmers. The evolution of learning: An experiment in genetic connectionism. In Connectionist Models Summer School. San Mateo, CA., 1990.
5. X-H Yu G-A Chen. Efficient backpropagation learning using optimal learning rate and momentum. Neural Networks, 10(3):517-527, 1997.
6. Vladimir Cherkassky and Robert Shepherd. Regularization effect of weight initialization in back propagation networks. In 1998 IEEE International J. Conference of Neural Networks (IJCNN’98), Anchorage, Alaska, pages 2258-2261. IEEE, May 1998.
7. C Schittenkopf G Deco and W Brauer. Two strategies to avoid overfitting in feedforward neural networks. Neural Networks, 10(3):505-516, 1997.
8. D. H. Deterding. Speaker Normalisation for Automatic Speech Recognition. PhD thesis, University of Cambridge, 1989.
9. A. Harry Eaton and L. Tracy Oliver. Improving the convergence of the back propagation algorithm. Neural Networks, 5:283-288, 1992.
10. S. E. Fahlman. An empirical study of learning speed in back propagation networks. Technical report, CMU-CS-88-162, Carnegie Mellon University, Pittsburgh, PA., 1988.
11. M. Gori and A. Tesi. On the problem of local minima in backpropagation. IEEE Transactions on PAMI, 14(1):76-86, 1992.
12. R. P. Gorman and T.J Sejnowski. Analysis of hidden units in a layered network trained to classify sonar targets. Neural Networks, 1:75-89, 1988.
13. S. Haykin. Neural Networks: A Comprehensive Foundation. IEEE Society Press, Macmillan College Publishing, New York 10022, 1994.
14. Donald O. Hebb. The Organisation of Behaviour: A Neuropsychological Theory. New York: Wiley, 1949.
15. Yoshio Hirose, Koichi Yamashit, and Shimpei Hijiya. Back propagation algorithm which varies the number of hidden units. Neural Networks, 4:61-66, 1991.
16. J. H. Holland. Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor, Michigan, 1975.
17. M. J. J. Holt and S. Semnani. Convergence of back-propagation in neural networks using a log-likelihood cost function. Electronics Letters, 26(23):1964-1965, 1990.
18. Robert A. Jacobs. Increased rates of convergence through learning rate adaptation. Neural Networks, 1:295-307, 1988.
19. K. Kruschke John and R. Javier Movellan. Fast learning algorithms for neural networks. IEEE Transactions on Circuits and Systems-II: Analogy and Digital Signal Processing, 39(7):453-473, 1992.
20. H. Kitano. Neurogenetic learning: an integrated method of designing and training neural networks using genetic algorithms. Physica D, 75:225-238, 1994.
21. John R. Koza. Genetic Programming: on the programming of computers by means of natural selection. MIT Press, Cambridge, Massachussetts, 1992.
22. John R. Koza. Genetic Programming II: Automatic discovery of reusable programs. The MIT Press, Cambridge, Massachussetts, 1994.
23. John K. Kruschke and Javier R. Movellan. Benefits of gain: Speeded learning and minimal hidden layers in back propagation networks. IEEE Transactions on Systems, Man and Cybernetics., 21(1), 1991.
24. T Denoeux R Lengelle. Initializing back propagation networks using prototypes. Neural Networks, 6(3):351-363, 1993.
25. L.E Scales. Introduction to non-linear optimization. New York:Springer-Verlag, 1985.
26. R. Linsker. From basic network principles to neural architecture: Emergence of spatial-opponent cells. In Proceedings of the National Acedemy of Science USA, volume 83, pages 7508-7512, 1986.
27. F. A. lodewykWessels and Etienne Barnard. Avoiding false local minima by proper initialisation of connections. IEEE Transactions on Neural Networks, 3(6):899-905, 1992.
28. Howard Demuth Martin Hagan and Mark Beale. Neural Network Design. PWS Publishing Company, Boston, MA 02116, 96.
29. Rangachari Anand, Kishan Mehrotra, Chilukuri Mohan and Sanjay Ranka. An impproved algorithm for neural network classification of imbalanced training sets. IEEE Transactions on Neural Networks, 4(6):962-969, 1993.
30. Martin F. Moller. A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks, 6:525-533, 1993.
31. D. J. Montana and L. Davis. Training feedforaward neural networks using genetic algorithms. In Proceedings of Eleventh International Joint Conference on Artifi- cial Intelligence (IJCAI-89), Detroit, MI, pages 762-767. Morgan Kaufmann, Palo Alto, CA, 1989.
32. Taek Mu and Hui Cheng. Contrast enhancement for backpropagation. IEEE Transactions on Neural Networks, 7(1):515-524, 1996.
33. A. Van Ooyen and B Nienhuis. Improving the convergence of the back propagation algorithm. Neural Networks, 5:465-471, 1992.
34. Alexander G. Parlos, B. Fermandez, Amir Atiya, J. Muthusami, and Wei K Tsai. An accelerated learning algorithm for multilayer perceptron networks. IEEE Transactions on Neural Networks, 5(3):493-497, 1994.
35. S. J. Perantonis and D. A. Karras. An efficient constrained training algorithm for feedforward networks. IEEE Transactions on Neural Networks, 6(6):237-149, Nov 1995.
36. J. Pujol and R. Poli. Efficient evolution of asymmetric recurrent neural networks using a pdgp-inspired two-dimensional representation. In the First European Workshop on Genetic Programming(EUROGP’98), Paris, Springer Lectre Notes in Computer Science, volume 1391, pages 130-141, 1998.
37. J. Pujol and R. Poli. Evolving neural networks using a dual representation with a combined crossover operator. In the IEEE International Conference on Evolutionary Computation (ICEC’98), Anchorage, Alaska, pages 416-421, 1998.
38. J. Pujol and R. Poli. Evolving the topology and the weights of neural networks using a dual representation. Special Issue on Evolutionary Learning of the Applied Intelligence Journal, 8(1):73-84, 1998.
39. Amr Radi and Riccardo Poli. Discovery of backpropagation learning rules using genetic programming. In 1998 IEEE International Conference of Evolutionary Computational (ICEC’98), Anchorage, Alaska, pages 371-375. IEEE, May 1998.
40. Amr Radi and Riccardo Poli. Genetic programming can discover fast and general learning rules for neural networks. In Third Annual Genetic Programming Conference (GP’98), Madison, Wisconsin, pages 314-323. Morgan Kaufmann, July 1998.
41. Martin Riedmiller. Advanced supervised learning in multi-layer perceptrons from backpropagation to adaptive learning algorithms. Computer Standards and Interfaces Special Issue on Neural Networks, 16(3):265-275, 1994.
42. Martin Riedmiller and Heinrich Braun. A direct method for faster backpropagation learning: The RPROP Algorithm. IEEE International Conference on Neural Networks 1993 (ICNN93), San Francisco, pages 586-591, 1993.
43. D. E. Rumelhart, G. E. Hinton, and R. J Williams. Parallel Distributed Processing. MIT Press, Cambridge, MA, 1986.
44. Dilip Sarkar. Methods to speed up error back propagation learning algorithm. ACM Computing Surveys, 27(4):519-542, 1995.
45. Wolfram Schiffmann and Merten Joost. Speeding up backpropagation algorithm by using cross-entropy combined with pattern normalization. International Journal of Uncertainity, Fuzziness and Knowledge-based Systems (IJUFKS), 6(2):177-126, 1998.
46. Femando F. Silva and Luis B. Almeida. Speeding Backpropagation, pages 151-158. Advanced Neural Computers, Elsevier Science Publishers B.V. (North-Holland), 1990.
47. Sara A. Solla, Esther Levin, and Michael Fleisher. Accelerated learning in layered neural networks. Complex System, 2:625-640, 1988.
48. Alessandro Sperduti and Antonina Starita. Speed up learning and network optimization with extended back propagation. Neural Networks, 6:365-383, 1993.
49. Ramana Vitthal P. Sunthar and Ch.Durgaprasada Rao. The generalized proportional-integral-derivative (pid)gradient descent back propagation algorithm. Neural Networks, 8(4):563-569, 1995.
50. J. G. Taylor. The Promise of neural networks. Springer-Verlag, London, 1993.
51. Tom Tollenaere. Super SUB:Fast adaptive back propagation with good scaling properties. Neural Networks, 1:561-573, 1990.
52. W.T Zink, T.P Vogl, J.K. Zigler and D.L Alkon. Accelerating the convergence of the backpropagation method. Biological Cybernetics, 59:256-264, Sept. 1988.
53. Michael K. Weir. A method for self determination of adaptive learning rate in back propagation. Neural Networks, 4:(371-379), 1991.
54. D. Whitley and T. Hanson. Optimizing neural networks using faster, more accurate genetic search. In J. D. Schaffer, editor, Third International Conference on Genetic Algorithms, Georg Mason University, pages 391-396. Morgan Kaufmann, 1989.
55. M. Spears William, K. A. De Jong, T. Baeck, David Fogel, and Hugo de Garis. An overview of evolutionary computation. In Proceedings of the European Conference on Machine Learning, pages 442-459, 1993.
56. Merten Joost Wolfram Schiffmann and R. Werner. Optimisation of the backpropagation algorithm for training multilayer perceptrons. Technical report 16/1992, University of Koblenz, Institute of Physics, 1992.
57. Byoung-Tak Zhang. Accelerated learning by active example selection. International Journal of Neural Systems, 5(1):67-75, 1994.