Fast Convergent Generalized Back-Propagation Algorithm with Constant Learning Rate

Neural Processing Letters

Volumn 9, Issue 1, 1999, Pages 13-23

  Ng, S C ^a   Leung, S H ^b   Luk, A ^b  

^a Hong Kong Tech Coll (Tsing Yi)   (Hong Kong)

^b CITY UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Constant learning rate; Convergence; Feedforward neural networks; Generalized back propagation; Gradient descent algorithm

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; BACKPROPAGATION; CONVERGENCE OF NUMERICAL METHODS; FEEDFORWARD NEURAL NETWORKS; ORDINARY DIFFERENTIAL EQUATIONS;

GENERALIZED BACKPROPAGATION ALGORITHMS; GRADIENT DESCENT ALGORITHMS;

LEARNING SYSTEMS;

EID: 0033077409     PISSN: 13704621     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1018611626332     Document Type: Article

References (11)

1. D.E. Rumelhart, G.E. Hinton and R.J. Williams, "Learning internal representations by error propagation", in D.E. Rumelhart and J.L. McClelland (eds), Parallel Distributed Processing, Vol. 1, pp. 318-362, Cambridge, Mass: M.I.T. Press, 1986.
2. Marco Gori and Alberto Tesi, "On the problem of local minima in back-propagation", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 14, No. 1, pp. 76-86, 1992.
3. E.K. Blum, "Approximation of Boolean functions by sigmoidal networks: Part I: XOR and other two-variable functions", Neural Computation, Vol. 1, No. 4, pp. 532-540, 1989.
4. A. van Ooyen and B. Nienhuis, "Improving the convergence of the back-propagation algorithm", Neural Networks, Vol. 5, pp. 465-471, 1992.
5. M. Ahmad and F.M.A. Salam, "Supervised learning using the cauchy energy function", International Conference on Fuzzy Logic and Neural Networks, 1992.
6. R.A. Jacobs, "Increased rates of convergence through learning rate adaptation", Neural Networks, Vol. 1, pp. 295-307, 1988.
7. M.K. Weir, "A method for self-determination of adaptive learning rates in back propagation", Neural Networks, Vol. 4, pp. 371-379, 1991.
8. X.H. Yu, G.A. Chen and S.X. Cheng, "Acceleration of backpropagation learning using optimised learning rate and momentum", Electronics Letters, Vol. 29, No. 14, pp. 1288-1289, 8 July 1993.
9. C.M. Kuan and K. Hornik, "Convergence of learning algorithms with constant learning rates", IEEE Trans. on Neural Networks, Vol. 2, No. 5, pp. 484-489, Sept. 1991.
10. P. Billingsley, Convergence of Probability Measures, New York: Wiley, 1968.
11. B. Verma, "Fast training of multilayer perceptrons", IEEE Trans, on Neural Networks, Vol. 8, No. 6, pp. 1314-1320, November 1997.