Towards the optimal learning rate for backpropagation

Neural Processing Letters

Volumn 11, Issue 1, 2000, Pages 1-5

  Mandic, Danilo P ^a   Chambers, Jonathon A ^b  

^a UNIVERSITY OF EAST ANGLIA   (United Kingdom)

^b IMPERIAL COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE ALGORITHMS; ADAPTIVE FILTERING; BACKPROPAGATION; DIGITAL FILTERS; ERRORS; LEARNING ALGORITHMS; LEARNING SYSTEMS;

ADAPTIVE LEARNING RATE; NORMALISED LEAST MEAN SQUARE; OPTIMAL GRADIENT LEARNING; TAYLOR SERIES EXPANSION;

FEEDFORWARD NEURAL NETWORKS;

EID: 0034140070     PISSN: 13704621     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1009686825582     Document Type: Article

References (12)

1. Ljung, L. and Soderstrom, T.: Theory and Practice of Recursive Identification, MIT Press, Cambridge, MA., 1983.
2. Treichler, J. R., Johnson, Jr., C. R. and Larimore, M. G.: Theory and Design of Adaptive Filters, John Wiley & Sons, New York, 1987.
3. Kwong, R. H. and Johnston, E. W.: A variable step size LMS algorithm, IEEE Transactions on Signal Processing 40(7) (1992), 1633-1641.
4. Evans, J. B., Xue, P. and Liu, B.: Analysis and implementation of variable step size adaptive algorithms, IEEE Transactions on Signal Processing 41(8) (1993), 2517-2535.
5. Mathews, V. J. and Xie, Z.: A stochastic gradient adaptive filter with gradient adaptive step size, IEEE Transactions on Signal Processing 41(6) (1993), 2075-2087.
6. Shan, T. J. and Kailaith, T.: Adaptive algorithms with an automatic gain control feature, IEEE Transactions on Acoustics, Speech and Signal Processing 35(1) (1988), 122-127.
7. Aboulnasr, T. and Mayyas, K.: A robust variable step-size LMS-type algorithm: Analysis and simulations, IEEE Transactions on Signal Processing 45(3) (1997), 631-639.
8. Haykin, S.: Adaptive Filter Theory, Prentice-Hall, 3d ed., Englewood Cliffs, NJ, 1996.
9. Jacobs, R. A.: Increased rates of convergence through learning rate adaptation, Neural Networks 1 (1988), 295-307.
10. Haykin, S.: Neural Networks - A Comprehensive Foundation, Prentice Hall, Englewood Cliffs, NJ, 1994.
11. Douglas, S. C. and Cichocki, A.: On-line step-size selection for training of adaptive systems, IEEE Signal Processing Magazine 14(6) (1997), 45-46.
12. Mandic, D. P. and Chambers, J. A.: A posteriori real time recurrent learning schemes for a recurrent neural network based non-linear predictor. IEE Proceedings-Vision, Image and Signal Processing 145(6) (1998), 365-370.