Understanding the behaviour and optimising the performance of back-propagation neural networks: An empirical study

Environmental Modelling and Software

Volumn 13, Issue 2, 1998, Pages 179-191

  Maier, Holger R ^a   Dandy, Graeme C ^a  

^a UNIVERSITY OF ADELAIDE   (Australia)

Author keywords

Artificial Neural Networks; Back propagation algorithm; Generalisation ability; Learning speed; Local minima; Optimisation; Stages of learning

Indexed keywords

ARTIFICIAL NEURAL NETWORK; ENVIRONMENTAL MODELING;

BACKPROPAGATION NEURAL NETWORKS; GENERALIZATION ABILITY;

BACKPROPAGATION; ENVIRONMENTAL ENGINEERING; LEARNING ALGORITHMS; OPTIMIZATION;

NEURAL NETWORKS;

EID: 0032034197     PISSN: 13648152     EISSN: None     Source Type: Journal    
DOI: 10.1016/S1364-8152(98)00019-X     Document Type: Article

References (34)

1. Amari S.-I., Murata N., Müller K.-R., Finke M., Yang H.H. Asymptotic statistical theory of overtraining and cross-validation. IEEE Transactions on Neural Networks. 8(5):1997;985-996.
2. Battiti R. First- and second-order methods for learning: between steepest descent and Newton's method. Neural Computation. 4:1992;141-166.
3. Breiman L. Comment on 'Neural networks: a review from a statistical perspective' by B. Cheng and D.M. Titterington. Statistical Science. 9(1):1994;38-42.
4. Cheung, R.K.M., Lustig, I., Kornhauser, A.L., 1990. Relative effectiveness of training set patterns for back propagation. Proceedings of the IJCNN International Joint Conference on Neural Networks, San Diego, California, I673-I678.
5. Clair T.A., Ehrman J.M. Variations in discharge and dissolved organic carbon and nitrogen export from terrestrial basins with changes in climate: a neural network approach. Limonl. Oceanogr. 41(5):1996;921-927.
6. Comrie A.C. Comparing neural networks and regression models for ozone forecasting. J. Air Waste Manage. Assoc. 47(6):1997;653-663.
7. Fletcher, R., 1987. Practical Methods of Optimization. Wiley, New York.
8. French M.N., Krajewski W.F., Cuykendall R.R. Rainfall forecasting in space and time using a neural network. Journal of Hydrology. 137:1992;1-31.
9. Goldberg, D.E., 1989. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, New York.
10. Golden, R.M., 1996. Mathematical Methods for Neural Network Analysis and Design. MIT Press, Cambridge.
11. Hassoun, M.H., 1995. Fundamentals of Artificial Neural Networks. MIT Press, Cambridge.
12. Holland, J.H., 1992. Adaptation in Natural and Artificial Systems. MIT Press, Cambridge.
13. Hubick, K.T., 1992. Artificial Neural Networks in Australia. Department of Industry, Technology and Commerce, Commonwealth of Australia, Canberra.
14. Jacobs R.A. Increased rates of convergence through learning rate adaption. Neural Networks. 1:1988;295-307.
15. Khotanzad A., Afkhami-Rohani R., Lu T.-L., Abaye A., Davis M., Maratukulam D.J. ANNSTLF - a neural-network-based electric load forecasting system. IEEE Transactions on Neural Networks. 8(4):1997;835-846.
16. Kirkpatrick S. Optimization by simulated annealing: quantitative studies. Journal of Statistical Physics. 34:1984;975-986.
17. Kirkpatrick S., Gilatt C.D., Vecchi M.P. Optimization by simulated annealing. Science. 220:1983;671-680.
18. Maier H.R., Dandy G.C. The use of artificial neural networks for the prediction of water quality parameters. Water Resources Research. 32(4):1996;1013-1022.
19. Maier H.R., Dandy G.C., Burch M.D. Use of artificial neural networks for modelling cyanobacteria Anabaena spp. in the River Murray, South Australia. Ecol. Model. 105(2/3):1998;257-272.
20. Maren, A., Harston, C., Pap, R., 1990. Handbook of Neural Computing Applications. Academic Press, San Diego.
21. Masters, T., 1993. Practical Neural Network Recipes in C++. Academic Press, San Diego.
22. Matyas J. Random optimization. Automation and Remote Control. 26:1965;246-253.
23. Minai, A.A., Williams, R.D., 1990. Back-propagation heuristics: a study of the extended delta-bar-delta algorithm. Proceedings of the IJCNN International Joint Conference on Neural Networks, San Diego, California, I595-I600.
24. Minns A.W., Hall M.J. Artificial neural networks as rainfall-runoff models. Hydrological Sciences Journal - Journal des Sciences Hydrologiques. 41(3):1996;399-417.
25. NeuralWare, Inc., 1991. Neural Computing, NeuralWorks Professional II/Plus and NeuralWorks Explorer.
26. Paruelo J.M., Tomasel F. Prediction of functional characteristics of ecosystems - a comparison of artificial neural networks and regression models. Ecological Modelling. 98(2)(3):1997;173-186.
27. Rumelhart, D.E., Hinton, G.E., Williams, R.J., 1986. Learning internal representations by error propagation. In: Rumelhart, D.E., McClelland, J.L. (Eds.) Parallel Distributed Processing, Vol. 1. MIT Press, Cambridge.
28. Schizas C.N., Pattichis C.S., Michaelides S.C. Forecasting minimum temperature with short time-length data using artificial neural networks. Neural Network World. 4(2):1994;219-230.
29. Shanno D.F. Conjugate gradient methods with inexact line searches. Mathematics of Operations Research. 3:1978;244-256.
30. Silva, F.M., Almeida, L.B., 1990. Acceleration techniques for the backpropagation algorithm. Proceedings of the EURASIP Workshop, Sesimbra, Portugal, pp. 110-119.
31. Solis F.J., Wets R.J.-B. Minimization by random search techniques. Mathematics of Operations Research. 6(1):1981;19-30.
32. Stone M. Cross-validatory choice and assessment of statistical predictions. Journal of the Royal Statistical Society B. 36:1974;111-147.
33. Tawfik M., Ibrahim A., Fahmy H. Hysteresis sensitive neural network for modeling rating curves. Journal of Computing in Civil Engineering. 11(3):1997;206-211.
34. Venkatesan C., Raskar S.D., Tambe S.S., Kulkarni B.D., Keshavamurty R.N. Prediction of all India summer monsoon rainfall using error-back-propagation neural networks. Meteorology and Atmospheric Physics. 62(3)(4):1997;225-240.