The effect of internal parameters and geometry on the performance of back-propagation neural networks: An empirical study

Environmental Modelling and Software

Volumn 13, Issue 2, 1998, Pages 193-209

  Maier, Holger R ^a   Dandy, Graeme C ^a  

^a UNIVERSITY OF ADELAIDE   (Australia)

Author keywords

Artificial neural networks; Back propagation algorithm; Generalisation ability; Geometry; Internal parameters; Learning speed; Network design

Indexed keywords

BACKPROPAGATION; ENVIRONMENTAL ENGINEERING; LEARNING ALGORITHMS;

BACKPROPAGATION NEURAL NETWORKS; GENERALIZATION ABILITY;

NEURAL NETWORKS;

ARTIFICIAL NEURAL NETWORK; ENVIRONMENTAL MODELING;

EID: 0032051569     PISSN: 13648152     EISSN: None     Source Type: Journal    
DOI: 10.1016/S1364-8152(98)00020-6     Document Type: Article

References (75)

1. Amari S.-i., Murata N., Müller K.-R., Finke M., Yang H.H. Asymptotic statistical theory of overtraining and cross-validation. IEEE Transactions on Neural Networks. 8(5):1997;985-996.
2. Battiti R. Accelerated back-propagation learning: two optimization methods. Complex Systems. 3:1989;331-342.
3. Battiti R. First- and second-order methods for learning: between steepest descent and Newton's method. Neural Computation. 4:1992;141-166.
4. Baum E.B., Haussler D. What size net gives valid generalization? Neural Computation. 1:1989;151-160.
5. Bebis G., Georgiopoulos M. Feed-forward neural networks: why network size is so important. IEEE Potentials. October:1994;27-31.
6. Breiman L. Comment on 'Neural networks: a review from a statistical perspective' by B. Cheng and D.M. Titterington. Statistical Science. 9(1):1994;38-42.
7. Burke L.I., Ignizio J.P. Neural networks and operations research: an overview. Computer and Operations Research. 19(3/4):1992;179-189.
8. Chakraborty K., Mehrotra K., Mohan C.K., Ranka S. Forecasting the behaviour of multivariate time series using neural networks. Neural Networks. 5:1992;961-970.
9. Chatfield C. Neural networks: forecasting breakthrough or just a passing fad? International Journal of Forecasting. 9:1993;1-3.
10. Cheng B., Titterington D.M. Neural networks: a review from a statistical perspective. Statistical Science. 9:1994;2-54.
11. Chung F.L., Lee T. A node pruning algorithm for backpropagation networks. International Journal of Neural Systems. 3:1992;301-314.
12. Connor J.T., Martin R.D., Atlas L.E. Recurrent neural networks and robust time series prediction. IEEE Transactions of Neural Networks. 5(2):1994;240-254.
13. Cottrell, G.W., Fleming, M., 1990. Face recognition using unsupervised feature extraction. Proceedings of the International Neural Networks Conference, vol. 1. Paris, France, pp. 322-325.
14. Cybenko G. Approximation by superpositions of a sigmoidal function. Mathematical Control Systems and Signaling. 2:1989;303-314.
15. Dai H.C., Macbeth C. Effects of learning parameters on learning procedure and performance of a BPNN. Neural Networks. 10(8):1997;1505-1521.
16. Dandy G.C., Crawley P.D. Optimisation of multiple reservoir systems including salinity effects. Water Resources Research. 28(4):1992;979-990.
17. Daniell, T.M., Wundke, A.D., 1993. Neural networks - assisting in water quality modelling. Preprints, Watercomp, Melbourne, Australia, pp. 51-57.
18. De Silets L., Golden B., Wang Q., Kumar R. Predicting salinity in the Chesapeake Bay using backpropagation. Computer and Operations Research. 19(3/4):1992;277-285.
19. De Veaux R.D., Darken C.J., Ungar L.H. Comment on 'Neural networks and related methods for classification' by B.D. Ripley. Journal of the Royal Statistical Society, B. 56(3):1994;446-447.
20. Fahlman, S.E., Lebiere, C., 1990. The cascade-correlation learning architecture. In: Touretzky, D. (Ed.), Advances in Neural Information Processing Systems 2. Morgan Kaufmann, San Mateo.
21. Flood I., Kartam N. Neural networks in civil engineering. I: Principles and understanding. Journal of Computing in Civil Engineering. 8(2):1994;131-148.
22. Gençay R., Liu T. Nonlinear modelling and prediction with feedforward and recurrent networks. Physica D. 108(1):1997;119-134.
23. Golden, R.M., 1996. Mathematical Methods for Neural Network Analysis and Design. MIT Press, Cambridge.
24. Goswami P., Srividya A novel neural network design for long range prediction of rainfall pattern. Current Science. 70(6):1996;447-457.
25. Hansen J.V., Nelson R.D. Neural networks and traditional time series methods: a synergistic combination in state economic forecasts. IEEE Transactions on Neural Networks. 8(4):1997;863-873.
26. Hassoun, M.H., 1995. Fundamentals of Artificial Neural Networks. MIT Press, Cambridge.
27. Hegazy T., Fazio P., Moselhi O. Developing practical neural network applications using back-propagation. Microcomputers in Civil Engineering. 9:1994;145-159.
28. Hertz, J.A., Krogh, A., Palmer, R.G., 1991. Introduction to the Theory of Neural Computation. Addison-Wesley Publishing Company, Redwood City, California.
29. Hill T., Marquez L., O'Connor M., Remus W. Artificial neural network models for forecasting and decision making. International Journal of Forecasting. 10:1994;5-15.
30. Hochreiter S., Schmidhuber J. Long short-term memory. Neural Computation. 9:1997;1735-1780.
31. Hush, D.R., Salas, J.M., Horne, B., 1991. Error surfaces for multi-layer perceptrons. International Joint Conference on Neural Networks, Seattle, pp. 759-764.
32. Jacobs R.A. Increased rates of convergence through learning rate adaptation. Neural Networks. 1:1988;295-307.
33. Karunanithi N., Grenney W.J., Whitley D., Bovee K. Neural networks for river flow prediction. Journal of Computing in Civil Engineering. 8(2):1994;201-220.
34. Khotanzad, A., Lu, J.-H., 1991. Shape and texture recognition by a neural network. In: Sethi, I.K., Jain, A.K. (Eds.), Artificial Neural Networks and Statistical Pattern Recognition - Old and New Connections. Elsevier Science Publishers B.V., Amsterdam.
35. Khotanzad A., Afkhami-Rohani R., Lu T.-L., Abaye A., Davis M., Maratukulam D.J. ANNSTLF - a neural-network-based electric load forecasting system. IEEE Transactions on Neural Networks. 8(4):1997;835-846.
36. Krishnapura V.G., Jutan A. ARMA neuron networks for modeling nonlinear dynamical systems. Canadian Journal of Chemical Engineering. 75(3):1997;574-582.
37. Kudrycki, T.P., 1988. Neural network implementation of a medical diagnosis expert system. Master's Thesis, College of Engineering, University of Cincinnati.
38. Kwok T.-Y., Yeung D.-Y. Constructive algorithms for structure learning in feedforward neural networks for regression problems. IEEE Transactions on Neural Networks. 8(3):1997;630-645.
39. Lek S., Delacoste M., Baran P., Dimopoulos I., Lauga J., Aulagnier S. Application of neural networks to modelling nonlinear relationships in ecology. Ecological Modelling. 90:1996;39-52.
40. Lin T., Horne B.G., Tiño P., Giles C.L. Learning long-term dependencies in NARX recurrent neural networks. IEEE Transactions on Neural Networks. 7(6):1996;1329-1338.
41. Lippmann R.P. An introduction to computing with neural nets. Acoustics, Speech, Signal Processing Magazine. 4:1987;4-22.
42. Loncelle, J., 1990. Constraining multilayer perceptrons: application to the contour detection. Proceedings of the International Neural Networks Conference, vol. 1. Paris, France, pp. 23-26.
43. Maier, H.R., 1995. A review of artificial neural networks. Research Report No. R131. Department of Civil and Environmental Engineering, The University of Adelaide, South Australia, Australia.
44. Maier H.R., Dandy G.C. Neural network models for forecasting univariate time series. Neural Network World. 5:1996;747-771.
45. Maier H.R., Dandy G.C. The use of artificial neural networks for the prediction of water quality parameters. Water Resources Research. 32(4):1996;1013-1022.
46. Maier H.R., Dandy G.C. Modelling cyanobacteria (blue-green algae) in the River Murray using artificial neural networks. Mathematics and Computers in Simulation. 43:1997;377-386.
47. Maren, A., Harston, C., Pap, R., 1990. Handbook of Neural Computing Applications. Academic Press Inc., San Diego, California.
48. Masters, T., 1993. Practical Neural Network Recipes in C++. Academic Press, San Diego.
49. NeuralWare, Inc., 1991. Neural Computing, NeuralWorks Professional II/Plus and NeuralWorks Explorer.
50. Poff N.L., Tokar S., Johnson P. Stream hydrological and ecological responses to climate change assessed with an artificial neural network. Limonl. and Oceanography. 41(5):1996;857-863.
51. Rangarajan, A., Chellappa, R., Manjunath, B.S., 1991. Markov random fields and neural networks with applications to early vision problems. In: Sethi, I.K., Jain, A.K. (Eds.), Artificial Neural Networks and Statistical Pattern Recognition - Old and New Connections. Elsevier Science Publishers B.V., Amsterdam.
52. Ripley B.D. Neural networks and related methods of classification. Journal of the Royal Statistical Society, B. 56(3):1994;409-456.
53. Roadknight C.M., Balls G.R., Mills G.E., Palmer-Brown D. Modeling complex environmental data. IEEE Transactions on Neural Networks. 8(4):1997;852-862.
54. Rogers L.L., Dowla F.U. Optimization of groundwater remediation using artificial neural networks with parallel solute transport modeling. Water Resources Research. 30(2):1994;457-481.
55. Rojas, R., 1996. Neural Networks: A Systematic Introduction. Springer-Verlag, Berlin.
56. Rumelhart, D.E., Hinton, G.E., Williams, R.J., 1986. Learning internal representations by error propagation. In: Rumelhart, D.E., McClelland, J.L. (Eds.), Parallel Distributed Processing, vol. 1. MIT Press.
57. Sarle, W.S., 1994. Neural networks and statistical models. Nineteenth Annual SAS Users Group International Conference.
58. Schizas C.N., Pattichis C.S., Michaelides S.C. Forecasting minimum temperature with short time-length data using artificial neural networks. Neural Network World. 4(2):1994;219-230.
59. Siegelmann H.T., Horne B.G., Giles C.L. Computational capabilities of recurrent NARX neural networks. IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics. 27(2):1997;208-215.
60. Sietsma J., Dow R.J.F. Creating artificial neural networks that generalize. Neural Networks. 4:1991;67-79.
61. Silva F., Almeida L. Acceleration techniques for the backpropagation algorithm. Lecture Notes in Computer Science. 412:1990;110-119.
62. Smith J., Eli R.N. Neural-network models of rainfall-runoff processes. Journal of Water Resources Planning and Management. 121(6):1995;499-508.
63. Solla S.A., Levin E., Fleisher M. Accelerated learning in layered neural networks. Complex Systems. 2:1988;625-639.
64. Stone M. Cross-validatory choice and assessment of statistical predictions. Journal of the Royal Statistical Society, B. 36:1974;111-147.
65. Tang Z., deAlmeida C., Fishwick P.A. Time series forecasting using neural networks vs Box-Jenkins methodology. Simulation. 57(5):1991;303-310.
66. Tangang F.T., Hsieh W.W., Tang B. Forecasting the equatorial pacific sea surface temperatures by neural network models. Climate Dynamics. 13(2):1997;135-147.
67. Tirakis, A., Sukissian, L., Kollias, S., 1990. An adaptive technique for segmentation and classification of textured images. Proceedings of the International Neural Networks Conference, vol. 1. Paris, France, pp. 31-34.
68. Vogl T.P., Mangis J.K., Rigler A.K., Zink W.T., Alkon D.L. Accelerating the convergence of the back-propagation method. Biological Cybernetics. 59:1988;257-263.
69. Warner B., Misra M. Understanding neural networks as statistical tools. American Statistician. 50(4):1996;284-293.
70. Weigend A.S., Rumelhart D.E., Huberman B.A. Predicting the future: a connectionist approach. International Journal of Neural Systems. 1(3):1990;193-209.
71. Wessels L., Barnard E. Avoiding false local minima by proper initialization of connections. IEEE Transactions on Neural Networks. 3(6):1992;899-905.
72. White H. Learning in artificial neural networks: a statistical perspective. Neural Computation. 1:1989;425-464.
73. Whitehead P.G., Howard A., Arulmani C. Modelling algal growth and transport in rivers - a comparison of time series analysis, dynamic mass balance and neural network techniques. Hydrobiologia. 349:1997;39-46.
74. Yeung D.S., Fong H.S. Handwritten chinese character recognition by rule-embedded neocognitron. Neural Computing and Applications. 2(4):1994;216-226.
75. Yu X.H., Chen G.A. Efficient backpropagation learning using optimal learning rate and momentum. Neural Networks. 10(3):1997;517-527.