Echo state networks as an alternative to traditional artificial neural networks in rainfall-runoff modelling

Hydrology and Earth System Sciences

Volumn 17, Issue 1, 2013, Pages 253-267

  De Vos, N J ^a  

^a NONE   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

FEED-FORWARD NETWORK; PERFORMANCE MEASURE; RAINFALL-RUNOFF MODELLING; RAINFALL-RUNOFF MODELS; RECURRENT ARTIFICIAL NEURAL NETWORKS; RELEVANT INFORMATIONS; STREAMFLOW FORECAST; TRAINING PROCEDURES;

COMPLEX NETWORKS; RUNOFF;

NEURAL NETWORKS;

ARTIFICIAL NEURAL NETWORK; RAINFALL-RUNOFF MODELING; RIVER BASIN; STREAMFLOW;

EID: 84879033086     PISSN: 10275606     EISSN: 16077938     Source Type: Journal    
DOI: 10.5194/hess-17-253-2013     Document Type: Article

References (54)

1. Abrahart, R. J. and See, L. M.: Comparing neural network and autoregressive moving average techniques for the provision of continuous river flow forecasts in two contrasting catchments, Hydrol. Process., 14, 2157-2172, 2000.
2. Abrahart, R. J., Heppenstall, A. J. and See, L. M.: Timing error correction procedure applied to neural network rainfall-runoff modelling, Hydrol. Sci. J., 52, 414-431, doi:10.1623/hysj.52.3.414, 2007. (Pubitemid 46851531)
3. Abrahart, R. J., Anctil, F., Coulibaly, P., Dawson, C. W., Mount, N. J., See, L. M., Shamseldin, A. Y., Solomatine, D. P., Toth, E. and Wilby, R. L.: Two decades of anarchy Emerging themes and outstanding challenges for neural network river forecasting, Prog. Phys. Geog., 36, 480-513, doi:10.1177/ 0309133312444943, 2012a.
4. Abrahart, R. J., Mount, N. J. and Shamseldin, A. Y.: Discussion of "Reservoir Computing approach to Great Lakes water level forecasting" by P. Coulibaly [J. Hydrol. 381 (2010) 76-88].
5. J. Hydrol., 422-423, 76-80, doi:10.1016/j.jhydrol.2011.10.006, 2012b.
6. Anctil, F., Michel, C., Perrin, C., and Andreassian, V.: A soil moisture index as an auxiliary ANN input for stream flow forecasting, J. Hydrol., 286, 155-167, 2004. (Pubitemid 38203954)
7. Atiya, A. F. and Parlos, A. G.: New results on recurrent network training: unifying the algorithms and accelerating convergence, IEEE T. Neural Netw., 11, 697-709, 2000.
8. Barreto, G. de A. and Aráujo, A. F. R.: A self-organizing NARX network and its application to prediction of chaotic time series, in: Proceedings of the IEEE Intl. Joint Conference on Neural Networks, vol. 3, Washington D.C., USA, 2144-2149, 2001. (Pubitemid 32805231)
9. Bengio, Y., Simard, P., and Frasconi, P.: Learning long-term dependencies with gradient-descent is difficult, IEEE T. Neural Netw., 5, 157-166, 1994.
10. Beven, K. J., Lamb, R., Quinn, P. F., Romanowicz, R., and Freer, J.: TOPMODEL, in: Computer Models of Watershed Hydrology, edited by: Singh, V. P., Water Resources Publications, Colorado, 627-668, 1995.
11. Brazil, L. E.: Multilevel calibration strategy for complex hydrologic simulation models, Ph.D. dissertation, Colo. State Univ., Fort Collins, 1988.
12. Buehner, M. and Young, P.: A tighter bound for the echo state property, Neural Netw., 17, 820-824, 2006.
13. Burnash, R. J. C.: The NWS river forecast system-catchment modeling, in: Computer Models of Watershed Hydrology, edited by: Singh, V. P., Water Resources Publications, Colorado, 311-366, 1995.
14. Campolo, M., Andreussi, P., and Soldati, A.: River flood forecasting with a neural network model, Water Resour. Res., 35, 1191-1197, 1999.
15. Chang, F. J., Chiang, L. C., and Huang, H. L.: Real-time recurrent learning network for stream-flow forecasting, Hydrol. Process., 16, 2577-2588, 2002.
16. Chiang, Y. M., Chiang, L. C., and Chang, F. J.: Comparison of static-feedforward and dynamic-feedback neural networks for rainfall-runoff modeling, J. Hydrol., 290, 297-311, 2004.
17. Coulibaly, P.: Reservoir computing approach to Great Lakes water level forecasting, J. Hydrol., 381, 76-88, 2010.
18. Coulibaly, P., Anctil, F., Rasmussen, P., and Bobee, B.: A recurrent neural networks approach using indices of low-frequency climatic variability to forecast regional annual runoff, Hydrol. Process., 14, 2755-2777, 2000.
19. Cunge, J. A.: Of data and models, J. Hydroinform., 5, 75-98, 2003.
20. de Vos, N. J. and Rientjes, T. H. M.: Constraints of artificial neural networks for rainfall-runoff modelling: trade-offs in hydrological state representation and model evaluation, Hydrol. Earth Syst. Sci., 9, 111-126, doi:10.5194/hess-9-111-2005, 2005. (Pubitemid 41122744)
21. de Vos, N. J. and Rientjes, T. H. M.: Correction of timing errors of artificial neural network rainfall-runoff models, in: Practical Hydroinformatics, edited by: Abrahart, R. J., See, L. M., and Solomatine, D. P., Water Science and Technology Library, Springer, 2008a.
22. de Vos, N. J. and Rientjes, T. H. M.: Multi-objective training of artificial neural networks for rainfall-runoff modeling,Water Resour. Res., 44, W08434, doi:10.1029/2007WR006734, 2008b.
23. Doya, K.: Bifurcations in the learning of recurrent neural networks, in: Proc. IEEE Int. Symposium on Circuits and Systems, vol. 6, San Diego, CA, USA, 2777-2780, 1992.
24. Duan, Q., Schaake, J., Andreassian, V., Franks, S., Goteti, G., Gupta, H. V., Gusev, Y. M., Habets, F., Hall, A., Hay, L., Hogue, T., Huang, M., Leavesley, G., Liang, X., Nasonova, O. N., Noilhan, J., Oudin, L., Sorooshian, S., Wagener, T., and Wood, E. F.: Model Parameter Estimation Experiment (MOPEX): an overview of science strategy and major results from the second and third workshops, J. Hydrol., 320, 3-17, 2006.
25. Elman, J. L.: Finding structure in time, Cognitive Sci., 14, 179-211, 1990.
26. Hagan, M. T. and Menhaj, M. B.: Training feedforward networks with the Marquardt algorithm, IEEE T. Neural Netw., 5, 989-993, 1994.
27. Hammer, B., Schrauwen, B., and Steil, J. J.: Recent advances in efficient learning of recurrent networks, in: European Symposium on Artificial Neural Networks, Bruges, Belgium, 213-226, 2009.
28. Haykin, S.: Neural Networks, a Comprehensive Foundation, Prentice Hall, Upper Saddle River, 1999.
29. Haykin, S.: Kalman Filtering and Neural Networks, Wiley, New York, 2001.
30. Hsu, K.-L., Gupta, H. V., and Sorooshian, S.: Artificial neural network modeling of the rainfall-runoff process,Water Resour. Res., 31, 2517-2530, 1995. (Pubitemid 26475080)
31. Hsu, K.-L., Gupta, H. V., and Sorooshian, S.: Application of a recurrent neural network to rainfall-runoff modeling, in: 24th Annual Water Resources Planning and Management Conference, Houston, TX, USA, 68-73, 1997.
32. Jaeger, H.: The echo state approach to analysing and training recurrent neural networks, Tech. Report GMD Report 148, German National Research Center for Information Technology, St. Augustin, Germany, 2001.
33. Jaeger, H.: A tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the "echo state network" approach, Tech. Report GMD Report 159, German National Research Center for Information Technology, St. Augustin, Germany, 2002.
34. Jaeger, H. and Haas, H.: Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication, Science, 304, 78-80, 2004. (Pubitemid 38455427)
35. Jain, A. and Srinivasulu, S.: Development of effective and efficient rainfall-runoff models using integration of deterministic, real-coded genetic algorithms and artificial neural network techniques, Water Resour. Res., 40, W04302, doi:10.1029/2003WR002355, 2004.
36. Jang, J. S., R.: ANFIS: Adaptive-network-based fuzzy inference systems, IEEE T. Syst. Man. Cyb., 23, 665-685, 1993.
37. Lindstrom, G., Johansson, B., Persson, M., Gardelin, M., and Bergstrom, S.: Development and test of the distributed HBV-96 hydrological model, J. Hydrol., 201, 272-288, 1997. (Pubitemid 28037764)
38. Lukosevicius, M.: Echo state networks with trained feedbacks, Tech. Report No. 4, Jacobs University Bremen, 2007.
39. Lukosevicius, M. and Jaeger, H.: Reservoir computing approaches to recurrent neural network training, Comput. Sci. Rev., 3, 127-149, 2009.
40. Maass,W., Natschl̈ager, T., and Markram, H.: Real-time computing without stable states: a new framework for neural computation based on perturbations, Neural Comput., 14, 2531-2560, 2002.
41. Møller, M. F.: A scaled conjugate gradient algorithm for fast supervised learning, Neural Netw., 6, 525-533, 1993.
42. Puskorius, G. V. and Feldkamp, L. A.: Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks, IEEE T. Neural Netw., 5, 279-297, 1994.
43. Rumelhart, D. E. and McLelland, J. L.: Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1, MIT Press, Cambridge, 1986.
44. Saad, E. W., Prokhorov, D. V., and Wunsch, I.: Comparative study of stock trend prediction using time delay, recurrent and probabilistic neural networks, IEEE T. Neural Netw., 9, 1456-1470, 1998.
45. Sacchi, R., Carneiro, A. A. F. M., and Aráujo, A. F. R.: A RBF network trained by the SONARX model and applied to obtain the operation policies of the hydropower systems, in: Brazilian Symposium on Neural Networks-SBRN, Brazil, 2004.
46. Sacchi, R., Ozturk, M. C., Principe, J. C., Carneiro, A. A. F., and da Silva, I. N.: Water inflow forecasting using the Echo State Network: a Brazilian case study, in: Proceedings of the IEEE Intl. Joint Conference on Neural Networks, Orlando, FL, USA, 2403-2408, 2007.
47. Shamseldin, A. Y.: Application of a neural network technique to rainfall-runoff modelling, J. Hydrol., 199, 272-294, 1997. (Pubitemid 27492871)
48. Steil, J. J.: Backpropagation-decorrelation: recurrent learning with O(N) complexity, in: Proceedings of the IEEE Intl. Joint Conference on Neural Networks, vol. 2, Budapest, Hungary, 843-848, 2004.
49. Sum, J., Chan, L., Leung, C., and Young, G.: Extended Kalman filter-based pruning method for recurrent neural networks, Neural Comput., 10, 1481-1506, 1998.
50. Verstraeten, D., Schrauwen, B., D'Haene, M., and Stroobandt, D.: An experimental unification of reservoir computing methods, Neural Netw., 20, 391-403, 2007. (Pubitemid 46856109)
51. Werbos, P. J.: Backpropagation through time: what it does and how to do it, Proc. IEEE, 78, 1550-1560, 1990.
52. Williams, R. J. and Zipser, D.: A learning algorithm for continually running fully recurrent neural networks, Neural Comput., 1, 270-280, 1989.
53. Wyffels, F., Schrauwen, B., and Stroobandt, D.: Stable output feedback in reservoir computing using ridge regression, in: Proc. 18th Int. Conference on Artificial Neural Networks, vol. 5163, Prague, Czech Republic, 808-817, 2008.
54. Yonaba, H., Anctil, F., and Fortin, V.: Comparing sigmoid transfer functions for neural network multistep ahead streamflow forecasting, J. Hydrol. Eng., 15, 275-283, 2010.