HydroTest: A web-based toolbox of evaluation metrics for the standardised assessment of hydrological forecasts

Environmental Modelling and Software

Volumn 22, Issue 7, 2007, Pages 1034-1052

  Dawson, C W ^a   Abrahart, R J ^b   See, L M ^c  

^a LOUGHBOROUGH UNIVERSITY   (United Kingdom)

^b UNIVERSITY OF NOTTINGHAM   (United Kingdom)

^c UNIVERSITY OF LEEDS   (United Kingdom)

Author keywords

Evaluation metrics; Hydrological models; Performance testing

Indexed keywords

FORECASTING; MATHEMATICAL MODELS; TIME SERIES ANALYSIS; WORLD WIDE WEB;

EVALUATION METRICS; HYDROLOGICAL MODELS; MODEL PERFORMANCE; PERFORMANCE TESTING;

HYDROLOGY;

FORECASTING METHOD; HYDROLOGICAL MODELING; MODEL TEST; PERFORMANCE ASSESSMENT; TIME SERIES ANALYSIS; WORLD WIDE WEB;

EID: 33846798345     PISSN: 13648152     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.envsoft.2006.06.008     Document Type: Article

References (97)

1. Abrahart R.J., and See L.M. Comparing neural network and autoregressive moving average techniques for the provision of continuous river flow forecasts in two contrasting catchments. Hydrological Processes 14 11-12 (2000) 2157-2172
2. Akaike H. A new look at the statistical model identification. IEEE Transactions on Automatic Control 19 (1974) 716-723
3. Alvisi S., Mascellani G., Franchini M., and Bardossy A. Water level forecasting through fuzzy logic and artificial neural network approaches. Hydrology and Earth System Sciences 10 1 (2006) 1-17
4. Anctil F., Michel C., Perrin C., and Vazken Andréassian V. A soil moisture index as an auxiliary ANN input for stream flow forecasting. Journal of Hydrology 286 1-4 (2004) 155-167
5. Anderson M.G., and Burt T.P. Hydrological Forecasting (1985), John Wiley & Sons Ltd., Chichester, UK
6. Armstrong J.S., and Collopy F. Error measures for generalizing about forecasting methods: Empirical comparisons. International Journal of Forecasting 8 1 (1992) 69-80
7. Atiya A.F., El-Shoura S.M., Shaheen S.I., and El-Sherif M.S. A comparison between neural-network forecasting techniques-case study: river flow forecasting. IEEE Transactions on Neural Networks 10 2 (1999) 402-409
8. ASCE. Criteria for evaluation of watershed models. Journal of Irrigation and Drainage Engineering 119 3 (1993) 429-442
9. ASCE. Artificial neural networks in hydrology. II: hydrologic applications. Journal of Hydrological Engineering 5 2 (2000) 124-137
10. Baratti R., Cannas B., Fanni A., Pintus M., Sechi G.M., and Toreno N. River flow forecast for reservoir management for neural networks. Neurocomputing 55 (2003) 421-437
11. Beran M. Hydrograph prediction-how much skill?. Hydrology and Earth System Sciences 3 2 (1999) 305-307
12. Blackie J.R., and Eeles C.W.O. Lumped catchment models. In: Anderson M.G., and Burt T.P. (Eds). Hydrological Forecasting (1985), Wiley, Chichester, UK 311-345
13. Boughton W. Calibrations of a daily rainfall-runoff model with poor quality data. Environmental Modelling and Software 21 (2006) 1114-1128
14. Bowden G.J., Maier H.R., and Dandy G.C. Optimal division of data for neural network models in water resources applications. Water Resources Research 38 2 (2002) 1010 10.1029/2001WR000266
15. Campolo M., Soldati A., and Andreussi P. Artificial neural network approach to flood forecasting in the River Arno. Hydrological Sciences Journal 48 3 (2003) 381-398
16. Cannas, B., Fanni, A., See, L.M., Sias, G. The role of data preprocessing for river flow forecasting using neural networks. Physics and Chemistry of the Earth, in press.
17. Chang F.-J., Hu H.-F., and Chen Y.-C. Counterpropagation fuzzy-neural network for streamflow reconstruction. Hydrological Processes 15 2 (2001) 219-232
18. Chang F.-J., Chang L.-C., and Huang H.-L. Real-time recurrent learning neural network for stream-flow forecasting. Hydrological Processes 16 13 (2002) 2577-2588
19. Chang L.-C., Chang F.-J., and Chiang Y.-M. A two-step-ahead recurrent neural network for stream-flow forecasting. Hydrological Processes 18 1 (2004) 81-92
20. Chen S.-H., Lin Y.-H., Chang L.-C., and Chang F.-J. The strategy of building a flood forecast model by neuro-fuzzy network. Hydrological Processes 20 7 (2006) 1525-1540
21. Chibanga R., Berlamont J., and Vandewalle J. Modelling and forecasting of hydrological variables using artificial neural networks: the Kafue River sub-basin. Hydrological Sciences Journal 48 3 (2003) 363-379
22. Chiew F.H.S., and McMahon T.A. Assessing the adequacy of catchment streamflow yield estimates. Australian Journal of Soil Research 31 5 (1993) 665-680
23. Clarke R.T. A review of some mathematical models used in hydrology, with observations on their calibration and use. Journal of Hydrology 19 1 (1973) 1-20
24. Clements M.P., and Hendry D.F. On the limitations of comparing mean square forecast errors. Journal of Forecasting 12 8 (1993) 617-637
25. Corani G., and Guariso G. An application of pruning in the design of neural networks for real time flood forecasting. Neural Computing and Applications 14 1 (2005) 66-77
26. Coulibaly P., Anctil F., and Bobée B. Daily reservoir inflow forecasting using artificial neural networks with stopped training approach. Journal of Hydrology 230 3-4 (2000) 244-257
27. Dawson C.W., and Wilby R.L. Hydrological modelling using artificial neural networks. Progress in Physical Geography 25 1 (2001) 80-108
28. Dawson C.W., Harpham C., Wilby R.L., and Chen Y. An evaluation of artificial neural network techniques for flow forecasting in the River Yangtze, China. Hydrology and Earth System Sciences 6 4 (2002) 619-626
29. Dawson C.W., Abrahart R.J., Shamseldin A.Y., and Wilby R.L. Flood estimation at ungauged sites using artificial neural networks. Journal of Hydrology 319 (2006) 391-409
30. Dawson C.W., See L.M., Abrahart R.J., and Heppenstall A.J. Symbiotic adaptive neuro-evolution applied to rainfall-runoff modelling in northern England. Neural Networks 19 2 (2006) 236-247
31. DeVos N.J., and Rientjes T.H.M. Constraints of artificial neural networks for rainfall-runoff modelling: trade-offs in hydrological state representation and model evaluation. Hydrology and Earth System Sciences 9 1-2 (2005) 111-126
32. Diskin M.H., and Simon E. A procedure for the selection of objective functions for hydrologic simulation models. Journal of Hydrology 34 1-2 (1977) 129-149
33. Efron B. Nonparametric estimates of standard error: the jackknife, the bootstrap, and other methods. Biometrika 68 3 (1981) 589-599
34. Efron B. Nonparametric standard errors and confidence intervals. Canadian Journal of Statistics 9 2 (1981) 139-172
35. Efron B., and Gong G. A leisurely look at the bootstrap, the jackknife and cross-validation. American Statistician 37 1 (1983) 36-48
36. Elshorbagy A., Simonovic S.P., and Panu U.S. Performance evaluation of neural networks for runoff prediction. Journal of Hydrologic Engineering 5 4 (2000) 424-427
37. Fernando D.A.K., and Jayawardena A.W. Runoff forecasting using RBF networks with OLS algorithm. Journal of Hydrologic Engineering 3 3 (1998) 203-209
38. Fildes R. The evaluation of extrapolative forecasting methods. International Journal of Forecasting 8 1 (1992) 81-98
39. Furundzic D. Application example of neural networks for time series analysis: rainfall-runoff modeling. Signal Processing 64 3 (1998) 383-396
40. Gardner E.S. The trade-offs in choosing a time series method. Journal of Forecasting 2 3 (1983) 263-267
41. Garrick M., Cunnane C., and Nash J.E. A criterion of efficiency for rainfall-runoff models. Journal of Hydrology 36 3-4 (1978) 375-381
42. Gaume E., and Gosset R. Over-parameterisation, a major obstacle to the use of artificial neural networks in hydrology?. Hydrology and Earth System Sciences 7 5 (2003) 693-706
43. Giustolisi O., and Laucelli D. Improving generalization of artificial neural networks in rainfall-runoff modelling. Hydrological Sciences Journal 50 3 (2005) 439-457
44. Green I.R.A., and Stephenson D. Criteria for comparison of single event models. Hydrological Sciences Journal 31 3 (1986) 395-411
45. Gupta H.V., Sorooshian S., and Yapo P.O. Toward improved calibration of hydrological models: multiple and noncommensurable measures of information. Water Resources Research 34 4 (1998) 751-763
46. Hall M.J. How well does your model fit the data?. Journal of Hydroinformatics 3 1 (2001) 49-55
47. Houghton-Carr H.A. Assessment criteria for simple conceptual daily rainfall-runoff models. Hydrological Sciences Journal 44 2 (1999) 237-261
48. Hsu K.-L., Gupta H.V., and Sorooshian S. Artificial neural network modeling of the rainfall-runoff process. Water Resources Research 31 10 (1995) 2517-2530
49. Hu T.S., Lam K.C., and Ng S.T. River flow time series prediction with a range-dependent neural network. Hydrological Sciences Journal 46 5 (2001) 729-745
50. Jain A., and Srinivasulu S. Development of effective and efficient rainfall-runoff models using integration of deterministic, real-coded genetic algorithms and artificial neural network techniques. Water Resources Research 40 4 (2004) W04302
51. Jain A., and Srinivasulu S. Integrated approach to model decomposed flow hydrograph using artificial neural network and conceptual techniques. Journal of Hydrology 317 3-4 (2005) 291-306
52. Jakeman A.J., Letcher R.A., and Norton J.P. Ten iterative steps in the development and evaluation of environmental models. Environmental Modelling and Software 21 (2006) 602-614
53. Jeong D.-I., and Kim Y.-O. Rainfall-runoff models using artificial neural networks for ensemble streamflow prediction. Hydrological Processes 19 19 (2005) 3819-3835
54. Karunasinghe D.S.K., and Liong S.-Y. Chaotic time series prediction with a global model: artificial neural network. Journal of Hydrology 323 1-4 (2006) 92-105
55. Karunanithi N., Grenney W., Whitley D., and Bovee K. Neural networks for river flow prediction. Journal of Computing in Civil Engineering 8 2 (1994) 201-220
56. Kessler E., and Neas B. On correlation, with applications to the radar and raingage measurement of rainfall. Atmospheric Research 34 1-4 (1994) 217-229
57. Kitanidis P.K., and Bras R.L. Real-time forecasting with a conceptual hydrologic model: 2. Application and results. Water Resources Research 16 6 (1980) 1034-1044
58. Khalil M., Panu U.S., and Lennox W.C. Groups and neural networks based streamflow data infilling procedures. Journal of Hydrology 241 3-4 (2001) 153-176
59. Klemes V. Operational testing of hydrological simulation models. Hydrological Sciences Journal 31 3 (1986) 13-24
60. Krause P., Boyle D.P., and Bäse F. Comparison of different efficiency criteria for hydrological model assessment. In: Krause P., Bongartz K., and Flügel W.-A. (Eds). Proceedings of the 8th Workshop for Large Scale Hydrological Modelling-Oppurg 2004. Advances in Geosciences vol. 5 (2005) 89-97
61. Kumar A.R.S., Sudheer K.P., Jain S.K., and Agarwal P.K. Rainfall-runoff modelling using artificial neural networks: comparison of network types. Hydrological Processes 19 (2005) 1277-1291
62. Lauzon N., Anctil F., and Baxter C.W. Classification of heterogeneous precipitation fields for the assessment and possible improvement of lumped neural network models for streamflow forecasts. Hydrology and Earth System Sciences Discussions 3 1 (2006) 201-227
63. Legates D.R., and Davis R.E. The continuing search for an anthropogenic climate change signal: limitations of correlation-based approaches. Geophysical Research Letters 24 18 (1997) 2319-2322
64. Legates D.R., and McCabe G.J. Evaluating the use of "goodness-of-fit" measures in hydrologic and hydroclimatic model validation. Water Resources Research 35 1 (1999) 233-241
65. Lin G.-F., and Chen L.-H. A non-linear rainfall-runoff model using radial basis function network. Journal of Hydrology 289 1-4 (2004) 1-8
66. Lin G.-F., and Chen L.-H. Time series forecasting by combining the radial basis function network and self-organizing map. Hydrological Processes 19 10 (2005) 1925-1937
67. Liong S.-Y., Lim W.-H., and Paudyal G. River stage forecasting in Bangladesh: neural network approach. Journal of Computing in Civil Engineering 14 1 (2000) 1-8
68. Lorrai M., and Sechi G. Neural nets for modelling rainfall-runoff transformations. Water Resources Management 9 4 (1995) 299-313
69. Maier H.R., and Dandy G.C. The use of artificial neural networks for the prediction of water quality parameters. Water Resources Research 32 4 (1996) 1013-1022
70. Maier H.R., and Dandy G.C. Neural networks for the prediction and forecasting of water resources variables: a review of modelling issues and applications. Environmental Modelling and Software 15 1 (2000) 101-124
71. Makridakis S. Accuracy measures: theoretical and practical concerns. International Journal of Forecasting 9 4 (1993) 527-529
72. Nash J.E., and Sutcliffe J.V. River flow forecasting through conceptual models 1: a discussion of principles. Journal of Hydrology 10 3 (1970) 282-290
73. NERC. Flood Studies Report. Hydrological Studies Vol. 1 (1975), Natural Environment Research Council, London, UK
74. Paik K., Kim J.H., Kim H.S., and Lee D.R. A conceptual rainfall-runoff model considering seasonal variation. Hydrological Processes 19 19 (2005) 3837-3850
75. Pearson K. Mathematical contributions to the theory of evolution. III. Regression, heredity and panmixia. Philosophical Transactions of the Royal Society of London Series A 187 (1896) 253-318
76. Pebesma E.J., Switzer P., and Loague K. Error analysis for the evaluation of model performance: rainfall-runoff event time series data. Hydrological Processes 19 8 (2005) 1529-1548
77. Rajurkar M.P., Kothyari U.C., and Chaube U.C. Modeling of the daily rainfall-runoff relationship with artificial neural network. Journal of Hydrology 285 1-4 (2004) 96-113
78. Riad S., Mania J., Bouchaou L., and Najjar Y. Predicting catchment flow in a semi-arid region via an artificial neural network technique. Hydrological Processes 18 13 (2004) 2387-2393
79. Refsgaard J.C., and Henriksen H.J. Modelling guidelines-terminology and guiding principles. Advances in Water Resources 27 1 (2004) 71-82
80. Refsgaard J.C., Henriksen H.J., Harrar W.G., Scholten H., and Kassahun A. Quality assurance in model based water management-review of existing practice and outline of new approaches. Environmental Modelling and Software 20 (2005) 1201-1215
81. Schwarz G. Estimating the dimension of a model. Annals of Statistics 6 2 (1978) 461-464
82. See, L., Abrahart, R.J., 1999. Multi-model data fusion for hydrological forecasting. Proceedings of the 4th International Conference on Geocomputation, Fredericksburg, Virginia, USA, 25-28 July.
83. Seibert J. On the need for benchmarks in hydrological modelling. Hydrological Processes 15 6 (2001) 1063-1064
84. Shamseldin A.Y. Application of a neural network technique to rainfall-runoff modelling. Journal of Hydrology 199 3-4 (1997) 272-294
85. Singh V.P., and Woolhiser D.A. Mathematical modeling of watershed hydrology. Journal of Hydrologic Engineering 7 4 (2002) 270-292
86. Stephenson D. Direct optimization of Muskingum routing coefficients. Journal of Hydrology 41 1-2 (1979) 161-165
87. Sudheer K.P., Nayak P.C., and Ramasastri K.S. Improving peak flow estimates in artificial neural network river flow models. Hydrological Processes 17 3 (2003) 677-686
88. Supharatid S. Application of a neural network model in establishing a stage-discharge relationship for a tidal river. Hydrological Processes 17 15 (2003) 3085-3099
89. Tang Y., Reed P., and Wagener T. How effective and efficient are multiobjective evolutionary algorithms at hydrologic model calibration?. Hydrology and Earth System Sciences 10 2 (2006) 289-307
90. Teegavarapu R., and Elshorbagy A. Fuzzy set based error measure for hydrologic model evaluation. Journal of Hydroinformatics 7 3 (2005) 199-207
91. Wagener T., and McIntyre N. Identification of rainfall-runoff models for operational applications. Hydrological Sciences Journal 50 5 (2005) 735-751
92. Watts G. Hydrological modelling in practice. In: Wilby R.L. (Ed). Contemporary Hydrology: Towards Holistic Environmental Science (1997), John Wiley, Chichester, UK 151-193
93. Wilmott C.J. On the validation of models. Physical Geography 2 2 (1981) 184-194
94. Willmott C.J., Ackleson S.G., Davis R.E., Feddema J.J., Klink K.M., Legates D.R., O'Donnell J., and Rowe C.M. Statistics for the evaluation and comparison of models. Journal of Geophysical Research 90 C5 (1985) 8995-9005
95. WMO. Intercomparison of Models of Snowmelt Runoff. Operational Hydrology Report No. 23, WMO Publ. No. 646 (1986), World Meteorological Organization, Geneva, Switzerland
96. Yapo P.O., Gupta H.V., and Sorooshian S. Automatic calibration of conceptual rainfall-runoff models: sensitivity to calibration data. Journal of Hydrology 181 1-4 (1996) 23-48
97. Yu P.-S., and Yang T.-C. Fuzzy multi-objective function for rainfall-runoff model calibration. Journal of Hydrology 238 1-2 (2000) 1-14