Performance and reliability of multimodel hydrological ensemble simulations based on seventeen lumped models and a thousand catchments

Hydrology and Earth System Sciences

Volumn 14, Issue 11, 2010, Pages 2303-2317

  Velazquez J A ^a   Anctil, F ^a   Perrin, C ^b  

^a Chaire de Recherche EDS en Prévisions et Actions Hydrologiques   (Canada)

^b CEMAGREF   (France)

Author keywords

[No Author keywords available]

Indexed keywords

ADDED VALUES; AVERAGE ERRORS; CONTINUOUS RANKED PROBABILITY SCORES; DETERMINISTIC SIMULATION; ENSEMBLE SIMULATION; HYDROLOGICAL MODELS; LUMPED MODELS; MODEL OUTPUTS; MULTI-MODEL; PREDICTIVE DISTRIBUTIONS; RANK HISTOGRAMS; RELATIVE OPERATING CHARACTERISTICS;

CATCHMENTS; OPTIMIZATION; PROBABILISTIC LOGICS; RELIABILITY; RUNOFF;

COMPUTER SIMULATION;

CATCHMENT; ENSEMBLE FORECASTING; GENETIC ALGORITHM; HISTOGRAM; HYDROLOGICAL MODELING; NUMERICAL MODEL; OPTIMIZATION; PERFORMANCE ASSESSMENT; RELIABILITY ANALYSIS; STREAMFLOW;

EID: 78649391813     PISSN: 10275606     EISSN: 16077938     Source Type: Journal    
DOI: 10.5194/hess-14-2303-2010     Document Type: Article

References (92)

1. Ajami, N. K., Duan, Q., Gao, X., and Sorooshian, S.: Multi-model combination techniques for hydrological simulations: application to Distributed Model Intercomparison Project results, J. Hydrometeorol., 7, 755-768, 2006.
2. Anctil, F., Lauzon, N., Andréassian, V., Oudin, L., and Perrin, C.: Improvement of rainfall-runoff forecasts through mean areal rainfall optimization, J Hydrol., 328, 717-725, 2006.
3. Andréassian, V., Oddos, A., Michel, C., Anctil, F., Perrin, C., and Loumagne, C.: Impact of spatial aggregation of inputs and parameters on the efficiency of rainfall-runoff models: A theoretical study using chimera watersheds, Water Resour. Res., 40 (5), W05209, doi:10.1029/2003WR002854, 2004.
4. Atger, F.: Verification of intense precipitation forecasts from single models and ensemble prediction systems, Nonlinear Proc. Geoph., 8, 401-417, 2001.
5. Bergström, S. and Forsman, A.: Development of a conceptual deterministic rainfall-runoff model, Nord. Hydrol., 4, 147-170, 1973.
6. Beven, K. J. and Binley, A.: The future of distributed models: Model calibration and uncertainty prediction, Hydrol. Processes, 6 (3), 279-298, 1992.
7. Bormann, H., Breuer, L., Croke, B., Gräff, T., Hubrechts, L., Huisman, J. A., Kite, G. W., Lanini, J., Leavesley, G., Lindström, G., Seibert, J., Viney, N. R., and Willems, P.: Reduction of predictive uncertainty by ensemble hydrological modelling of catchment processes and land use change effects, in: Proceedings of the 11th Conference of the Euromediterranean Network of Experimental and Representative Basins (ERB), Luxembourg, 20-22 September 2006. IHP-VI/Technical Documents in Hydrology 81, 133-139, 2007.
8. Boucher, M. A., Perreault, L., and Anctil, F.: Tools for the assessment of hydrological ensemble forecasts obtained by neural networks, J. Hydroinform., 11, 297-307, 2009.
9. Bradley, A. A., Hashino, T., and Schwartz, S. S.: Distributionsoriented verification of probability forecasts for small data samples, Weather Forecast., 18, 903-917, 2003.
10. Burnash, R. J. C., Ferral, R. L., and Mc Guire, R. A.: A generalized streamflow simulation system-Conceptual modeling for digital computers, U. S. Department of Commerce, National Weather Service and State of California, Department of Water Resources, 1973.
11. Candille, G. and Talagrand, O.: Evaluation of probabilistic prediction systems for a scalar variable, Q. J. Roy. Meteor. Soc., 131, 2131-2150, 2005.
12. Chiew, F. H. S., Peel, M. C., and Western, A. W.: Application and testing of the simple rainfall-runoff model SIMHYD, in: Mathematical Models of Small Watershed Hydrology and Applications, edited by: Singh, V. P. and Frevret, D. K., Water Resources Publications, Highlands Ranch, 335-367, 2002.
13. Clark, M. P. and Slater, A. G.: Probabilistic quantitative precipitation estimation in complex terrain, J. Hydrometeor., 7, 3-22, 2006.
14. Clark, M. P., Slater, A. G., Rupp, D. E., Woods, R. A., Vrugt, J. A., Gupta, H. V., Wagener T., and Hay, L. E.: Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models, Water Resour. Res., 44, W00B02, doi:10.1029/2007WR006735, 2008.
15. Cormary, Y. and Guilbot, A.: Étude des relations pluie-débit sur trois bassins versants d'investigation, IAHS-AISH Publ., 108, 265-279, 1973.
16. Coulibaly, P., Hache, M., Fortin, V., and Bobée, B.: Improving daily reservoir inflow forecasts with model combination, J. Hydrol. Eng., 9 (2), 91-99, 2005.
17. Devineni, N., Sankarasubramanian, A., and Ghosh, S.: Multimodel ensembles of streamflow forecasts: Role of predictor state in developing optimal combinations, Water Resour. Res., 44, W09404, doi:10.1029/2006WR005855, 2008.
18. Duan, Q., Ajami, N. K., Gao, X., and Sorooshian, S.: Multi-Model Ensemble Hydrologic Prediction Using Bayesian Model Averaging, Adv. Water Resour., 30, 1371-1386, 2007.
19. Durand, Y., Brun E., Mérindol L., Guyomarc'h G., Lesassre B., and Martin E.: A meteorological estimation of relevant parameters for snow models, Ann. Glaciol., 18, 65-71, 1993.
20. Edijatno, Nascimento, N. O., Yang, X., Makhlouf, Z., and Michel, C.: GR3J: a daily watershed model with three free parameters, Hydrolog. Sci. J., 44 (2), 263-277, 1999.
21. Evensen, G.: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics, J. Geophys. Res., 99 (C5), 10143-10162, doi:10.1029/94JC00572, 1994.
22. Fortin, V. and Turcotte, R.: Le modèle hydrologique MOHYSE, Note de cours pour SCA7420, Département des sciences de la terre et de l'atmosphère, Université du Québec à Montréal, 2006.
23. Fortin, V., Favre, A. C., and Said, M.: Probabilistic forecasting from ensemble prediction systems: Improving upon the best-member method by using a different weight and dressing kernel for each member, Q. J. Roy. Meteor. Soc., B132, 1349-1369, 2006.
24. Garçon, R.: Modèle global Pluie-Débit pour la prévision et la prédétermination des crues, La Houille blanche, 7 (8), 88-95, 1999.
25. Georgakakos, K. P., Seo, D.-J., Gupta, H., Schaake, J., and Butts, M. B.: Characterizing streamflow simulation uncertainty through multimodel ensembles, J Hydrol., 298, 222-241, 2004.
26. Girard, G., Morin G., and Charbonneau, R.: Modèle précipitationsdébits à discrétisation spatiale, Cahiers ORSTOM Série Hydrologie, IX 4, 35-52, 1972.
27. Gneiting, T., Raftery, A. E., Westveld III, A. H., and Goldman, T.: Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation, Mon. Weather Rev., 133 (5), 1098-1118, 2005.
28. Gneiting, T. and Raftery, A. E.: Strictly proper scoring rules, prediction, and estimation, J. Am. Stat. Assoc., 102 (477), 359-378, 2007.
29. Goldberg, D. E.: Genetic Algorithms in search, optimization, and machine learning. Reading, MA, Addison-Wesley, 412 pp, 1989.
30. Hagerdorn, R., Doblas-Reyes, F., and Palmer, T.: The rationale behind the success of multi-model ensembles in seasonal forecasting - I. Basic concept., Tellus A, 57 (3), 219-233, 2005. (Pubitemid 40659620)
31. Hamill, T.: Interpretation of Rank Histograms for Verifying Ensemble Forecasts, Mon. Weather Rev., 129, 550-560, 2001.
32. Hamill, T. M., Whitaker, J. S., and Wei, X.: Ensemble reforecasting: Improving medium-range forecast skill using retrospective forecasts, Mon. Weather Rev., 132, 1434-1447, 2004.
33. Hamill, T. M., Whitaker, J. S., and Mullen, S. L.: Reforecasts: An important new dataset for improving weather predictions, B. A. Meteorol. Soc., 87, 33-46, 2006.
34. He, Y., Wetterhall, F., Cloke, H. L., Pappenberger, F., Wilson, M., Freer, J., and McGregor, G.: Tracking the uncertainty in flood alerts driven by grand ensemble weather predictions, Meteorol. Appl., 16 (1), 91-101, 2009.
35. He, Y., Wetterhall, F., Bao, H., Cloke, H., Li, Z., Pappenberger, F., Hu, Y., Manful, D., and Huang Y.: Ensemble forecasting using TIGGE for the July-September 2008 floods in the Upper Huai catchment: a case study, Atmos. Sci. Lett., 11 (2), 132-138, 2010.
36. Jakeman, A. J., Littlewood, I. G., and Whitehead, P. G.: Computation of the instantaneous unit hydrograph and identifiable component flows with applications to two small upland catchments, J. Hydrol., 117, 275-300, 1990.
37. Jaun, S., Ahrens, B., Walser, A., Ewen, T., and Schär, C.: A probabilistic view on the August 2005 floods in the upper Rhine catchment, Nat. Hazards Earth Syst. Sci., 8, 281-291, doi:10.5194/nhess-8-281-2008, 2008.
38. Jiang, Z. and Mu, M.: A comparision study of the methods of conditional nonlinear optimal perturbations and singular vectors in ensemble predictions, Adv. Atmos. Sci., 26 (3), 465-470, 2009.
39. Johnson, C. and Bowler, N.: On the Reliability and Calibration of Ensemble Forecasts, Mon. Wea. Rev., 137, 1717-1720, 2009.
40. Jones, A. E. and Morse, A. P.: Application and Validation of a Seasonal Ensemble Prediction System Using a Dynamic Malaria Model, J. Climate, 23, 4202-4215, doi:10.1175/2010JCLI3208.1, 2010.
41. Krishnamurti, T. N.: Improved Weather and Seasonal Climate Forecasts from Multimodel Superensemble, Science, 285 (1548), 1548-1550, 1999.
42. Krueger, T., Freer J., Quinton, J. N., Macleod, C. J. A., Bilotta, G. S., Brazier, R. E., Butler, P., and Haygarth, P. M.: Ensemble evaluation of hydrological model hypotheses, Water Resour. Res., 46, W07516, doi:10.1029/2009WR007845, 2010.
43. Leavesley, G. H., Markstrom, S. L., Brewer, M. S., and Viger, R. J.: The modular modelling system (MMS) - The physical process modelling component of a database-centered decision support system for water and power management, Water Air Soil Poll., 90 (1-2), 303-311, 1996.
44. Le Moine, N., Andréassian, V., Perrin, C., and Michel, C.: How can rainfall-runoff model handle intercatchment groundwater flows? Theoretical study based on 1040 French catchments, Water Resour. Res., 43, W06428, doi:10.1029/2006WR005608, 2007. (Pubitemid 47124449)
45. Le Moine, N.: Le bassin versant de surface vu par le souterrain: une voie d'amélioration des performances et du réalisme des modèles pluie-débit?, Ph. D. thesis, Pierre et Marie Curie University, Paris, France, 324 pp, 2008.
46. Liu, Y. L., Freer, J., Beven, K., and Matgen, P.: Towards a limits of acceptability approach to the calibration of hydrological models: Extending observation error, J. Hydrol., 367 (1-2), 93-103, 2009.
47. Loumagne, C., Vidal, J. J., Torterotot, J. P., and Roche, P. A.: Procédure de décision multimodèle pour une prévision des crues en temps réel application au bassin supérieur de la Garonne, Revue des sciences de l'eau, 8, 539-561, 1995.
48. Marshall, L., Nott D., and Sharma, A.: Hydrological model selection: A Bayesian alternative, Water Resour. Res., 41, W10422, doi:10.1029/2004WR003719, 2005. (Pubitemid 41705706)
49. Marshall, L., Sharma, A., and Nott D.: Modeling the catchment via mixtures: Issues of model specification and validation, Water Resour. Res., 42, W11409, doi:10.1029/2005WR004613, 2006. (Pubitemid 46134880)
50. Marshall, L., Nott D., and Sharma, A.: Towards dynamic catchment modelling: a Bayesian hierarchical mixtures of experts framework, Hydrol. Process., 21 (7), 847-861, 2007.
51. Mason, S. J.: A model for assessment of weather forecast, Aust. Met. Mag., 30, 291-303, 1982.
52. Matheson, J. E. and Winkler, R. L.: Scoring rules for continuous probability distributions, Manage. Sci., 22, 1087-1096, 1976.
53. Mathevet, T.: Quels modèles pluie-débit globaux pour le pas de temps horaire? Développement empirique et comparaison de modèles sur un large échantillon de bassins versants, Ph. D. thesis, ENGREF (Paris), Cemagref (Antony), France, 463 pp., 2005.
54. Merz, R., Parajka, J., and Blöschl, G.: Scale effects in conceptual hydrological modeling, Water Resour. Res., 45, W09405, doi:10.1029/2009WR007872, 2009.
55. Michel, C., Perrin, C., and Andréassian, V.: The exponential store: a correct formulation for rainfall-runoff modeling, Hydrolog. Sci. J., 48 (1), 109-124, 2003.
56. Moore, R. J. and Clarke, R. T.: A distribution function approach to rainfall-runoff modeling, Water Resour. Res., 17 (5), 1367-1382, doi:10.1029/WR017i005p01367, 1981.
57. Moradkhani, H., Hsu, K.-L., Gupta, H., and Sorooshian, S.: Uncertainty assessment of hydrologic model states and parameters: Sequential data assimilation using the Particle Filter, Water Resour. Res., 41 (5), 1-17, W05012, doi:10.1029/2004WR003604, 2005. (Pubitemid 40856462)
58. Neuman, S. P.: Maximun likelihood Bayesian averaging of uncertain model predictions, Stoch. Env. Res. Risk A., 17, 291-305, 2003.
59. Nielsen, S. A. and Hansen, E.: Numerical simulation of the rainfallrunoff process on a daily basis, Nord. Hydrol., 4, 171-190, 1973.
60. O'Connell, P. E. and Clarke R. T.: Adaptive hydrological forecasting. A review, Hydrolog. Sci. Bulletin, 26 (2), 179-205, 1981.
61. Oudin, L., Hervieu, F., Michel, C., Perrin, C., Andréassian, V., Anctil, F., and Loumagne, C.: Which potential evapotranspiration input for a rainfall-runoff model? Part 2 - Towards a simple and efficient PE model for rainfall-runoff modeling, J. Hydrol., 303 (1-4), 290-306, 2005. (Pubitemid 40307140)
62. Oudin, L., Andréassian, V., Mathevet, T., and Perrin, C.: Dynamic averaging of rainfall-runoff model simulations from complementary model parameterizations, Water Resour. Res., 42, W07410, doi:10.1029/2005WR004636, 2006. (Pubitemid 44300477)
63. Palmer, T. N., Alessandri, A., Andersen, U., Cantelaube, P., Davey, M., Delecluse, P., Deque, M., Diez, E., Doblas-Reyes, F. J., Feddersen, H., Graham, R., Gualdi, S., Gueremy, J. F., Hagedorn
64. R., Hoshen, M., Keenlyside, N., Latif, M., Lazar, A., Maisonnave, E., Marletto, V., Morse, A. P., Orfila, B., Rogel, P., Terres, J. M. and. and Thomson, M. C.: Development of a European multimodel ensemble system for seasonal to inter-annual prediction (DEMETER)., B. Am. Meteorol. Soc., 85, 853-872, 2004.
65. Perrin, C., Michel, C., and Andréassian, V.: Does a large number of parameters enhance model performance? Comparative assessment of common catchment model structures on 429 catchments, J. Hydrol., 242 (3-4), 275-301, 2001.
66. Perrin, C., Michel, C., and Andréassian, V.: Improvement of a parsimonious model for streamflow simulation, J. Hydrol., 279, 275-289, 2003.
67. Peterson, W. W., Birdsall, T. G., and Fox, W. C.: The theory of signal detectability, Trans., IRE Prof. Group. Inf. Theory, PGIT, 2-4, 171-212, 1954.
68. Quintana-Segui, P., Le Moigne, P., Durand, Y., Martin, E., Habets, F., Baillon, M., Canellas, C., Franchisteguy, L. and Morel, S.: Analysis of near-surface atmospheric variables: Validation of the SAFRAN analysis over France, J. Appl. Meteorol. Clim., 47 (1), 92-107, 2008.
69. Raftery, A. E., Balabdaoui, F., Gneiting, T., and Polakowski, M.: Using Bayesian model averaging to calibrate forecast ensembles, Tech. Rep. 440, Dep. of Stat., Univ. of Wash., Seattle, 2003.
70. Raftery, A. E., Gneiting, T., Balabdaoui, F., and Polakowski, M.: Using bayesian model averaging to calibrate forecast ensembles, Mon. Wea. Rev., 133, 1155-1174, 2005.
71. Roulston, M. S. and Smith, L. A.: Combining dynamical and statistical ensembles, Tellus, 55 A, 16-30, 2003.
72. Schaake, J. C., Hamill, T. M., Buizza, R., and Clark, M.: The hydrological ensemble prediction experiment, B. Am. Meteorol. Soc. 88, 1541-1547, 2007.
73. Shamseldin, A. Y., O'Connor, K. M., and Liang G. C.: Methods for combining the outputs of different rainfall-runoff models, J. Hydrol., 197, 203-229, 1997.
74. Smith, M. B., Seo, D. J., Koren, V. I., Reed, S. M., Zhang, Z., Duan, Q., Moreda, F., and Cong, S.: The distributed model intercomparison project (DMIP): motivation and experiment design, J. Hydrology, 298 (1-4), 4-26, 2004.
75. Stensrud, D. J. and Yussouf, N.: Reliable probabilistic quantitative precipitation forecasts from a short-range ensemble forecasting system, Weather Forecast., 22 (1), 3-17, 2007.
76. Stephenson, D. B., Coelho, C. A. S., Balmaseda, M., and Doblas-Reyes, F. J.: Forecast assimilation: A unified framework for the combination of multi-model weather and climate predictions, Tellus, 57 A, 253-264, 2005.
77. Sugawara, M: Automatic calibration of the Tank Model, Hydrolog. Sci. J., 24 (3), 375-388, 1979.
78. Székely, G. J. and Rizzo, M. L.: A new test for multivariate normality, J. Multivariate Anal., 93 (1), 58-80, 2005.
79. Talagrand, O., Vautard, R., and Strauss, B.: Evaluation of the probabilistic prediction systems, in: Proceedings, ECMWF Workshop on Predictability, Shinfield Park, Reading, Berkshire, ECMWF, 1-25, 1999.
80. Thiery, D.: Utilisation d'un modèle global pour identifier sur un niveau pièzometrique des influences multiples dues à diverses activités humaines, IAHS-AISH P., 136, 71-77, 1982.
81. Thyer, M., Renard, B., Kavetski, D., Kuczera, G., Franks, S. W., and Srikanthan, S.: Critical evaluation of parameter consistency and predictive uncertainty in hydrological modeling: A case study using Bayesian total error analysis, Water Resour. Res., 45, W00B14, doi:10.1029/2008WR006825, 2009.
82. Viney, N. R., Bormann, H., Breuer, L., Bronstert, A., Croke, B. F. W., Frede, H., Gräff, T., Hubrechts, L., Huisman, J. A., Jakeman, A. J., Kite, G. W., Lanini, J., Leavesley, G., Lettenmaier, D. P., Lindström, G., Seibert, J., Sivapalan, M. and Willems, P.: Assessing the impact of land use change on hydrology by ensemble modelling (LUCHEM) II: Ensemble combinations and predictions, Adv. Water Resour., 32 (2), 147-158, 2009.
83. Vrugt, J. A. and Robinson, B. A.: Treatment of uncertainty using ensemble methods: Comparison of sequential data assimilation and Bayesian model averaging, Water Resour. Res., 43, W01411, doi:10.1029/2005WR004838, 2007. (Pubitemid 46364289)
84. Vrugt, J. A., ter Braak, C. J. F., Clark, M. P., Hyman, J. M., and Robinson B. A.: Treatment of input uncertainty in hydrologic modeling: Doing hydrology backward with Markov chain Monte Carlo simulation, Water Resour. Res., 44, W00B09, doi:10.1029/2007WR006720, 2008.
85. Wang, X. and Bishop, C. H.: Improvement of ensemble reliability with a new dressing kernel, Q. J. Roy. Meteor. Soc., 31, 965-986, 2005.
86. Warmerdam, P. M., Khole, J., and Chormansky, J.: Modelling rainfall-runoff process in the Hupsele Breek research Bassin, Ecohydrological process in small basins, in: Proceedings of the Strasbourg Conference (24-26 September 1996), IHP-V/Technical documents in Hydrology, 14, 155-160, 1997.
87. Weber, F., Perreault, L., Fortin, V., and Gaudet, J.: Performance measures for probabilistic hydrologic forecasts used at BC-Hydro and Hydro-Québec, in: EGU Conference, April 2006.
88. Wilks, D. S.: Statistical Methods in the Atmospheric Sciences, Academic Press, 1995.
89. WMO, Exchange of long range forecasts verification scores: A verification system for long range forecasts, in: Commission for basic systems meeting of expert team to develop a verification system for long range forecasts, Montreal, Canada, 22-26 April 2002, 2002.
90. Wöhling, T. and Vrugt, J. A.: Combining multiobjective optimization and Bayesian model averaging to calibrate forecast ensembles of soil hydraulic models, Water Resour. Res., 44, W12432, doi:10.1029/2008WR007154, 2008.
91. Wood, A. W. and Schaake, J. C.: Correcting errors in streamflow forecast ensemble mean and spread, J. Hydrometeorol., 9 (1), 132-148, 2008.
92. Yadav, M., Wagener, T., and Gupta, H.: Regionalization of constraints on expected watershed response behavior for improved predictions in ungauged basins, Adv. Water Resour., 30 (8), 1756-1774, 2007.