Verification of temperature, precipitation, and streamflow forecasts from the NOAA/NWS Hydrologic Ensemble Forecast Service (HEFS): 1. Experimental design and forcing verification

Journal of Hydrology

Volumn 519, Issue PD, 2014, Pages 2869-2889

  Brown, James D ^a   Wu, Limin ^b,d   He, Minxue ^c,d   Regonda, Satish ^c,d   Lee, Haksu ^b,d   Seo, Dong Jun ^e  

^a Bristol and West House   (United Kingdom)

^b LEN Technologies Inc   (United States)

^c RIVERSIDE TECHNOLOGY INC   (United States)

^d NOAA EARTH SYSTEM RESEARCH LABORATORY   (United States)

^e University of Texas at Arlington   (United States)

Author keywords

Bias correction; Ensemble verification; Operational forecasting; Precipitation; Skill; Temperature

Indexed keywords

CLIMATOLOGY; PRECIPITATION (CHEMICAL); STREAM FLOW; TEMPERATURE;

BIAS CORRECTION; GLOBAL FORECAST SYSTEMS; HISTORICAL OBSERVATION; NATIONAL WEATHER SERVICES; OPERATIONAL FORECASTING; RIVER FORECAST CENTERS; SKILL; STATISTICAL TECHNIQUES;

WEATHER FORECASTING;

EID: 84918831729     PISSN: 00221694     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jhydrol.2014.05.028     Document Type: Article

References (51)

1. Beven K.J. On model uncertainty, risk and decision making. Hydrol. Process. 2000, 14:2605-2606.
2. Bradley A.A., Schwartz S.S., Hashino T. Distributions-oriented verification of ensemble streamflow predictions. J. Hydrometeorol. 2004, 5(3):532-545.
3. Brier G.W. Verification of forecasts expressed in terms of probability. Mon. Weather Rev. 1950, 78:1-3.
4. Bröcker J., Smith L.A. Increasing the reliability of reliability diagrams. Weather Forecasting 2007, 22(3):651-661.
5. Brown J.D. Prospects for the open treatment of uncertainty in environmental research. Prog. Phys. Geogr. 2010, 34:75-100. 10.1177/0309133309357000.
6. Brown J.D., Seo D.-J. Evaluation of a nonparametric post-processor for bias-correction and uncertainty estimation of hydrologic predictions. Hydrol. Process. 2013, 27(1):83-105. 10.1002/hyp.9263.
7. Brown J.D., Demargne J., Seo D.-J., Liu Y. The Ensemble Verification System (EVS): a software tool for verifying ensemble forecasts of hydrometeorological and hydrologic variables at discrete locations. Environ. Model. Softw. 2010, 25:854-872.
8. Brown J.D., He M., Regonda S., Wu L., Lee H., Seo D.J., 2014. Verification of temperature, precipitation, and streamflow forecasts from the NOAA/NWS Hydrologic Ensemble Forecast Service (HEFS): 2. Streamflow verification, J. Hydrol. 519, 2847-2868.
9. Brown J.D., Seo D.-J., Du J. Verification of precipitation forecasts from NCEP's Short Range Ensemble Forecast (SREF) system with reference to ensemble streamflow prediction using lumped hydrologic models. J. Hydrometeorol. 2012, 13(3):808-836.
10. Clark M., Gangopadhyay S., Hay L., Rajagopalan B., Wilby R. The Schaake Shuffle: a method for reconstructing space-time variability in forecasted precipitation and temperature fields. J. Hydrometeorol. 2004, 5:243-262.
11. Cloke H.L., Pappenberger F., van Andel S.-J., Schaake J., Thielen J., Ramos M.-H. Hydrological ensemble prediction systems. Hydrol. Process. 2013, 27:1-4. 10.1002/hyp.9679.
12. Day G.N. Extended streamflow forecasting using NWSRFS. J. Water Resour. Plan. Manage. 1985, 111(2):157-170.
13. Demargne J., Brown J.D., Liu Y., Seo D.-J., Wu L., Toth Z., Zhu Y. Diagnostic verification of hydrometeorological and hydrologic ensembles. Atmos. Sci. Lett. 2010, 11(2):114-122.
14. Demargne J., Wu L., Regonda S., Brown J., Lee H., He M., Seo D.-J., Hartman R., Fresch M., Zhu Y. The science of NOAA's operational Hydrologic Ensemble Forecast Service. Bull. Am. Meteorol. Soc 2014, 95(1):79-98. 10.1175/BAMS-D-12-00081.1.
15. Demeritt D., Nobert S., Cloke H.L., Pappenberger F. The European Flood Alert System and the communication, perception, and use of ensemble predictions for operational flood risk management. Hydrol. Process. 2013, 27:147-157. 10.1002/hyp.9419.
16. Glahn H., Lowry D. The use of model output statistics (MOS) in objective weather forecasting. J. Appl. Meteorol. 1972, 11(8):1203-1211.
17. Gneiting T., Raftery A.E. Weather forecasting with ensemble methods. Science 2005, 310(5746):248-249.
18. Gneiting T., Balabdaoui F., Raftery A.E. Probabilistic forecasts, calibration and sharpness. J. Roy. Stat. Soc. Ser. B: Stat. Methodol. 2007, 69(2):243-268.
19. Green D.M., Swets J.M. Signal Detection Theory and Psychophysics 1966, John Wiley & Sons, New York, p. 455.
20. Hamill T.M., Whitaker J.S., Mullen S.L. Reforecasts: an important data set for improving weather predictions. Bull. Am. Meteorol. Soc. 2006, 87(1):33-46.
21. Hamill T.M., Hagedorn R., Whitaker J.S. Probabilistic forecast calibration using ECMWF and GFS ensemble reforecasts. Part II: Precipitation. Mon. Weather Rev. 2008, 136:2620-2632.
22. Hamill T.M., Bates G.T., Whitaker J.S., Murray D.R., Fiorino M., Galarneau T., Zhu Y., Lapenta W. NOAA's second-generation global medium-range ensemble reforecast dataset. Bull. Am. Meteorol. Soc. 2013, 94:1553-1556. 10.1175/BAMS-D-12-00014.1.
23. Hanley J. The robustness of the ''binormal'' assumptions used in fitting ROC curves. Med. Decis. Making 1988, 8:197-203.
24. Helton J.C., Johnson J.D., Salaberry C.J., Storlie C.B. Survey of sampling based methods for uncertainty and sensitivity analysis. Reliab. Eng. Syst. Safety 2006, 91:1175-1209.
25. Hengl T., Heuvelink G.B.M., Stein A. A generic framework for spatial prediction of soil variables based on regression-kriging. Geoderma 2004, 120(1-2):75-93. 10.1016/j.geoderma.2003.08.018.
26. Hersbach H. Decomposition of the continuous ranked probability score for ensemble prediction systems. Weather Forecasting 2000, 15:559-570.
27. Hsu W.-R., Murphy A.H. The attributes diagram: a geometrical framework for assessing the quality of probability forecasts. Int. J. Forecast. 1986, 2:285-293.
28. Forecast Verification: A Practitioner's Guide in Atmospheric Science 2011, John Wiley & Sons, Chichester. second ed. I.T. Jolliffe, D.B. Stephenson (Eds.).
29. Kavetski D., Franks S.W., Kuczera G. Confronting input uncertainty in environmental modeling. Calibration of Watershed Models 2002, 49-68. AGU Books, Washington, DC. Q. Duan, H. Gupta, S. Sorooshian, A.N. Rousseau, R. Turcotte (Eds.).
30. Kelly K.S., Krzysztofowicz R. A bivariate meta-Gaussian density for use in hydrology. Stoch. Hydrol. Hydraul. 1997, 11:17-31.
31. Kitzmiller D., Coauthors Evolving multisensor precipitation estimation methods: their impacts on flow prediction using a distributed hydrologic model. J. Hydrometeorol. 2011, 12:1414-1431.
32. Liu Y., Duan Q., Zhao L., Ye A., Tao Y., Miao C., Mu X., Schaake J.C. Evaluating the predictive skill of post-processed NCEP GFS ensemble precipitation forecasts in China's Huai river basin. Hydrol. Process. 2013, 27:57-74. 10.1002/hyp.9496.
33. Matott L.S., Babendreier J.E., Parucker S.T. Evaluating uncertainty in integrated environmental models: a review of concepts and tools. Water Resour. Res. 2009, 45:WO6421. doi: http://dx.doi.org/10.1029/2008WR007301.
34. Metz C.E., Pan X. "Proper'' binormal ROC curves: theory and maximum-likelihood estimation. J. Math. Psychol. 1999, 43:1-33.
35. Montanari A., Grossi G. Estimating the uncertainty of hydrological forecasts: a statistical approach. Water Resour. Res. 2008, 44:W00B08. doi: http://dx.doi.org/10.1029/2008WR006897.
36. Murphy A.H., Winkler R.L. A general framework for forecast verification. Mon. Weather Rev. 1987, 115:1330-1338.
37. Pappenberger F., Buizza R. The skill of ECMWF precipitation and temperature predictions in the Danube basin as forcings of hydrological models. Weather Forecasting 2009, 24:749-766.
38. Pappenberger F., Beven K.J., Hunter N., Bates P.D., Couweleeuw B.T., Thielen J., de Roo A.P.J. Cascading model uncertainty from medium range weather forecasts (10days) through a rainfall-runoff model to flood inundation predictions within the European Flood Forecasting System (EFFS). Hydrol. Earth Syst. Sci. 2005, 9:381-393.
39. Perica, S., 1998. Integration of meteorological forecasts/climate outlooks into an ensemble streamflow prediction system. In: 14th Conference on Probability and Statistics in the Atmospheric Sciences, American Meteorological Society, Phoenix, Arizona.
40. Raff, D., Brekke, L., Werner, K., Wood, A., White, K., 2013. Short-term water management decisions: user needs for improved climate, weather, and hydrologic information. In: A Report of the U.S. Army Corps of Engineers, Bureau of Reclamation, and National Oceanic and Atmospheric Administration, CWTS 2013-1. Available at: (accessed 01.04.13). http://www.ccawwg.us/docs/Short-Term_Water_Management_Decisions_Final_3_Jan_2013.pdf.
41. Ramos M.H., van Andel S.J., Pappenberger F. Do probabilistic forecasts lead to better decisions?. Hydrol. Earth Syst. Sci. Discuss. 2012, 9(13569-13607):2012. 10.5194/hessd-9-13569-2012.
42. Regonda S., Seo D.-J., Lawrence B., Brown J.D., Demargne J. Short-term ensemble streamflow forecasting using operationally produced single-valued streamflow forecasts - a Hydrologic Model Output Statistics (HMOS) approach. J. Hydrol. 2013, 497(8):80-96. 10.1016/j.jhydrol.2013.05.028.
43. Schaake J., Demargne J., Hartman R., Mullusky M., Welles E., Wu L., Herr H., Fan X., Seo D.J. Precipitation and temperature ensemble forecasts from single-value forecasts. Hydrol. Earth Syst. Sci. 2007, 4:655-717.
44. Seo D.-J., Herr H.D., Schaake J.C. A statistical post-processor for accounting of hydrologic uncertainty in short-range ensemble streamflow prediction. Hydrol. Earth Syst. Sci. 2006, 3:1987-2035.
45. Seo, D.-J., Demargne, J., Wu, L., Liu, Y., Brown, J.D., Regonda, S., Lee, H., 2010. Hydrologic ensemble prediction for risk-based water resources management and hazard mitigation. In: 4th Federal Interagency Hydrologic Modeling Conference, Las Vegas, NV, June 27-July 1, 2010.
46. Smith B.L., Yuter S.E., Neiman P.J., Kingsmill D.E. Water vapor fluxes and orographic precipitation over Northern California associated with a landfalling atmospheric river. Mon. Weather Rev. 2010, 138:74-100. http://dx.doi.org/10.1175/2009MWR2939.1.
47. van Andel S.J., Weerts A., Schaake J., Bogner K. Post-processing hydrological ensemble predictions intercomparison experiment. Hydrol. Process. 2013, 27:158-161. 10.1002/hyp.9595.
48. Wagner J.A. Weather and circulation of January 1979. Mon. Weather Rev. 1979, 107:499-506.
49. Wilks D.S. Statistical Methods in the Atmospheric Sciences 2006, Elsevier, San Diego. second ed.
50. Wilks D.S., Hamill T.M. Comparison of ensemble-MOS methods using GFS reforecasts. Mon. Weather Rev. 2007, 135:2379-2390. http://dx.doi.org/10.1175/MWR3402.1.
51. Wu L., Seo D.-J., Demargne J., Brown J.D., Cong S., Schaake J. Generation of ensemble precipitation forecast from single-valued quantitative precipitation forecast via meta-Gaussian distribution models. J. Hydrol. 2011, 399(3-4):281-298.