Uncertainty quantification in complex simulation models using ensemble copula coupling

Statistical Science

Volumn 28, Issue 4, 2013, Pages 616-640

  Schefzik, Roman ^a   Thorarinsdottir, Thordis L ^b   Gneiting, Tilmann ^a  

^a UNIVERSITY OF HEIDELBERG   (Germany)

^b NORWEGIAN COMPUTING CENTER   (Norway)

Author keywords

Bayesian model averaging; Empirical copula; Ensemble calibration; Nonhomogeneous regression; Numerical weather prediction; Probabilistic forecast; Schaake shuffle; Sklar's theorem

Indexed keywords

EID: 84891800962     PISSN: 08834237     EISSN: None     Source Type: Journal    
DOI: 10.1214/13-STS443     Document Type: Article

References (95)

1. AAS, K., CZADO, C., FRIGESSI, A. and BAKKEN, H. (2009). Pair-copula constructions of multiple dependence. Insurance Math. Econom. 44 182-198. MR2517884
2. BAO, L., GNEITING, T., GRIMIT, E. P., GUTTORP, P. and RAFTERY, A. E. (2010). Bias correction and Bayesian model averaging for ensemble forecasts of surface wind direction. Monthly Weather Review 138 1811-1821.
3. BEN BOUALLèGUE, Z. (2013). Calibrated short-range ensemble precipitation forecasts using extended logistic regression with interaction terms. Weather and Forecasting 28 515-524.
4. BERROCAL, V. J., RAFTERY, A. E. and GNEITING, T. (2007). Combining spatial statistical and ensemble information in probabilistic weather forecasts. Monthly Weather Review 135 1386- 1402.
5. BERROCAL, V. J., RAFTERY, A. E. and GNEITING, T. (2008). Probabilistic quantitative precipitation field forecasting using a two-stage spatial model. Ann. Appl. Stat. 2 1170-1193. MR2655654
6. BERROCAL, V. J., RAFTERY, A. E., GNEITING, T. and STEED, R. C. (2010). Probabilistic weather forecasting for winter road maintenance. J. Amer. Statist. Assoc. 105 522-537. MR2724842
7. BREMNES, J. B. (2004). Probabilistic forecasts of precipitation in terms of quantiles using NWP model output. Monthly Weather Review 132 338-347.
8. BREMNES, J. B. (2007). Improved calibration of precipitation forecasts using ensemble techniques. Part 2: Statistical calibration methods. Technical Report 04/2007, Norwegian Meteorological Institute.
9. BRöCKER, J. (2012). Evaluating raw ensembles with the continuous ranked probability score. Quarterly Journal of the Royal Meteorological Society 138 1611-1617.
10. BRöCKER, J. and SMITH, L. A. (2008). From ensemble forecasts to predictive distribution functions. Tellus Ser. A 60 663-678.
11. CHALOULOS, G. and LYGEROS, J. (2007). Effect of wind correlation on aircraft conflict probability. Journal of Guidance, Control, and Dynamics 30 1742-1752.
12. CHILèS, J.-P. and DELFINER, P. (2012). Geostatistics: Modeling Spatial Uncertainty, 2nd ed. Wiley, Hoboken, NJ. MR2850475
13. CHMIELECKI, R. M. and RAFTERY, A. E. (2011). Probabilistic visibility forecasting using Bayesian model averaging. Monthly Weather Review 139 1626-1636.
14. CLARK, M., GANGOPADHYAY, S., HAY, L., RAJAGOPALAN, B. and WILBY, R. (2004). The Schaake shuffle: A method for reconstructing space-time variability in forecasted precipitation and temperature fields. Journal of Hydrometeorology 5 243- 262.
15. CLOKE, H. L. and PAPPENBERGER, F. (2009). Ensemble flood forecasting: A review. Journal of Hydrology 375 613-626.
16. COOKE, R. (1906). Forecasts and verifications in Western Australia. Monthly Weather Review 34 23-24.
17. CRESSIE, N. and WIKLE, C. K. (2011). Statistics for Spatio- Temporal Data. Wiley, Hoboken, NJ. MR2848400
18. CZADO, C., GNEITING, T. and HELD, L. (2009). Predictive model assessment for count data. Biometrics 65 1254-1261. MR2756513
19. DAVISON, A. C., PADOAN, S. A. and RIBATET, M. (2012). Statistical modeling of spatial extremes. Statist. Sci. 27 161-186. MR2963980
20. DAWID, A. P. (1984). Statistical theory. The prequential approach. J. Roy. Statist. Soc. Ser. A 147 278-292. MR0763811
21. DEHEUVELS, P. (1979). La fonction de dépendance empirique et ses propriétés. Un test non paramétrique d'indépendance. Acad. Roy. Belg. Bull. Cl. Sci. (5) 65 274-292. MR0573609
22. DELLE MONACHE, L., HACKER, J. P., ZHOU, Y., DENG, X. and STULL, R. B. (2006). Probabilistic aspects of meteorological and ozone regional ensemble forecasts. Journal of Geophysical Research 111 D24307.
23. DEMARTA, S. andMCNEIL, A. J. (2005). The t copula and related copulas. International Statistical Review 73 111-129.
24. DETTE, H. and VOLGUSHEV, S. (2008). Non-crossing nonparametric estimates of quantile curves. J. R. Stat. Soc. Ser. B Stat. Methodol. 70 609-627. MR2420417
25. DI NARZO, A. F. andCOCCHI, D. (2010). A Bayesian hierarchical approach to ensemble weather forecasting. J. R. Stat. Soc. Ser. C. Appl. Stat. 59 405-422. MR2756542
26. ECKEL, A. F. and MASS, C. F. (2005). Aspects of effective mesoscale, short-range ensemble forecasting. Weather and Forecasting 20 328-350.
27. FERMANIAN, J.-D., RADULOVÍC , D. andWEGKAMP, M. (2004). Weak convergence of empirical copula processes. Bernoulli 10 847-860. MR2093613
28. FLOWERDEW, J. (2012). Calibration and combination of mediumrange ensemble precipitation forecasts. Technical Report 567, United Kingdom Met Office.
29. FRALEY, C., RAFTERY, A. E. and GNEITING, T. (2010). Calibrating multi-model forecast ensembles with exchangeable and missing members using Bayesian model averaging. Monthly Weather Review 138 190-202.
30. FRALEY, C.,RAFTERY, A. E.,GNEITING, T., SLOUGHTER, J.M. and BERROCAL, V. J. (2011). Probabilistic weather forecasting in R. R Journal 3 55-63.
31. GEL, Y., RAFTERY, A. E. and GNEITING, T. (2004). Calibrated probabilistic mesoscale weather field forecasting: The geostatistical output perturbation (GOP) method (with discussion and rejoinder). J. Amer. Statist. Assoc. 99 575-590.
32. GENEST, C. and FAVRE, A. C. (2007). Everything you always wanted to know about copula modeling but were afraid to ask. Journal of Hydrologic Engineering 12 347-368.
33. GLAHN, H. R., PEROUTKA, M.,WEIDENFELD, J.,WAGNER, J., ZYLSTRA, G. and SCHUKNECHT, B. (2009). MOS uncertainty estimates in an ensemble framework. Monthly Weather Review 137 246-268.
34. GNEITING, T. (2008). Editorial: Probabilistic forecasting. J. Roy. Statist. Soc. Ser. A 171 319-321. MR2427336
35. GNEITING, T. (2011). Making and evaluating point forecasts. J. Amer. Statist. Assoc. 106 746-762. MR2847988
36. GNEITING, T., BALABDAOUI, F. and RAFTERY, A. E. (2007). Probabilistic forecasts, calibration and sharpness. J. R. Stat. Soc. Ser. B Stat. Methodol. 69 243-268. MR2325275
37. GNEITING, T. and RAFTERY, A. E. (2005). Weather forecasting with ensemble methods. Science 310 248-249.
38. GNEITING, T. and RAFTERY, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. J. Amer. Statist. Assoc. 102 359-378. MR2345548
39. GNEITING, T., RAFTERY, A. E.,WESTVELD, A. H. and GOLDMAN, T. (2005). Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Monthly Weather Review 133 1098-1118.
40. GNEITING, T., STANBERRY, L. I., GRIMIT, E. P., HELD, L. and JOHNSON, N. A. (2008). Assessing probabilistic forecasts of multivariate quantities, with applications to ensemble predictions of surface winds (with discussion and rejoinder). TEST 17 211-264.
41. GRAF, S. and LUSCHGY, H. (2000). Foundations of Quantization for Probability Distributions. Lecture Notes in Math. 1730. Springer, Berlin. MR1764176
42. HAGEDORN, R., HAMILL, T. M. and WHITAKER, J. S. (2008). Probabilistic forecast calibration using ECMWF and GFS ensemble reforecasts. Part I: Two-meter temperatures. Monthly Weather Review 136 2608-2619.
43. HAGEDORN, R.,BUIZZA, R.,HAMILL, T. M., LEUTBECHER, M. and PALMER, T. N. (2012). Comparing TIGGE multimodel forecasts with reforecast-calibrated ECMWF ensemble forecasts. Quarterly Journal of the Royal Meteorological Society 138 1814-1827.
44. HAMILL, T. M. (2001). Interpretation of rank histograms for verifying ensemble forecasts. Monthly Weather Review 129 550- 560.
45. HAMILL, T. M. and COLUCCI, S. J. (1997). Verification of Eta- RSM short-range ensemble forecasts. Monthly Weather Review 125 1312-1327.
46. HAMILL, T. M., HAGEDORN, R. and WHITAKER, J. S. (2008). Probabilistic forecast calibration using ECMWF and GFS ensemble reforecasts. Part II: Precipitation. Monthly Weather Review 136 2620-2632.
47. HAMILL, T. M., WHITAKER, J. S. and MULLEN, S. L. (2006). Reforecasts: An important dataset for improving weather predictions. Bulletin of the American Meteorological Society 87 33-46.
48. JOE, H. (1997). Multivariate Models and Dependence Concepts. Monographs on Statistics and Applied Probability 73. Chapman & Hall, London. MR1462613
49. KALNAY, E. (2003). Atmospheric Modeling, Data Assimilation and Predictability. Cambridge Univ. Press, Cambridge.
50. KANN, A.,WITTMANN, C.,WANG, Y. andMA, X. (2009). Calibrating 2-m temperature of limited-area ensemble forecasts using high-resolution analysis. Monthly Weather Review 137 3373-3387.
51. KLEIBER, W., RAFTERY, A. E. and GNEITING, T. (2011). Geostatistical model averaging for locally calibrated probabilistic quantitative precipitation forecasting. J. Amer. Statist. Assoc. 106 1291-1303. MR2896836
52. KLEIBER, W., RAFTERY, A. E., BAARS, J., GNEITING, T., MASS, C. and GRIMIT, E. P. (2011). Locally calibrated probabilistic temperature foreasting using geostatistical model averaging and local Bayesian model averaging. Monthly Weather Review 139 2630-2649.
53. KNEIB, T. (2013). Beyond mean regression (with discussion and rejoinder). Statistical Modelling 13 275-385. KOLESáROVá, A.,MESIAR, R.,MORDELOVá, J. and SEMPI, C. (2006). Discrete copulas. IEEE Transactions on Fuzzy Systems 14 698-705.
54. KRZYSZTOFOWICZ, R. and TOTH, Z. (2008). Bayesian processor of ensemble (BPE): Concept and implementation. Slides presented at the 4th NCEP/NWS Ensemble User Workshop, Laurel, MD.
55. LEUTBECHER, M. and PALMER, T. N. (2008). Ensemble forecasting. J. Comput. Phys. 227 3515-3539. MR2400226
56. MAYOR, G., SUñER, J. and TORRENS, J. (2007). Sklar's theorem in finite settings. IEEE Transactions on Fuzzy Systems 15 410- 416.
57. MCNEIL, A. J., FREY, R. and EMBRECHTS, P. (2005). Quantitative Risk Management: Concepts, Techniques and Tools. Princeton Univ. Press, Princeton, NJ. MR2175089
58. MCNEIL, A. J. and NEšLEHOVá, J. (2009). Multivariate Archimedean copulas, d-monotone functions and l1-norm symmetric distributions. Ann. Statist. 37 3059-3097. MR2541455
59. MöLLER, A., LENKOSKI, A. and THORARINSDOTTIR, T. L. (2013). Multivariate probabilistic forecasting using ensemble Bayesian model averaging and copulas. Quarterly Journal of the Royal Meteorological Society 139 982-991.
60. MOLTENI, F., BUIZZA, R., PALMER, T. N. and PETROLIAGIS, T. (1996). The new ECMWF ensemble prediction system: Methodology and validation. Quarterly Journal of the Royal Meteorological Society 122 73-119.
61. NELSEN, R. B. (2006). An Introduction to Copulas, 2nd ed. Springer, New York. MR2197664
62. PALMER, T. N. (2002). The economic value of ensemble forecasts as a tool for risk assessment: From days to decades. Quarterly Journal of the Royal Meteorological Society 128 747-774.
63. PINSON, P. (2012). Adaptive calibration of (u, v)-wind ensemble forecasts. Quarterly Journal of the Royal Meteorological Society 138 1273-1284.
64. PINSON, P. (2013).Wind energy: Forecasting challenges for its operational management. Statist. Sci. 28 564-585.
65. PINSON, P., MADSEN, H., NIELSEN, H. A., PAPAEFTHYMIOU, G. and KLöCKL, B. (2009). From probabilistic forecasts to statistical scenarios of short-term wind power production. Wind Energy 12 51-62.
66. RAFTERY, A. E., KáRNý, M. and ETTLER, P. (2010). Online prediction under model uncertainty via dynamic model averaging: Application to a cold rolling mill. Technometrics 52 52-66. MR2654989
67. RAFTERY, A. E., GNEITING, T., BALABDAOUI, F. and POLAKOWSKI, M. (2005). Using Bayesian model averaging to calibrate forecast ensembles. Monthly Weather Review 133 1155- 1174.
68. ROQUELAURE, S. and BERGOT, T. (2008). A local ensemble prediction system for fog and low clouds: Construction, Bayesian model averaging calibration, and validation. Journal of Applied Meteorology and Climatology 47 3072-3088.
69. ROULIN, E. andVANNITSEM, S. (2012). Postprocessing of ensemble precipitation predictions with extended logistic regression based on hindcasts. Monthly Weather Review 140 874-888.
70. RUIZ, J. J. and SAULO, C. (2012). How sensitive are probabilistic precipitation forecasts to the choice of calibration algorithms and the ensemble generation method? Part I: Sensitivity to calibration methods. Meteorological Applications 19 302-313.
71. RüSCHENDORF, L. (1976). Asymptotic distributions of multivariate rank order statistics. Ann. Statist. 4 912-923. MR0420794
72. RüSCHENDORF, L. (2009). On the distributional transform, Sklar's theorem, and the empirical copula process. J. Statist. Plann. Inference 139 3921-3927. MR2553778
73. SCHAAKE, J., DEMARGNE, J., HARTMAN, R., MULLUSKY, M., WELLES, E., WU, L., HERR, H., FAN, X. and SEO, D. J. (2007). Precipitation and temperature ensemble forecasts from single-valued forecasts. Hydrology and Earth System Sciences Discussions 4 655-717.
74. SCHAAKE, J., PAILLEUX, J., THIELEN, J., ARRITT, R., HAMILL, T., LUO, L., MARTIN, E., MCCOLLOR, D. and PAPPENBERGER, F. (2010). Summary of recommendations of the first Workshop on Postprocessing and Downscaling Atmospheric Forecasts for Hydrologic Applications held at Météo- France, Toulouse, France, 15-18 June 2009. Atmospheric Science Letters 11 59-63.
75. SCHEFZIK, R. (2011). Ensemble copula coupling. Diploma thesis, Faculty of Mathematics and Informatics, Heidelberg Univ.
76. SCHEFZIK, R., THORARINSDOTTIR, T. L. and GNEITING, T. (2013). Supplement to "Uncertainty quantification in complex simulation models using ensemble copula coupling." DOI:10.1214/13-STS443SUPP.
77. SCHEUERER, M. (2013). Probabilistic quantitative precipitation forecasting using ensemble model output statistics. Quarterly Journal of the Royal Meteorological Society. To appear. DOI:10.1002/qj.2183.
78. SCHMEITS, M. J. and KOK, K. J. (2010). A comparison between raw ensemble output, (modified) Bayesian model averaging, and extended logistic regression using ECMWF ensemble precipitation reforecasts. Monthly Weather Review 138 4199-4211.
79. SCHOELZEL, C. and FRIEDERICHS, P. (2008). Multivariate nonnormally distributed random variables in climate research- Introduction to the copula approach. Nonlinear Processes in Geophysics 15 761-772.
80. SCHOELZEL, C. and HENSE, A. (2011). Probabilistic assessment of regional climate change in Southwest Germany by ensemble dressing. Climate Dynamics 36 2003-2014.
81. SCHUHEN, N., THORARINSDOTTIR, T. L. and GNEITING, T. (2012). Ensemble model output statistics for wind vectors. Monthly Weather Review 140 3204-3219.
82. SEGERS, J. (2012). Asymptotics of empirical copula processes under non-restrictive smoothness assumptions. Bernoulli 18 764- 782. MR2948900
83. SKLAR, M. (1959). Fonctions de répartition à n dimensions et leurs marges. Publ. Inst. Statist. Univ. Paris 8 229-231. MR0125600
84. SLOUGHTER, M., GNEITING, T. and RAFTERY, A. E. (2010). Probabilistic wind spread forecasting using ensembles and Bayesian model averaging. J. Amer. Statist. Assoc. 105 25-35. MR2757189
85. SLOUGHTER, J. M., GNEITING, T. and RAFTERY, A. E. (2013). Probabilistic wind vector forecasting using ensembles and Bayesian model averaging. Monthly Weather Review 141 2107- 2119.
86. SLOUGHTER, J. M., RAFTERY, A. E., GNEITING, T. and FRALEY, C. (2007). Probabilistic quantitative precipitation forecasting using Bayesian model averaging. Monthly Weather Review 135 3209-3220.
87. STEIN, M. L. (1999). Interpolation of Spatial Data: Some Theory for Kriging. Springer, New York.
88. THORARINSDOTTIR, T. L. and GNEITING, T. (2010). Probabilistic forecasts of wind speed: Ensemble model ouput statistics by using heteroscedastic censored regression. J. Roy. Statist. Soc. Ser. A 173 371-388. MR2751882
89. THORARINSDOTTIR, T. L. and JOHNSON, M. S. (2012). Probabilistic wind gust forecasting using non-homogeneous Gaussian regression. Monthly Weather Review 140 889-897.
90. TREFETHEN, N. (2012). From the President: Discrete or continuous?. SIAM News 45 4.
91. VAN DER VAART, A.W. andWELLNER, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York. MR1385671
92. VOISIN, N., PAPPENBERGER, F., LETTENMAIER, D. P., BUIZZA, R. and SCHAAKE, J. C. (2011). Application of a medium-range global hydrologic probabilistic forecast scheme to the Ohio River basin. Weather and Forecasting 26 425-446.
93. WILKS, D. S. (2002). Smoothing forecast ensembles with fitted probability distributions. Quarterly Journal of the Royal Meteorological Society 128 2821-2836.
94. WILKS, D. S. (2009). Extending logistic regression to provide fullprobability- distribution MOS forecasts. Meteorological Applications 16 361-368.
95. WILKS, D. S. and HAMILL, T. M. (2007). Comparison of ensemble-MOS methods using GFS reforecasts. Monthly Weather Review 135 2379-2390.