Stochastic representations of model uncertainties at ECMWF: state of the art and future vision

Quarterly Journal of the Royal Meteorological Society

Volumn 143, Issue 707, 2017, Pages 2315-2339

  Leutbecher, Martin ^a   Lock, Sarah Jane ^a   Ollinaho, Pirkka ^b   Lang, Simon T K ^a   Balsamo, Gianpaolo ^a   Bechtold, Peter ^a   Bonavita, Massimo ^a   Christensen, Hannah M ^c   Diamantakis, Michail ^a   Dutra, Emanuel ^a   English, Stephen ^a   Fisher, Michael ^a   Forbes, Richard M ^a   Goddard, Jacqueline ^a   Haiden, Thomas ^a   Hogan, Robin J ^a   Juricke, Stephan ^c   Lawrence, Heather ^a   MacLeod, Dave ^c   Magnusson, Linus ^a   Malardel, Sylvie ^a   Massart, Sebastien ^a   Sandu, Irina ^a   Smolarkiewicz, Piotr K ^a   Subramanian, Aneesh ^c   Vitart, Frédéric ^a   Wedi, Nils ^a   Weisheimer, Antje ^a,c   more..

^a EUROPEAN CENTRE FOR MEDIUM RANGE WEATHER FORECASTS   (United Kingdom)

^b FINNISH METEOROLOGICAL INSTITUTE   (Finland)

^c UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

dynamical core; Earth system model; ensemble data assimilation; ensemble forecasts; model uncertainty; numerical weather prediction; stochastic parametrization; weak constraint 4D Var

Indexed keywords

COMPUTATIONAL EFFICIENCY; FORECASTING; STOCHASTIC SYSTEMS; UNCERTAINTY ANALYSIS; WEATHER FORECASTING;

DYNAMICAL CORE; EARTH SYSTEM MODEL; ENSEMBLE DATA ASSIMILATION; ENSEMBLE FORECASTS; MODEL UNCERTAINTIES; NUMERICAL WEATHER PREDICTION; STOCHASTIC PARAMETRIZATION; WEAK-CONSTRAINT 4D-VAR;

STOCHASTIC MODELS;

DATA ASSIMILATION; ENSEMBLE FORECASTING; NUMERICAL MODEL; PERTURBATION; PREDICTION; STOCHASTICITY; UNCERTAINTY ANALYSIS; WEATHER FORECASTING;

EID: 85016942223     PISSN: 00359009     EISSN: 1477870X     Source Type: Journal    
DOI: 10.1002/qj.3094     Document Type: Review

References (133)

1. Andrejczuk M, Cooper F, Juricke S, Palmer T, Weisheimer A, Zanna L. 2016. Oceanic stochastic parametrisation in a seasonal forecast system. Mon. Weather Rev. 144: 1867–1875. https://doi.org/10.1175/MWR-D-15-0245.1.
2. Arnold HM. 2013. ‘Stochastic parametrisation and model uncertainty’, PhD thesis. University of Oxford: Oxford, UK.
3. Augier P, Lindborg E. 2013. A new formulation of the spectral energy budget of the atmosphere, with application to two high-resolution general circulation models. J. Atmos. Sci. 70: 2293–2308.
4. Baker L, Rudd A, Migliorini S, Bannister R. 2014. Representation of model error in a convective-scale ensemble prediction system. Nonlinear Processes Geophys. 21: 19–39.
5. Balsamo G, Agust-Panareda A, Albergel C, Beljaars A, Boussetta S, Dutra E, Komori T, Lang S, Munoz-Sabater J, Pappenberger F, Rosnay PD, Sandu I, Wedi N, Weisheimer A, Wetterhall F, Zsoter E. 2014. ‘Representing the Earth surfaces in the Integrated Forecasting System: Recent advances and future challenges’, Technical Memorandum 729. ECMWF: Reading, UK.
6. Bauer P, Thorpe A, Brunet G. 2015. The quiet revolution of numerical weather prediction. Nature 525: 47–55.
7. Bechtold P, Köhler M, Jung T, Doblas-Reyes F, Leutbecher M, Rodwell MJ, Vitart F, Balsamo G. 2008. Advances in simulating atmospheric variability with the ECMWF model: From synoptic to decadal time-scales. Q. J. R. Meteorol. Soc. 134: 1337–1351.
8. Berner J, Shutts GJ, Leutbecher M, Palmer TN. 2009. A spectral stochastic kinetic energy backscatter scheme and its impact on flow-dependent predictability in the ECMWF ensemble prediction system. J. Atmos. Sci. 66: 603–626.
9. Berner J, Ha SY, Hacker JP, Fournier A, Snyder C. 2011. Model uncertainty in a mesoscale ensemble prediction system: Stochastic versus multiphysics representations. Mon. Weather Rev. 139: 1972–1995. https://doi.org/10.1175/2010MWR3595.1.
10. Berner J, Fossell KR, Ha SY, Hacker JP, Snyder C. 2015. Increasing the skill of probabilistic forecasts: Understanding performance improvements from model-error representations. Mon. Weather Rev. 143: 1295–1320. https://doi.org/10.1175/MWR-D-14-00091.1.
11. Bishop C. 2016. Workshop on model uncertainty. In Using Trajectories of Ensemble Analyses and Tools from Weak Constraint 4DVar to Represent Model Uncertainty, 64. ECMWF/WWRP: Reading, UK. www.ecmwf.int/en/elibrary/16551-ecmwf-wwrp-workshop-model-uncertainty-proceedings (accessed 19 July 2017).
12. Bonavita M, Hólm E, Isaksen L, Fisher M. 2014. ‘The evolution of the ECMWF hybrid data assimilation system’, Technical Memorandum 743. ECMWF: Reading, UK.
13. Bouttier F, Vié B, Nuissier O, Raynaud L. 2012. Impact of stochastic physics in a convection-permitting ensemble. Mon. Weather Rev. 140: 3706–3721. https://doi.org/10.1175/MWR-D-12-00031.1.
14. Bowler NE. 2017. On the diagnosis of model error statistics using weak-constraint data assimilation. Q. J. R. Meteorol. Soc. 143: 1916–1928. doi:10.1002/qj.3051.
15. Bowler NE, Arribas A, Mylne KR. 2008a. The benefits of multianalysis and poor man's ensembles. Mon. Weather Rev. 136: 4113–4129.
16. Bowler NE, Arribas A, Mylne KR, Robertson KB, Beare SE. 2008b. The MOGREPS short-range ensemble prediction system. Q. J. R. Meteorol. Soc. 134: 703–722.
17. Brankart JM, Candille G, Garnier F, Calone C, Melet A, Bouttier PA, Brasseur P, Verron J. 2015. A generic approach to explicit simulation of uncertainty in the NEMO ocean model. Geosci. Model Dev. 8: 1285–1297. https://doi.org/10.5194/gmd-8-1285-2015.
18. Buizza R, Leutbecher M. 2015. The forecast skill horizon. Q. J. R. Meteorol. Soc. 141: 3366–3382. https://doi.org/10.1002/qj.2619.
19. Buizza R, Miller M, Palmer TN. 1999. Stochastic representation of model uncertainties in the ECMWF ensemble prediction system. Q. J. R. Meteorol. Soc. 125: 2887–2908.
20. Buizza R, Leutbecher M, Isaksen L. 2008. Potential use of an ensemble of analyses in the ECMWF ensemble prediction system. Q. J. R. Meteorol. Soc. 134: 2051–2066.
21. Cardinali C, Z̆agar N, Radnoti G, Buizza R. 2014. Representing model error in ensemble data assimilation. Nonlinear Processes Geophys. 21: 971–985.
22. Charron M, Pellerin G, Spacek L, Houtekamer PL, Gagnon N, Mitchell HL, Michelin L. 2010. Toward random sampling of model error in the Canadian ensemble prediction system. Mon. Weather Rev. 138: 1877–1901.
23. Christensen HM, Lock SJ, Moroz IM, Palmer TN. 2017. Introducing independent patterns into the Stochastically Perturbed Parametrization Tendencies (SPPT) scheme. Q. J. R. Meteorol. Soc., doi:10.1002/qj.3075.
24. Clark TL. 1979. Numerical simulations with a three-dimensional cloud model: Lateral boundary condition experiments and multicellular severe storm simulations. J. Atmos. Sci. 36: 2191–2215.
25. Cossette JF, Smolarkiewicz PK, Charbonneau P. 2014. The Monge–Ampère trajectory correction for semi-Lagrangian schemes. J. Comput. Phys. 274: 208–229.
26. Cullen M, Salmond D, Smolarkiewicz P. 2000. ‘Key numerical issues for the future development of the ECMWF model’. In Proceedings of the ECMWF Workshop on Developments in Numerical Methods for Very High Resolution Global Models, 5–7. ECMWF: Reading, UK.
27. Davies L, Jakob C, May P, Kumar VV, Xie S. 2013. Relationships between the large-scale atmosphere and the small-scale convective state for Darwin, Australia. J. Geophys. Res. Atmos. 118: 11534–11545. doi:10.1002/jgrd.50645.
28. Davini P, von Hardenberg J, Corti S, Christensen HM, Juricke S, Subramanian A, Watson PAG, Weisheimer A, Palmer TN. 2017. Climate SPHINX: Evaluating the impact of resolution and stochastic physics parameterisations in climate simulations. Geosci. Model Dev. Discuss. 10: 1383–1402. https://doi.org/10.5194/gmd-10-1383-2017.
29. Deconinck W, Hamrud M, Kühnlein C, Mozdzynski G, Smolarkiewicz PK, Szmelter J, Wedi NP. 2016. Accelerating Extreme-Scale Numerical Weather Prediction. In Parallel Processing and Applied Mathematics, 583–593. Springer International Publishing.
30. DeMott CA, Klingaman NP, Woolnough SJ. 2015. Atmospheric–ocean coupled processes in the Madden–Julian oscillation. Rev. Geophys. 53: 1099–1154. https://doi.org/10.1002/2014RG000478.
31. Derber J, Bouttier F. 1999. A reformulation of the background-error covariance in the ECMWF global data assimilation system. Tellus A 51: 195–221.
32. Descamps L, Labadie C, Joly A, Bazile E, Arbogast P, Cébron P. 2015. PEARP, the Météo-France short-range ensemble prediction system. Q. J. R. Meteorol. Soc. 141: 1671–1685. https://doi.org/10.1002/qj.2469.
33. Diamantakis M, Magnusson L. 2016. Sensitivity of the ECMWF model to semi-Lagrangian departure point iterations. Mon. Weather Rev. 144: 3233–3250. https://doi.org/10.1175/MWR-D-15-0432.1.
34. Durran DR, Gingrich M. 2014. Atmospheric predictability: Why butterflies are not of practical importance. J. Atmos. Sci. 71: 2476–2488.
35. Eckermann SD. 2011. Explicitly stochastic parameterization of nonorographic gravity wave drag. J. Atmos. Sci. 68: 1749–1765. https://doi.org/10.1175/2011JAS3684.1.
36. ECMWF/WWRP. (ed.) 2016. In Proceedings of Workshop on Model Uncertainty. ECMWF: Reading, UK. www.ecmwf.int/en/elibrary/16551-ecmwf-wwrp-workshop-model-uncertainty-proceedings (accessed 19 July 2017).
37. Fisher M. 2003. ‘Background error covariance modelling.’ In Proceedings of the ECMWF Seminar on Recent Developments in Data Assimilation for Atmosphere and Oceana, 45–63. ECMWF: Reading, UK, https://www.ecmwf.int/en/learning/workshops-and-seminars/past-workshops/2003-annual-seminar (accessed 19 July 2017).
38. Flemming J, Inness A, Flentje H, Huijnen V, Moinat P, Schultz M, Stein O. 2009. Coupling global chemistry transport models to ECMWF's Integrated Forecast System. Geosci. Model Dev. 2: 253–265.
39. Flemming J, Benedetti A, Inness A, Engelen R, Jones L, Huijnen V, Remy S, Parrington M, Suttie M, Bozzo A, Peuch VH, Akritidis D, Katragkou E. 2016. The CAMS interim reanalysis of carbon monoxide, ozone and aerosol for 2003–2015. Atmos. Chem. Phys. Discuss. 2016: 1–82. https://doi.org/10.5194/acp-2016-666.
40. Gebhardt C, Theis S, Paulat M, Bouallégue ZB. 2011. Uncertainties in COSMO-DE precipitation forecasts introduced by model perturbations and variation of lateral boundaries. Atmos. Res. 100: 168–177. https://doi.org/10.1016/j.atmosres.2010.12.008.
41. Gneiting T. 2002. Nonseparable, stationary covariance functions for space–time data. J. Am. Stat. Assoc. 97: 590–600.
42. Gneiting T. 2014. ‘Calibration of medium-range weather forecasts’, Technical Memorandum 719. ECMWF: Reading, UK.
43. Grabowski WW, Smolarkiewicz PK. 1999. CRCP: A cloud resolving convection parameterization for modeling the tropical convecting atmosphere. Physica D 133: 171–178.
44. Haiden T, Duffy S. 2016. Use of high-density observations in precipitation verification. ECMWF Newsl. 147: 20–25.
45. Haiden T, Forbes R, Ahlgrimm M, Bozzo A. 2015. The skill of ECMWF cloudiness forecasts. ECMWF Newsl. 143: 14–19.
46. Hodyss D, McLay JG, Moskaitis J, Serra EA. 2014. Inducing tropical cyclones to undergo Brownian motion: A comparison between Itô and Stratonovich in a numerical weather prediction model. Mon. Weather Rev. 142: 1982–1996.
47. Hogan RJ, Bozzo A. 2015. Mitigating errors in surface temperature forecasts using approximate radiation updates. J. Adv. Model. Earth Syst. 7: 836–853.
48. Holm DD. 2015. Variational principles for stochastic fluid dynamics. Proc. R. Soc. A 471. https://doi.org/10.1098/rspa.2014.0963.
49. IFS Documentation. 2016. IFS Documentation CY43R1. ECMWF: Reading, UK. Operational implementation: 16 November 2016. www.ecmwf.int/en/elibrary/17117-part-iv-physical-processes (accessed 19 July 2017).
50. Inness PM, Slingo JM, Guilyardi E, Cole J. 2003. Simulation of the Madden–Julian oscillation in a coupled general circulation model. Part II: The role of the basic state. J. Clim. 16: 365–382.
51. Isaksen L, Bonavita M, Buizza R, Fisher M, Haseler J, Leutbecher M, Raynaud L. 2010. ‘Ensemble of data assimilations at ECMWF’, Technical Memorandum 636. ECMWF: Reading, UK. https://www.ecmwf.int/en/elibrary/10125-ensemble-data-assimilations-ecmwf (accessed 19 July 2017).
52. Iversen T, Deckmyn A, Santos C, Sattler K, Bremnes JB, Feddersen H, Frogner IL. 2011. Evaluation of ‘GLAMEPS’—a proposed multimodel EPS for short range forecasting. Tellus A 63: 513–530. https://doi.org/10.1111/j.1600-0870.2010.00507.x.
53. Juricke S, Goessling HF, Jung T. 2014. Potential sea ice predictability and the role of stochastic sea ice strength perturbations. Geophys. Res. Lett. 41: 8396–8403. https://doi.org/10.1002/2014GL062081.2014GL062081.
54. Juricke S, Palmer TN, Zanna L. 2017. Stochastic subgrid-scale ocean mixing: Impacts on low frequency variability. J. Clim. 30: 4997–5019. https://doi.org/10.1175/JCLI-D-16-0539.1.
55. Khairoutdinov MF, Randall DA. 2003. Cloud resolving modeling of the ARM summer 1997 IOP: Model formulation, results, uncertainties, and sensitivities. J. Atmos. Sci. 60: 607–625.
56. Kleiber W. 2016. High resolution simulation of nonstationary Gaussian random fields. Comput. Stat. Data Anal. 101: 277–288. http://dx.doi.org/10.1016/j.csda.2016.03.005.
57. Klinker E, Rabier F, Kelly G, Mahfouf JF. 2000. The ECMWF operational implementation of four-dimensional variational assimilation. III: Experimental results and diagnostics with operational configuration. Q. J. R. Meteorol. Soc. 126: 1191–1215.
58. Kober K, Craig GC. 2016. Physically based stochastic perturbations (PSP) in the boundary layer to represent uncertainty in convective initiation. J. Atmos. Sci. 73: 2893–2911. https://doi.org/10.1175/JAS-D-15-0144.1.
59. Kroese D, Botev Z. 2013. Spatial process generation, lectures on stochastic geometry, spatial statistics and random fields. In Analysis, Modeling and Simulation of Complex Structures, II. Springer-Verlag: Berlin. In Press.
60. Kurowski MJ, Grabowski WW, Smolarkiewicz PK. 2015. Anelastic and compressible simulation of moist dynamics at planetary scales. J. Atmos. Sci. 72: 3975–3995.
61. Lang STK, Bonavita M, Leutbecher M. 2015. On the impact of re-centring initial conditions for ensemble forecasts. Q. J. R. Meteorol. Soc. 141: 2571–2581. https://doi.org/10.1002/qj.2543.
62. Leutbecher M, Lang STK. 2014. On the reliability of ensemble variance in subspaces defined by singular vectors. Q. J. R. Meteorol. Soc. 140: 1453–1466. https://doi.org/10.1002/qj.2229.
63. Lewis JM. 2005. Roots of ensemble forecasting. Mon. Weather Rev. 133: 1865–1885.
64. Lindgren F, Rue H, Lindström J. 2011. An explicit link between Gaussian fields and Gaussian Markov random fields: The stochastic partial differential equation approach. J. R. Stat. Soc. B 73: 423–498.
65. Lorenz EN. 1969. The predictability of a flow which possesses many scales of motion. Tellus 21: 289–307.
66. Lott F, Guez L, Maury P. 2012. A stochastic parameterization of non-orographic gravity waves: Formalism and impact on the Equatorial stratosphere. Geophys. Res. Lett. 39: L06 807. https://doi.org/10.1029/2012GL051001.
67. McCabe A, Swinbank R, Tennant W, Lock A. 2016. Representing model uncertainty in the Met Office convection-permitting ensemble prediction system and its impact on fog forecasting. Q. J. R. Meteorol. Soc. 142: 2897–2910. https://doi.org/10.1002/qj.2876.
68. MacLeod DA, Cloke HL, Pappenberger F, Weisheimer A. 2016. Improved seasonal prediction of the hot summer of 2003 over Europe through better representation of uncertainty in the land surface. Q. J. R. Meteorol. Soc. 142: 79–90. https://doi.org/10.1002/qj.2631.
69. Mahfouf JF, Rabier F. 2000. The ECMWF operational implementation of four-dimensional variational assimilation. II: Experimental results with improved physics. Q. J. R. Meteorol. Soc. 126: 1171–1190. https://doi.org/10.1002/qj.49712656416.
70. Malardel S, Wedi NP. 2016. How does subgrid-scale parametrization influence nonlinear spectral energy fluxes in global NWP models?. J. Geophys. Res. Atmos. 121: 5395–5410. https://doi.org/10.1002/2015JD023970.
71. Malardel S, Wedi N, Deconinck W, Diamantakis M, Kühnlein C, Mozdzynski G, Hamrud M, Smolarkiewicz P. 2016. A new grid for the IFS. ECMWF Newsl. 146: 23–28.
72. Massart S, Bonavita M. 2016. ‘Ensemble of data assimilations applied to the CAMS greenhouse gases analysis’, Technical Memorandum 780. ECMWF: Reading, UK.
73. Mauritsen T, Stevens B, Roeckner E, Crueger T, Esch M, Giorgetta M, Haak H, Jungclaus J, Klocke D, Matei D, Mikolajewicz U, Notz D, Pincus R, Schmidt H, Tomassini L. 2012. Tuning the climate of a global model. J. Adv. Model. Earth Syst. 4. https://doi.org/10.1029/2012MS000154.
74. Morcrette JJ, Barker HW, Cole JNS, Iacono MJ, Pincus R. 2008a. Impact of a new radiation package, McRad, in the ECMWF Integrated Forecasting System. Mon. Weather Rev. 136: 4773–4798. https://doi.org/10.1175/2008MWR2363.1.
75. Morcrette JJ, Mozdzynski G, Leutbecher M. 2008b. A reduced radiation grid for the ECMWF Integrated Forecasting System. Mon. Weather Rev. 136: 4760–4772.
76. Myhre G, Shindell D, Bréon FM, Collins W, Fuglestvedt J, Huang J, Koch D, Lamarque JF, Lee D, Mendoza B, Nakajima T, Robock A, Stephens G, Takemura T, Zhang H. 2013. Anthropogenic and natural radiative forcing. In Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change Stocker TF, Qin D, Plattner GK, Tignor M, Allen SK, Doschung J, Nauels A, Xia Y, Bex V, Midgley PM. (eds.): 659–740. Cambridge University Press. http://www.climatechange2013.org/.
77. Nastrom GD, Gage KS. 1985. A climatology of atmospheric wavenumber spectra of wind and temperature observed by commercial aircraft. J. Atmos. Sci. 42: 950–960. https://doi.org/10.1175/1520-0469(1985)042<0950:ACOAWS>2.0.CO;2.
78. Ollinaho P, Lock SJ, Leutbecher M, Bechtold P, Beljaars A, Bozza A, Forbes RM, Haiden T, Hogan RJ, Sandu I. 2017. Towards process-level representation of model uncertainties: Stochastically perturbed parametrisations in the ECMWF ensemble. Q. J. R. Meteorol. Soc. 143: 408–422. https://doi.org/10.1002/qj.2931.
79. Orth R, Dutra E, Pappenberger F. 2016. Improving weather predictability by including land surface model parameter uncertainty. Mon. Weather Rev. 144: 1551–1569. https://doi.org/10.1175/MWR-D-15-0283.1.
80. Palmer TN. 2001. A nonlinear dynamical perspective on model error: A proposal for non-local stochastic-dynamic parametrization in weather and climate prediction models. Q. J. R. Meteorol. Soc. 127: 279–304.
81. Palmer TN. 2012. Towards the probabilistic Earth-system simulator: A vision for the future of climate and weather prediction. Q. J. R. Meteorol. Soc. 138: 841–861. https://doi.org/10.1002/qj.1923.
82. Palmer T. 2014. Climate forecasting: Build high-resolution global climate models. Nature 515: 338–339. https://doi.org/10.1038/515338a.
83. Palmer TN, Shutts GJ, Hagedorn R, Doblas-Reyes FJ, Jung T, Leutbecher M. 2005. Representing model uncertainty in weather and climate prediction. Annu. Rev. Earth Planet. Sci. 33: 163–193.
84. Palmer T, Buizza R, Doblas-Reyes F, Jung T, Leutbecher M, Shutts G, Steinheimer M, Weisheimer A. 2009. ‘Stochastic parametrization and model uncertainty’, Technical Memorandum 598. ECMWF: Reading, UK.
85. Pegion P, Bates G, Gehne M, Hamill T, Kolczynski W, Whitaker J, Zhu Y. 2016. Stochastic parameterization development in the NOAA/NCEP Global Forecast System. In Workshop on Model Uncertainty, 47. ECMWF/WWRP: Reading. UK. www.ecmwf.int/en/elibrary/16551-ecmwf-wwrp-workshop-model-uncertainty-proceedings (accessed 19 July 2017).
86. Peters K, Jakob C, Davies L, Khouider B, Majda AJ. 2013. Stochastic behavior of tropical convection in observations and a multicloud model. J. Atmos. Sci. 70: 3556–3575.
87. Piccolo C, Cullen M. 2016. Ensemble data assimilation using a unified representation of model error. Mon. Weather Rev. 144: 213–224. https://doi.org/10.1175/MWR-D-15-0270.1.
88. Plant R, Craig GC. 2008. A stochastic parameterization for deep convection based on equilibrium statistics. J. Atmos. Sci. 65: 87–105.
89. Rabier F, Järvinen H, Klinker E, Mahfouf JF, Simmons A. 2000. The ECMWF operational implementation of four-dimensional variational assimilation. I: Experimental results with simplified physics. Q. J. R. Meteorol. Soc. 126: 1143–1170.
90. Randall D, Khairoutdinov M, Arakawa A, Grabowski W. 2003. Breaking the cloud parameterization deadlock. Bull. Am. Meteorol. Soc. 84: 1547–1564.
91. Raynaud L, Berre L, Desroziers G. 2012. Accounting for model error in the Météo-France ensemble data assimilation system. Q. J. R. Meteorol. Soc. 138: 249–262.
92. Reynolds C, Ridout J, McLay J. 2011. Examination of parameter variations in the U. S. Navy global ensemble. Tellus A 63: 841–857. https://doi.org/10.1111/j.1600-0870.2011.00532.x.
93. Rodwell MJ, Ferranti L, Haiden T, Magnusson L, Bidlot J, Bormann N, Dahoui M, Chiara GD, Duffy S, Forbes R, Hólm E, Ingleby B, Janousek M, Lang S, Mogensen K, Prates F, Rabier F, Richardson DS, Tsonevsky I, Vitart F, Yamaguchi M. 2015. ‘New developments in the diagnosis and verification of high-impact weather forecasts’. Technical Memorandum 759. ECMWF: Reading, UK.
94. Rodwell MJ, Lang STK, Ingleby NB, Bormann N, Hólm E, Rabier F, Richardson DS, Yamaguchi M. 2016. Reliability in ensemble data assimilation. Q. J. R. Meteorol. Soc. 142: 443–454. https://doi.org/10.1002/qj.2663.
95. Romine GS, Schwartz CS, Berner J, Fossell KR, Snyder C, Anderson JL, Weisman ML. 2014. Representing forecast error in a convection-permitting ensemble system. Mon. Weather Rev. 142: 4519–4541. https://doi.org/10.1175/MWR-D-14-00100.1.
96. Saetra Ø, Hersbach H, Bidlot JR, Richardson DS. 2004. Effects of observation errors on the statistics for ensemble spread and reliability. Mon. Weather Rev. 132: 1487–1501.
97. Sanchez C, Williams KD, Collins M. 2016. Improved stochastic physics schemes for global weather and climate models. Q. J. R. Meteorol. Soc. 142: 147–159.
98. Sandu I, Beljaars A, Bechtold P, Mauritsen T, Balsamo G. 2013. Why is it so difficult to represent stably stratified conditions in numerical weather prediction (NWP) models?. J. Adv. Model. Earth Syst. 5: 117–133. https://doi.org/10.1002/jame.20013.
99. Schefzik R, Thorarinsdottir TL, Gneiting T. 2013. Uncertainty quantification in complex simulation models using ensemble copula coupling. Stat. Sci. 28: 616–640. https://doi.org/10.1214/13-STS443.
100. Schlather M. 2012. Construction of covariance functions and unconditional simulation of random fields. In Advances and Challenges in Space-time Modelling of Natural Events, 25–54. Springer: Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17086-7_2.
101. Selz T, Craig GC. 2015a. Simulation of upscale error growth with a stochastic convection scheme. Geophys. Res. Lett. 42: 3056–3062. https://doi.org/10.1002/2015GL063525.
102. Selz T, Craig GC. 2015b. Upscale error growth in a high-resolution simulation of a summertime weather event over Europe. Mon. Weather Rev. 143: 813–827. https://doi.org/10.1175/MWR-D-14-00140.1.
103. Separovic L, Charron M, Erfani A, Gagnon N, Zadra A, Vaillancourt P. 2016. ‘Model error representation in the Canadian ensemble prediction systems’, Workshop on Model Uncertainty, Reading, UK. www.ecmwf.int/en/elibrary/16551-ecmwf-wwrp-workshop-model-uncertainty-proceedings (accessed 19 July 2017).
104. Shutts G. 2005. A kinetic energy backscatter algorithm for use in ensemble prediction systems. Q. J. R. Meteorol. Soc. 131: 3079–3102.
105. Shutts GJ. 2013. Coarse graining the vorticity equation in the ECMWF integrated forecasting system: The search for kinetic energy backscatter. J. Atmos. Sci. 70: 1233–1241. https://doi.org/10.1175/JAS-D-12-0216.1.
106. Shutts G. 2015. A stochastic convective backscatter scheme for use in ensemble prediction systems. Q. J. R. Meteorol. Soc. 141: 2602–2616. https://doi.org/10.1002/qj.2547.
107. Shutts G, Pallarès AC. 2014. Assessing parametrization uncertainty associated with horizontal resolution in numerical weather prediction models. Philos. Trans. R. Soc. A 372: 20130284. https://doi.org/10.1098/rsta.2013.0284.
108. Shutts GJ, Palmer TN. 2007. Convective forcing fluctuations in a cloud-resolving model: Relevance to the stochastic parameterization problem. J. Clim. 20: 187–202.
109. Shutts G, Leutbecher M, Weisheimer A, Stockdale T, Isaksen L, Bonavita M. 2011. Representing model uncertainty: Stochastic parametrizations at ECMWF. ECMWF Newsl. 129: 19–24.
110. Sigrist F, Künsch HR, Stahel WA. 2015. Stochastic partial differential equation based modelling of large space–time data sets. J. R. Stat. Soc. B 77: 3–33.
111. Smolarkiewicz PK, Pudykiewicz JA. 1992. A class of semi-Lagrangian approximations for fluids. J. Atmos. Sci. 49: 2082–2096.
112. Smolarkiewicz P, Deconinck W, Hamrud M, Kühnlein C, Mozdzynski G, Szmelter J, Wedi N. 2015. An all-scale, finite-volume module for the IFS. ECMWF Newsl. 145: 24–29.
113. Smolarkiewicz PK, Deconinck W, Hamrud M, Kühnlein C, Mozdzynski G, Szmelter J, Wedi NP. 2016. A finite-volume module for simulating global all-scale atmospheric flows. J. Comput. Phys. 314: 287–304.
114. Subramanian A, Weisheimer A, Palmer T, Vitart F, Bechtold P. 2017. Impact of stochastic physics on tropical precipitation and climate variability in the ECMWF IFS. Q. J. R. Meteorol. Soc. 143: 852–865. https://doi.org/10.1002/qj.2970.
115. Sullivan PP, Weil JC, Patton EG, Jonker HJ, Mironov DV. 2016. Turbulent winds and temperature fronts in large-eddy simulations of the stable atmospheric boundary layer. J. Atmos. Sci. 73: 1815–1840.
116. Tennant WJ, Shutts GJ, Arribas A, Thompson SA. 2011. Using a stochastic kinetic energy backscatter scheme to improve MOGREPS probabilistic forecast skill. Mon. Weather Rev. 139: 1190–1206.
117. Tennant WJ. 2016. Impacts of a spatially varying random parameter scheme on cloud cover in MOGREPS. In Workshop on model uncertainty. ECMWF/WWRP: Reading, UK, www.ecmwf.int/sites/default/files/elibrary/2016/16301-impacts-spatially-varying-random-parameterscheme-cloud-cover-mogreps.pdf.
118. Theis S, Buchhold M, Gebhardt C, Kohlhepp R, Röpnack A. 2016. ‘Model uncertainty representation in COSMO-DE-EPS’, In Workshop on Model Uncertainty, Reading, UK. www.ecmwf.int/en/elibrary/16551-ecmwf-wwrp-workshop-model-uncertainty-proceedings (accessed 19 July 2017).
119. Todling R. 2015a. A lag-1 smoother approach to system-error estimation: Sequential method. Q. J. R. Meteorol. Soc. 141: 1502–1513.
120. Todling R. 2015b. A complementary note to ‘A lag-1 smoother approach to system-error estimation’: The intrinsic limitations of residual diagnostics. Q. J. R. Meteorol. Soc. 141: 2917–2922.
121. Trémolet Y. 2007. Model-error estimation in 4D-Var. Q. J. R. Meteorol. Soc. 133: 1267–1280. https://doi.org/10.1002/qj.94.
122. Tsyrulnikov M, Gayfulin D. 2016. A limited-area spatio-temporal stochastic pattern generator for simulation of uncertainties in ensemble applications. Meteorol. Z.: In Press. https://doi.org/10.1127/metz/2017/0815.
123. Váňa F, Düben P, Lang S, Palmer T, Leutbecher M, Salmond D, Carver G. 2017. Single precision in weather forecasting models: An evaluation with the IFS. Mon. Weather Rev. 145: 495–502. https://doi.org/10.1175/MWR-D-16-0228.1.
124. Vitart F, Molteni F. 2010. Simulation of the Madden–Julian oscillation and its teleconnections in the ECMWF forecast system. Q. J. R. Meteorol. Soc. 136: 842–855.
125. Watson PAG, Christensen HM, Palmer TN. 2015. Does the ECMWF IFS convection parameterization with stochastic physics correctly reproduce relationships between convection and the large-scale state?. J. Atmos. Sci. 72: 236–242. https://doi.org/10.1175/JAS-D-14-0252.1.
126. Weaver A, Courtier P. 2001. Correlation modelling on the sphere using a generalized diffusion equation. Q. J. R. Meteorol. Soc. 127: 1815–1846.
127. Wedi NP, Bauer P, Deconinck W, Diamantakis M, Hamrud M, Kühnlein C, Malardel S, Mogensen K, Mozdzynski G, Smolarkiewicz P. 2015. ‘The modelling infrastructure of the Integrated Forecasting System: Recent advances and future challenges’, Technical Memorandum 760. ECMWF: Reading, UK.
128. Weisheimer A, Palmer TN, Doblas-Reyes FJ. 2011. Assessment of representations of model uncertainty in monthly and seasonal forecast ensembles. Geophys. Res. Lett. 38: L16703. https://doi.org/10.1029/2011GL048123.
129. Weisheimer A, Corti S, Palmer T, Vitart F. 2014. Addressing model error through atmospheric stochastic physical parametrizations: Impact on the coupled ECMWF seasonal forecasting system. Philos. Trans. A. Math. Phys. Eng. Sci 372: 20130290. https://doi.org/10.1098/rsta.2013.0290.
130. Wheeler MC, Hendon HH. 2004. An all-season real-time multivariate MJO index: Development of an index for monitoring and prediction. Mon. Weather Rev. 132: 1917–1932.
131. Wilks DS. 2005. Effects of stochastic parametrizations in the Lorenz ’96 system. Q. J. R. Meteorol. Soc. 131: 389–407.
132. Williams PD, Howe NJ, Gregory JM, Smith RS, Joshi MM. 2016. Improved climate simulations through a stochastic parameterization of ocean eddies. J. Clim. 29: 8763–8781. https://doi.org/10.1175/JCLI-D-15-0746.1.
133. Yamaguchi M, Lang STK, Leutbecher M, Rodwell MJ, Radnoti G, Bormann N. 2016. Observation-based evaluation of ensemble reliability. Q. J. R. Meteorol. Soc. 142: 506–514. https://doi.org/10.1002/qj.2675.