Forecast verification: Current status and future directions

Meteorological Applications

Volumn 15, Issue 1, 2008, Pages 3-18

  Casati, Barbara ^a,i   Wilson, L J ^a   Stephenson, D B ^b   Nurmi, P ^c   Ghelli, Anna ^d   Pocernich, M ^e   Damrath, U ^f   Ebert, E E ^g   Brown, B G ^e   Mason, S ^h  

^a ENVIRONMENT CANADA   (Canada)

^b UNIVERSITY OF EXETER   (United Kingdom)

^c FINNISH METEOROLOGICAL INSTITUTE   (Finland)

^d EUROPEAN CENTRE FOR MEDIUM RANGE WEATHER FORECASTS   (United Kingdom)

^e NATIONAL CENTER FOR ATMOSPHERIC RESEARCH   (United States)

^f DEUTSCHER WETTERDIENST   (Germany)

^g BUREAU OF METEOROLOGY   (Australia)

^h Och Spine at New York Presbyterian Hospitals   (United States)

ⁱ NONE   (Canada)

Author keywords

Extreme events verification; Operational verification; Probability forecasts and ensemble verification; Spatial verification approaches; User oriented verification; Value; Verification packages

Indexed keywords

FORECASTING; OPEN SOURCE SOFTWARE; OPEN SYSTEMS;

EXTREME EVENTS; PROBABILITY FORECASTS; SPATIAL VERIFICATIONS; USER ORIENTED; VALUE;

VERIFICATION;

ENSEMBLE FORECASTING; EXTREME EVENT; FORECASTING METHOD; PROBABILITY; SOFTWARE; SPATIAL ANALYSIS;

EID: 42549087576     PISSN: 13504827     EISSN: 14698080     Source Type: Journal    
DOI: 10.1002/met.52     Document Type: Conference Paper

References (132)

1. Agresti A. 2002. Categorical Data Analysis, 2nd edn. Wiley: Chichester, UK; 710.
2. Altalo MG, Hale M. 2004. Turning weather forecasts into business forecasts. Environmental Finance, May: 20-21.
3. Altalo MG, Smith LA. 2004. Using ensemble weather forecasts to manage utilities risk. Environmental Finance 20: 8-9.
4. Anderson JL. 1996. A method for producing and evaluating probabilistic forecasts from ensemble model integrations. Journal of Climate 9: 1518-1530.
5. Atger F. 2001. Verification of intense precipitation forecasts from single models and ensemble prediction systems. Nonlinear Processes in Geophysics 8: 401-417.
6. Atger F. 2004. Estimation of the reliability of ensemble-based probabilistic forecasts. Quarterly Journal of the Royal Meteorological Society 130: 627-646.
7. Baddeley AJ. 1992. An error metric for binary images. In Robust Computer Vision, Forstner W, Ruwiedel S (eds). Wichmann: Bonn, Germany; 59-78.
8. Baldwin ME, Lakshmivarahan S, Kain JS. 2002. Development of an "events-oriented" approach to forecast verification. Preprints, 19th Conference on Weather Analysis and Forecasting. American Meteorological Society: San Antonio, TX; 255-258.
9. Beirlant J, Goegebeur Y, Segers J, Teugels J. 2004. Statistics of Extremes: Theory and Applications. John Wiley and Sons: Chichester, UK; 490.
10. Bougeault P. 2002. WGNE survey of verification methods for numerical prediction of weather elements and severe weather events. CAS/JSC WGNE Report No. 18, Appendix C. WMO/TD.No. 1173, Toulouse, France. Available on the internet at http://www.wmo.ch/pages/prog/wcrp/pdf/wgne18rpt.pdf.
11. Bowler NE, Pierce CE, Seed AW. 2007. STEPS: A probabilistic forecasting scheme which merges an extrapolation nowcast with downscaled NWP. Quarterly Journal of the Royal Meteorological Society 132: 2127-2155.
12. Brewster KA. 2003. Phase correcting data assimilation and application to storm-scale numerical prediction. Monthly Weather Review 131: 480-507.
13. Brier GW. 1950. Verification of forecasts expressed in terms of probability. Monthly Weather Review 78: 1-3.
14. Briggs WM, Levine RA. 1997. Wavelets and field forecast verification. Monthly Weather Review 125: 1329-1341.
15. Briggs WM, Ruppert D. 2004. Assessing the skill of Yes/No predictions. Biometrics 61(3): 799-807.
16. Brill KF. 2002. Automated east-west phase error calculation. Technical report, US Department of Commerce/NOAA/NSW/NCEP/14PC. Available at http:/ /www.bom.gov.au/bmrc/wefor/staff/eee/verif/Brill/ Brill-EW-PhaseError.html.
17. Bröcker J, Smith LA. 2007a. Increasing the reliability of reliability diagrams. Weather and Forecasting 22(3): 651-661.
18. Bröcker J, Smith LA. 2007b. Scoring probabilistic forecasts: on the importance of being proper. Weather and Forecasting 22(2): 382-388.
19. Bröcker J, Smith LA. 2008. From ensemble forecasts to predictive distribution functions. Tellus in press.
20. Candille G, Talagrand O. 2005. Evaluation of probabilistic prediction systems for a scalar variable. Quarterly Journal of the Royal Meteorological Society 131: 2131-2150.
21. Carstensen B, Plummer M, Läärä E, Myatt M, Clayton D. 2007. Epi: A package for statistical analysis in epidemiology. R package version 0.7.0, http://www.pubhealth.ku.dk/~bxc/Epi.
22. Casati B, Wilson LJ. 2007. A new spatial scale decomposition of the Brier score: application to the verification of lightning probability forecasts. Monthly Weather Review 135: 3052-3069.
23. Casati B, Ross G, Stephenson DB. 2004. A new intensity-scale approach for the verification of spatial precipitation forecasts. Meteorological Applications 11: 141-154.
24. Coles S. 2001. An Introduction to Statistical Modeling of Extreme Values. Springer-Verlag: London, UK; 208.
25. Coles S, Heffernan J, Tawn J. 1999. Dependence measures for extreme value analyses. Extremes 2(4): 339-365.
26. Davis C, Brown B, Bullock R. 2006a. Object-based verification of precipitation forecasts. Part I: methodology and application to Mesoscale Rain Areas. Monthly Weather Review 134: 1772-1784.
27. Davis C, Brown B, Bullock R. 2006b. Object-based verification of precipitation forecasts. Part II: application to convective rain systems. Monthly Weather Review 134: 1785-1795.
28. De Elia R, Laprise R, Denis B. 2002. Forecasting skill limits of nested, limited-area models: a perfect model approach. Monthly Weather Review 130: 2006-2023.
29. Demargne J, Seo DJ, Wu L, Schaake J, Brown J. 2007. Verifying hydrologic forecasts in the US National Weather Service. In Oral Presentation at the Third International Workshop on Verification Methods, Reading, UK. PDF available from: http://www.ecmwf.int/newsevents/meetings/ workshops/2007/jwgv/index.html.
30. Denhard M, Trepte S, Göber M, Anger B. 2007. Verification of the SRNWP-PEPS. In Oral Presentation at the Third International Workshop on Verification Methods, Reading, UK. PDF available from: http:// www.eemwf/int/newsevents/meetings/workshops/2007/jwgv/index.html.
31. Denis B, Laprise R, Caya D. 2003. Sensitivity of a regional climate model to the resolution of the lateral boundary conditions. Climate Dynamics 20: 107-126.
32. Donaldson RJ, Dyer RM, Kraus MJ. 1975. An objective evaluator of techniques for predicting severe weather events. In Preprints, Ninth Conference on Severe Local Storms. American Meteorological Society: Norman, OK; 321-326.
33. Doswell CA, David-Jones R, Keller DL. 1990. On summary measures of skill in rare event forecasting based on contingency tables. Weather and Forecasting 5: 576-585.
34. Douglas RJ. 2000. Rearrangements of functions with applications to meteorology and ideal Fluid Flow. Internal Report 118, JCMM. Department of Meteorology, University of Reading, RG6 6BB, Reading, UK.
35. Du J, Mullen SL, Sanders F. 2000, Removal of distortion error from an ensemble forecast. Journal of Applied Meteorology 35: 1177-1188.
36. Ebert EE. 2008. Fuzzy verification of high resolution gridded forecasts: a review and proposed framework. Meteorological Applications 15: 53-66.
37. Ebert EE, McBride JL. 2000. Verification of precipitation in weather systems: Determination of systematic errors. Journal of Hydrology 239: 179-202.
38. Ebert EE, Wilson LJ, Brown BG, Nurmi P, Brooks HE, Bally J, Jaeneke M. 2004. Verification of nowcasts from the WWRP Sydney 2000 forecast demonstration project. Weather and Forecasting 19: 73-96.
39. Epstein ES. 1969. A scoring system for probability forecasts of ranked categories. Journal of Applied Meteorology 8: 985-987.
40. Ferro CAT. 2007a. Comparing probabilistic forecasting systems with the Brier score. Weather and Forecasting 22: 1076-1089.
41. Ferro CAT. 2007b. A probability model for verifying deterministic forecasts of extreme events. Weather and Forecasting 22: 1089-1100.
42. Ferro CAT, Richardson DS, Weigel AP. 2008. On the effect of ensemble size on the discrete and continuous ranked probability scores. Meteorological Applications 15: 19-24.
43. Finley JP. 1884. Tornado prediction. American Meteorological Journal 1: 85-88.
44. Germann U, Zawadzki I. 2002. Scale-dependence of the predictability of precipitation from continental radar images. Part I: description of the methodology. Monthly Weather Review 130: 2859-2873.
45. Germann U, Zawadzki I. 2004. Scale-dependence of the predictability of precipitation from continental radar images. Part II. Journal of Applied Meteorology 43: 74-89.
46. Gilbert GK. 1884. Finley's tornado predictions. American Meteorological Journal 1: 166-172.
47. Gilleland E, Lee T, Halley-Gotway J, Bullock R, Brown B. 2008. Computationally efficient spatial forecast verification using Baddeley's Δ image metric. Monthly Weather Review in press.
48. Gneiting T, Raftery AE. 2007. Strictly proper scoring rules, prediction and estimation. Journal of the American Statistics Association 102: 359-378.
49. Göber M, Wilson CA, Milton SF, Stephenson DB. 2004. Fairplay in the verification of operational quantitative precipitation forecasts. Journal of Hydrology 288: 225-236.
50. Good U. 1952. Rational decisions. Journal of the Royal Statistical Society Series B-Methodological 14: 107-114.
51. Grams JS, Gallus WA, Koch SE, Wharton LS, Loughe A, Ebert EE. 2006. The use of a modified Ebert-McBride technique to evaluate mesoscale model QPF as a function of convective system morphology during IHOP 2002, Weather and Forecasting 21: 288-306.
52. Grimit EP, Gneiting T, Berrocal VJ, Johnson NA. 2006. The continuous ranked probability score for circular variables and its application to mesoscale forecast ensemble verification. Quarterly Journal of the Royal Meteorological Society 132: 1-17.
53. Hamill TM. 2001. Interpretation of rank histograms for verifying ensemble forecasts. Monthly Weather Review 129: 550-560.
54. Hamill TM, Colucci SJ. 1997. Verification of Eta-RSM short range ensemble forecasts. Monthly Weather Review 125: 711-724.
55. Hamill TM, Juras J. 2006. Measuring forecast skill: is it real skill or is it the varying climatology? Quarterly Journal of the Royal Meteorological Society 132: 2905-2923.
56. Harris D, Foufoula-Georgiou E, Droegemeier KK, Levit JJ. 2001. Multiscale statistical properties of a high-resolution precipitation forecast. Journal of Hydrometeorology 2: 406-418.
57. Hersbach H. 2000. Decomposition of the continuous rank probability score for ensemble prediction systems. Weather and Forecasting 15: 559-570.
58. Hoffman RN, Liu Z, Louis J-F, Grassotti C. 1995. Distortion representation of forecast errors. Monthly Weather Review 123: 2758-2770.
59. Holland L, Fowler T, Brown B, Nance L. 2007. Designing a state-of-the-art verification system. In Oral Presentation at the Third International Workshop on Verification Methods, Reading, UK. PDF available from: http://www.ecmwf.int/newsevents/meetings/workshops/ 2007/jwgv/index.html.
60. Hopson T, Weiss J, Webster P. 2007. Verifying the relationship between ensemble forecast spread and skill. In Oral Presentation at the Third International Workshop on Verification Methods, Reading, UK. PDF available from: http://www.ecmwf.int/newsevents/meetings/workshops/2007/ jwgv/index.html.
61. Hsu WR, Murphy AH. 1986. The attributes diagram. A geometrical framework for assessing the quality of probability forecasts. International Journal of Forecasting 2: 285-293.
62. Intergovernmental Panel on Climate Change. 2001. 3rd Assessment Report, Available from http://www.ipcc.ch.
63. Jolliffe IT, Stephenson DB. 2003. Forecast Verification: A Practitioner's Guide in Atmospheric Science. John Wiley and Sons: Chichester, UK; 240.
64. Katz RW, Murphy AH. 1997. Economic Value of Weather and Climate Forecasts. Cambridge University Press: New York; 222.
65. Keenan T, Joe P, Wilson J, Collier C, Golding B, Burgess D, May P, Pierce C, Bally J, Crook A, Sills D, Berry L, Bell I, Fox N, Pielke R Jr, Ebert EE, Eilts M, O'Loughlin K, Webb R, Carbone R, Browning K, Roberts R, Mueller C. 2002, The sydney 2000 world weather research programme forecast demonstration project: overview and current status. Bulletin of the American Meteorological Society 84: 1041-1054.
66. Lalaurette F. 2003. Early detection of abnormal weather conditions using a probabilistic extreme forecast index. Quarterly Journal of the Royal Meteorological Society 129(594): 3037-3057.
67. Liljas E, Murphy A. 1994. Anders Ångstrp̈m and his early papers on probability forecasting and the use/value of weather forecasts. Bulletin of the American Meteorological Society 75: 1227-1236.
68. Mailier PJ, Jolliffe IT, Stephenson DB. 2006. Quality of weather forecasts: Review and Recommendations. Royal Meteorological Society Project Report, 89.
69. Marsigli C, Montani A, Paccagnella T. 2006. Verification of the COSMO-LEPS new suite in terms of precipitation distribution. COSMO Newsletter No. 6, Available at http://www.cosmo-model.org/public/ newsLetters.htm.
70. Marsigli C, Montani A, Paccagnella T. 2008. A spatial verification method applied to the evaluation of high-resolution ensemble forecasts. Meteorological Applications 15: 127-145.
71. Marzban C. 1998. Scalar measures of performance in rare-event situations. Weather and Forecasting 13: 753-763.
72. Marzban C, Sandgathe S. 2006. Cluster analysis for verification of precipitation fields. Weather and Forecasting 21: 824-838.
73. Mason I. 1982. A model for assessment of weather forecasts. Australian Meteorological Magazine 30: 291-303.
74. Mason S. 2007. Do high skill scores mean good forecasts? In Oral Presentation at the Third International Workshop on Verification Methods, Reading, UK, PDF available from: http://www.ecmwf.int /newsevents/meetings/workshops/2007/jwgv/index.html.
75. Mason S. 2008. Understanding forecast verification statistics. Meteorological Applications 15: 31-40.
76. Mittermaier M. 2007. Using time-lag ensemble techniques to assess behaviour of high-resolution precipitation forecasts. In Oral Presentation at the Third International Workshop on Verification Methods, Reading, UK, PDF available from: http://www.ecmwf.int /newsevents/meetings/workshops/2007/jwgv/index.html.
77. Murphy AH. 1973. A new vector partition of the probability score. Journal of Applied Meteorology 12: 595-600.
78. Murphy AH. 1991. Probabilities, odds, and forecasts of rare events. Weather and Forecasting 6: 302-307.
79. Murphy AH. 1993. What is a good forecast? An essay on the nature of goodness in weather forecasting. Weather and Forecasting 8: 281-293.
80. Murphy AH. 1996. The Finley affair: a signal event in the history of forecast verification. Weather and Forecasting 11: 3-20.
81. Murphy AH. 1998. The early history of probability forecasts: some extensions and clarifications. Weather and Forecasting 13: 5-15.
82. Murphy AH, Winkler RL. 1987. A general framework for forecast verification. Monthly Weather Review 115: 1330-1338.
83. Nachamkin JE. 2004. Mesoscale verification using meteorological composites. Monthly Weather Review 132: 941-955.
84. Nachamkin JE, Chen S, Schmidt J. 2005. Evaluation of heavy precipitation forecasts using composite-based methods: a distributions-oriented approach. Monthly Weather Review 133: 2163-2177.
85. Nehrkorn T, Hoffman RN, Grassotti C, Louis J-F. 2003. Feature calibration and alignment to represent model forecast errors: empirical regularization. Quarterly Journal of the Royal Meteorological Society 129: 195-218.
86. Nurmi P. 2003. Recommendations on the verification of local weather forecasts. ECMWF Technical Memorandum 430: 19.
87. Nurmi P, Näsman S, Zingerle C. 2007. Entity-based verification in the intercomparison of three NWP models during a heavy snowfall event. Geophysical Research Abstracts 9: 09247, Oral presentation at the EGU General Assembly, Vienna, Austria, PDF available from: http:// www.cosis.net/abstracts/EGU2007/09247/EGU2007-J-09247.pdf.
88. Pocernich M. 2007. Verification: forecast verification utilities. R package version 1.20. R Development Core Team (2007). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, ISBN 3-900051-07-0, URL http://www.r-project.org.
89. Rezacova D, Sokol Z. 2005. The use of radar data in the verification of a high resolution quantitative forecast of convective precipitation. In Extended Abstract, WWRP Symposium on Nowcasting and Very Short Range Weather Forecasting, Toulouse, 5-9 September 2005.
90. Richardson DS. 2000. Skill and economic value of the ECMWF ensemble prediction system. Quarterly Journal of the Royal Meteorological Society 126: 649-668.
91. Roberts NM, Lean HW. 2007. Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events. Monthly Weather Review 136: 78-97.
92. Rollins KS, Shaykewich J. 2003. Using willingness-to-pay to assess the economic value of weather forecasts for multiple commercial sectors. Meteorological Applications 10: 31-38.
93. Roulston MS, Smith LA. 2002. Evaluating probabilistic forecasts using information theory. Monthly Weather Review 130: 1653-1660.
94. Roulston MS, Smith LA. 2003. Combining dynamical and statistical ensembles. Tellus Series A-Dynamic Meteorology and Oceanography 55: 16-30.
95. Roulston MS, Smith LA. 2004. The boy who cried wolf revisited: the impact of false alarm intolerance on cost-loss scenarios. Weather and Forecasting 19(2): 391-397.
96. Roulston MS, Ellepola J, Smith LA. 2005. Forecasting wave height probabilities with numerical weather prediction models. Ocean Engineering 32(14-15): 1841-1863.
97. Roulston MS, Kaplan DT, Hardenberg J, Smith LA. 2003. Using medium-range weather forecasts to unprove the value of wind energy production. Renewable Energy 28(4): 585-602.
98. Roy G, Turcotte V. 2007. Verification des algorithms radars du GemLam 2.5. Internal Report, Severe Weather National Lab, Environment Canada.
99. Schaefer JT. 1990. The critical success index as an indicator of warning skill. Weather and Forecasting 5: 570-575.
100. Schubiger F, Ament F, Baehler T. 2007. Verification of the COSMO 7km model and new developments for the 2.2 Km model with a special emphasis on precipitation. In Oral Presentation at the Third International Workshop on Verification Methods, Reading, UK, PDF available from: http://www.ecmwf.int/newsevents/meetings/workshops/2007/jwgv/index.html.
101. Sing T, Sander O, Beerenwinkel N, Lengauer T. 2007. ROCR: Visualizing the performance of scoring classifiers. R package version 1.0-1, http:// rocr.bioinf.mpi-sb.mpg.de.
102. Smith LA. 1997. The maintenance of uncertainty. Proceedings International School of Physics "Enrico Fermi", Course CXXXIII. Societa Italiana di Fisica: Bologna; 177-246.
103. Smith LA. 2001. Disentangling uncertainty and error: on the predictability of nonlinear systems. In Nonlinear Dynamics and Statistics, Mees AI (ed.). Birkhauser: Boston, MA; 31-64.
104. Smith LA. 2003. Predictability past predictability present. In Predictability of Weather and Climate, Palmer T, Hagedorn R (eds). Cambridge University Press 2006: Cambridge, UK.
105. Smith LA, Hansen JA. 2004. Extending the limits of forecast verification with the minimum spanning tree. Monthly Weather Review 132: 1522-1528.
106. Smith LA, Roulston M, von Hardenberg J. 2001. End-to-end ensemble forecasting: towards evaluating the economic value of an ensemble prediction system. ECMWF Technical Memorandum 336: 29.
107. Stanski HR, Wilson LJ, Burrows WR. 1989. Survey of common verification methods in meteorology. World Weather Watch Technical Report No. 8, WMO/ TD No.358. WMO: Geneva, 114.
108. Stephenson DB. 2008. Definition, diagnosis, and origin of extreme weather and climate events. In Climate Extremes and Society, Diaz HF, Murnane RJ (eds). Cambridge University Press: New York; 348.
109. Stephenson DB, Casad B, Ferro CAT, Wilson C. 2008a. The extreme dependency score: a non-vanishing score for forecasts of rare events. Meteorological Applications 15: 41-51.
110. Stephenson DB, Coelho CAS, Jolliffe IT. 2008b. Two extra components in the Brier Score Decomposition. Weather and Forecasting in press.
111. Swets JA. 1986. Form of empirical ROCs in discrimination and diagnostic tasks: Implications for theory and measurement of performance. Psychological Bulletin 99: 181-198.
112. Swets SA, Pickett RM. 1982. Evaluation of Diagnostic Systems: Methods from Signal Detection Theory. Academic Press: New York.
113. Theis SE, Hense A, Damrath U. 2005. Probabilistic precipitation forecasts from a deterministic model: a pragmatic approach. Meteorological Applications 12: 257-268.
114. Toth Z, Talagrand O, Zhu Y. 2006. The attributes of forecast systems: a general framework for the evaluation and calibration of weather forecasts. In Predictability of Weather and Climate, Palmer T, Hagedorn R (eds). Cambridge University Press: Cambridge, UK; 584-595.
115. Tremblay A, Cober SG, Glazer A, Isaac G, Mailhot J. 1996. An intercomparison of mesoscale forecasts of aircraft icing using SSM/I retrievals. Weather and Forecasting 11: 66-77.
116. Turner BJ, Zawadzki I, Germann U. 2004. Predictability of precipitation from continental radar images. Part III: operational nowcasting implementation (MAPLE). Journal of Applied Meteorology 43: 231-248.
117. Tustison B, Foufoula-Georgiou E, Harris D. 2003. Scale-recursive estimation for multisensor Quantitative Precipitation Forecast verification: a preliminary assessment. Journal of Geophysical Research 108: 11775-11784.
118. Venugopal V, Basu S, Foufoula-Georgiu E. 2005. A new metric for comparing precipitation patterns with an application to ensemble forecasts. Journal of Geophysical Research 110: D08111.
119. Volkert H. 2005. The Mesoscale Alpine Programme (MAP) - a multifacetted success story. Preprints ICAM/MAP 2005, Zadar, Croatia, 23-27 May 2005, 226-230.
120. Weigel AP, Liniger MA, Appenzeller C. 2007. The discrete Brier and ranked probability skill scores. Monthly Weather Review 135: 118-124.
121. Weisheimer A, Smith LA, Judd K. 2004. A new view of forecast skill: bounding boxes from the DEMETER ensemble seasonal forecasts. Tellus 57(3): 265-279.
122. Wernli H, Paulat M, Frei C. 2006. The concept of a new error score SAL for the verification of high-resolution QPF. Presented at the 2nd International Symposium on Quantitative Precipitation Forecasting and Hydrology, Boulder, US, 4-8 June 2006.
123. Wilks DS. 2004. The Minimum Spanning Tree (MST) histogram as a verification tool for multidimensional ensemble forecasts. Monthly Weather Review 132: 1329-1340.
124. Wilks DS. 2006. Statistical Methods in Atmospheric Science, 2nd edn. Academic Press: Burlington, MA; 627.
125. Wilson LJ. 2000. Comments on "Probabilistic predictions of precipitation using the ECMWF ensemble prediction system". Weather and Forecasting 15: 361-364.
126. Wilson LJ, Gneiting T. 2007. Another look at proper scoring rules. In Oral Presentation at the Third International Workshop on Verification Methods, Reading, UK, PDF available from: http://www.ecmwf.int/ new-sevents/nieetings/workshops/2007/jwgv/index.html.
127. Wilson LJ, Burrows WR, Lanzinger A. 1999. A strategy for verification of weather element forecasts from an ensemble prediction system. Monthly Weather Review 127: 956-970.
128. WMO. 2000. Guidelines of performance assessment of public weather services. Document available at http://www.wmo.ch/web/aom) pwsp/ downloads/guidelines/TD-1023.pdf.
129. Yu X. 2005. Overview of Beijing 2008 Olympics WWRP Nowcast FDP and implementation plan. In Presented at the WWRP Symposium on Nowcasting and Very Short Range Forecasting, Toulouse, 5-9 September 2005.
130. Zepeda-Arce J, Foufoula-Georgiou E, Droegemeier KK. 2000. Spacetime rainfall organization and its role in validating quantitative precipitation forecasts. Journal Of Geophysical Research 105(D8): 10129-10146.
131. Zhu Y, Toth Z, Wobus R, Richardson D, Mylne K. 2002. The economic value of ensemble-based weather forecasts. Bulletin of the American Meteorological Society 83: 73-83.
132. Zingerle C, Nurmi P. 2008. Monitoring and verifying cloud forecasts originating from operational numerical models. Meteorological Applications submitted.