Forecast Verification

International Geophysics

Volumn 100, Issue , 2011, Pages 301-394

  Wilks, D S ^a  

^a NONE

Author keywords

Accuracy; Bias; Diagnostic verification; Forecast evaluation; Skill

Indexed keywords

EID: 79955741782     PISSN: 00746142     EISSN: None     Source Type: Book Series    
DOI: 10.1016/B978-0-12-385022-5.00008-7     Document Type: Book

References (192)

1. Accadia C., Mariani S., Casaioli M., Lavagnini A. Sensitivity of precipitation forecast skill scores to bilinear interpolation and a simple nearest-neighbor average method on high-resolution verification grids. Weather and Forecasting 2003, 18:918-932.
2. Gerrity, Jr.J.P. A note on Gandin and Murphy's equitable skill score. Monthly Weather Review 1992, 120:2709-2712.
3. Murphy A.H., Brown B.G., Chen Y.-S. Diagnostic verification of temperature forecasts. Weather and Forecasting 1989, 4:485-501.
4. Wilks D.S., Godfrey C.M. Diagnostic verification of the IRI new assessment forecasts, 1997-2000. Journal of Climate 2002, 15:1369-1377.
5. Harper K., Uccellini L.W., Kalnay E., Carey K., Morone L. 50th anniversary of operational numerical weather prediction. Bulletin of the American Meteorological Society 2007, 88:639-650.
6. Tracton M.S., Mo K., Chen W., Kalnay E., Kistler R., White G. Dynamical extended range forecasting (DERF) at the National Meteorological Center. Monthly Weather Review 1989, 117:1604-1635.
7. McAvaney B.J. Model Evaluation. Climate Change 2001: The Scientific Basis 2001, 471-523. 72 co-authors, Cambridge University Press. J.T. Houghton (Ed.).
8. Kalnay E., Kanamitsu M., Baker W.E. Global numerical weather prediction at the National Meteorological Center. Bulletin of the American Meteorological Society 1990, 71:1410-1428.
9. Gilleland E., Ahijevych D., Brown B.G., Casati B., Ebert E.E. Intercomparison of spatial forecast verification methods. Weather and Forecasting 2009, 24:1416-1430.
10. Gilleland E., Lindstrom J., Lindgren F. Analyzing the image warp forecast verification method on precipitation fields from the ICP. Weather and Forecasting 2010, 25:1249-1262.
11. Wilks D.S. The minimum spanning tree histogram as a verification tool for multidimensional ensemble forecasts. Monthly Weather Review 2004, 132:1329-1340.
12. Wilks D.S. A skill score based on economic value for probability forecasts. Meteorological Applications 2001, 8:209-219.
13. Wilks D.S. Sampling distributions of the Brier score and Brier skill score under serial dependence. Quarterly Journal of the Royal Meteorological Society 2011, 136:2109-2118.
14. Murphy A.H., Daan H. Forecast evaluation. Probability, Statistics, and Decision Making in the Atmospheric Sciences 1985, 379-437. Boulder, CO, Westview. A.H. Murphy, R.W. Katz (Eds.).
15. Goldsmith B.S. NWS verification of precipitation type and snow amount forecasts during the AFOS era 1990, 28. National Weather Service.
16. Pepe M.S. The Statistical Evaluation of Medical Tests for Classification and Prediction 2003, 302. Oxford University Press.
17. Jolliffe I.T., Stephenson D.B. Forecast Verification 2003, 240. Wiley.
18. Muller R.H. Verification of short-range weather forecasts (a survey of the literature). Bulletin of the American Meteorological Society 1944, 25. 47-53, 88-95.
19. Murphy A.H. The Finley affair: a signal event in the history of forecast verification. Weather and Forecasting 1996, 11:3-20.
20. Murphy A.H. Forecast verification. Economic Value of Weather and Climate Forecasts 1997, 19-74. Cambridge University Press. R.W. Katz, A.H. Murphy (Eds.).
21. Stanski H.R., Wilson L.J., Burrows William R. Survey of Common Verification Methods in Meteorology 1989, 114. World Weather Watch Technical Report No. 8, World Meteorological Organization TD No. 358.
22. Murphy A.H. What is a good forecast? An essay on the nature of goodness in weather forecasting. Weather and Forecasting 1993, 8:281-293.
23. Murphy A.H., Winkler R.L. A general framework for forecast verification. Monthly Weather Review 1987, 115:1330-1338.
24. Brier G.W., Allen R.A. Verification of weather forecasts. Compendium of Meteorology 1951, 841-848. American Meteorological Society. T.F. Malone (Ed.).
25. Murphy A.H., Winkler R.L. Diagnostic verification of probability forecasts. International Journal of Forecasting 1992, 7:435-455.
26. Katz R.W., Murphy A.H. Economic Value of Weather and Climate Forecasts 1997, 222. Cambridge University Press.
27. Ehrendorfer M., Murphy A.H. Comparative evaluation of weather forecasting systems: sufficiency, quality, and accuracy. Monthly Weather Review 1988, 116:1757-1770.
28. Krzysztofowicz R., Long D. Fusion of detection probabilities and comparison of multisensor systems. IEEE Transactions on Systems, Man, and Cybernetics 1990, 20:665-677.
29. Krzysztofowicz R., Long D. Beta probability models of probabilistic forecasts. International Journal of Forecasting 1991, 7:47-55.
30. Murphy A.H., Ye Q. Comparison of objective and subjective precipitation probability forecasts: the sufficiency relation. Monthly Weather Review 1990, 118:1783-1792.
31. Millner A. Getting the most out of ensemble forecasts: a valuation model based on user- forecast interactions. Journal of Applied Meteorology and Climatology 2008, 47:2561-2571.
32. Stewart T.R. Forecast value: descriptive decision studies. Economic Value of Weather and Climate Forecasts 1997, 147-181. Cambridge University Press. R.W. Katz, A.H. Murphy (Eds.).
33. Murphy A.H. Forecast verification: its complexity and dimensionality. Monthly Weather Review 1991, 119:1590-1601.
34. Gneiting T., Balabdaoui F., Raftery A.E. Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society 2007, B69:243-268.
35. Glahn H.R., Jorgensen D.L. Climatological aspects of the Brier p-score. Monthly Weather Review 1970, 98:136-141.
36. Winkler R.L. Evaluating probabilities: asymmetric scoring rules. Management Science 1994, 40:1395-1405.
37. Winkler R.L. Scoring rules and the evaluation of probabilities. Test 1996, 5:1-60.
38. Hamill T.M., Juras J. Measuring forecast skill: is it real skill or is it the varying climatology?. Quarterly Journal of the Royal Meteorological Society 2006, 132:2905-2923.
39. Juras J. Comments on "Probabilistic predictions of precipitation using the ECMWF ensemble prediction system. Weather and Forecasting 2000, 15:365-366.
40. Finley J.P. Tornado prediction. American Meteorological Journal 1884, 1:85-88.
41. Gilbert G.K. Finley's tornado predictions. American Meteorological Journal 1884, 1:166-172.
42. Doswell C.A., Davies-Jones R., Keller D.L. On summary measures of skill in rare event forecasting based on contingency tables. Weather and Forecasting 1990, 5:576-585.
43. Ebert E.E., McBride J.L. Verification of precipitation in weather systems: determination of systematic errors. Journal of Hydrology 2000, 239:179-202.
44. Schaefer J.T. The critical success index as an indicator of warning skill. Weather and Forecasting 1990, 5:570-575.
45. Stensrud D.J., Wandishin M.S. The correspondence ratio in forecast evaluation. Weather and Forecasting 2000, 15:593-602.
46. Stephenson D.B. Use of the "odds ratio" for diagnosing forecast skill. Weather and Forecasting 2000, 15:221-232.
47. Mason I.B. Binary events. Forecast Verification 2003, 37-76. Wiley. I.T. Jolliffe, D.B. Stephenson (Eds.).
48. Woodcock F. The evaluation of yes/no forecasts for scientific and administrative purposes. Monthly Weather Review 1976, 104:1209-1214.
49. Doolittle M.H. Association ratios. Bulletin of the Philosophical Society, Washington 1888, 7:122-127.
50. Heidke P. Berechnung des Erfolges und der Güte der Windstärkevorhersagen im Sturmwarnungsdienst. Geografika Annaler 1926, 8:301-349.
51. Peirce C.S. The numerical measure of the success of predictions. Science 1884, 4:453-454.
52. Hanssen A.W., Kuipers W.J.A. On the relationship between the frequency of rain and various meteorological parameters. Mededeelingen en Verhandelingen 1965, 81:2-15.
53. Flueck J.A. A study of some measures of forecast verification 1987, 69-73. American Meteorological Society.
54. Gringorten I.I. Verification to determine and measure forecasting skill. Journal of Applied Meteorology 1967, 6:742-747.
55. Clayton H.H. A method of verifying weather forecasts. Bulletin of the American Meteorological Society 1927, 8:144-146.
56. Clayton H.H. Rating weather forecasts. Bulletin of the American Meteorological Society 1934, 15:279-283.
57. Stephenson D.B., Casati B., Ferro C.A.T., Wilson C.A. The extreme dependency score: a non-vanishing measure for forecasts of rare events. Meteorological Applications 2008, 15:41-50.
58. Ghelli A., Primo C. On the use of the extreme dependency score to investigate the performance of an NWP model for rare events. Meteorological Applications 2009, 16:537-544.
59. Hogan R.J., O'Connor E.J., Illingworth A.J. Verification of cloud-fraction forecasts. Quarterly Journal of the Royal Meteorological Society 2009, 135:1494-1511.
60. Yule G.U. On the association of attributes in statistics. Philosophical Transactions of the Royal Society, London 1900, 194A:257-319.
61. Wandishin M.S., Brooks H.E. On the relationship between Clayton's skill score and expected value for forecasts of binary events. Meteorological Applications 2002, 9:455-459.
62. Brill K.F. A general analytic method for assessing sensitivity to bias of performance measures for dichotomous forecasts. Weather and Forecasting 2009, 24:307-318.
63. Roebber P.J. Visualizing multiple measures of forecast quality. Weather and Forecasting 2009, 24:601-608.
64. Gandin L.S., Murphy A.H. Equitable skill scores for categorical forecasts. Monthly Weather Review 1992, 120:361-370.
65. Hogan R.J., Ferro C.A.T., Jolliffe I.T., Stephenson D.B. Equitability revisited: why the "equitable threat score" is not equitable. Weather and Forecasting 2010, 25:710-726.
66. Mason I.B. On reducing probability forecasts to yes/no forecasts. Monthly Weather Review 1979, 107:207-211.
67. Livezey R.E. Categorical events 2003, I.T. Jolliffe and D.B. Stephenson, Forecast Verification. Wiley, In, 77-96.
68. Bradley A.A., Hashino T., Schwartz S.S. Distributions-oriented verification of probability forecasts for small data samples. Weather and Forecasting 2003, 18:903-917.
69. Katz R.W., Murphy A.H., Winkler R.L. Assessing the value of frost forecasts to orchardists: a dynamic decision-making approach. Journal of Applied Meteorology 1982, 21:518-531.
70. Murphy A.H., Wilks D.S. A case study in the use of statistical models in forecast verification: precipitation probability forecasts. Weather and Forecasting 1998, 13:795-810.
71. Hayashi Y. Statistical interpretations of ensemble-time mean predictability. Journal of the Meteorological Society of Japan 1986, 64:167-181.
72. Murphy A.H. Skill scores based on the mean square error and their relationships to the correlation coefficient. Monthly Weather Review 1988, 116:2417-2424.
73. Murphy A.H. Climatology, persistence, and their linear combination as standards of reference in skill scores. Weather and Forecasting 1992, 7:692-698.
74. Cooke W.E. Forecasts and verifications in western Australia. Monthly Weather Review 1906, 34:23-24.
75. Murphy A.H., Winkler R.L. Probability forecasting in meteorology. Journal of the American Statistical Association 1984, 79:489-500.
76. Toth Z., Talagrand O., Candille G., Zhu Y. Probability and Ensemble Forecasts. Forecast Verification 2003, 137-163. Wiley. I.T. Jolliffe, D.B. Stephenson (Eds.).
77. Murphy A.H. A new vector partition of the probability score. Journal of Applied Meteorology 1973, 12:595-600.
78. Kharin V.V., Zwiers F.W. Improved seasonal probability forecasts. Journal of Climate 2003, 16:1684-1701.
79. Stephenson D.B., Coelho C.A.S., Jolliffe I.T. Two extra components in the Brier score decomposition. Weather and Forecasting 2008, 23:752-757.
80. Brocker J. Some remarks on the reliability of categorical probability forecasts. Monthly Weather Review 2008, 136:4488-4502.
81. Brocker J., Smith L.A. Increasing the reliability of reliability diagrams. Weather and Forecasting 2007, 22:651-661.
82. Mason S.J., Goddard L., Graham N.E., Yulaleva E., Sun L., Arkin P.A. The IRI seasonal climate prediction system and the 1997/98 El Niño event. Bulletin of the American Meteorological Society 1999, 80:1853-1873.
83. Hsu W.-R., Murphy A.H. The attributes diagram: a geometrical framework for assessing the quality of probability forecasts. International Journal of Forecasting 1986, 2:285-293.
84. Murphy A.H., Ehrendorfer M. On the relationship between the accuracy and value of forecasts in the cost-loss ratio situation. Weather and Forecasting 1987, 2:243-251.
85. Good I.J. Rational decisions. Journal of the Royal Statistical Society 1952, 14A:107-114.
86. Roulston M.S., Smith L.A. Evaluating probabilistic forecasts using information theory. Monthly Weather Review 2002, 130:1653-1660.
87. Winkler R.L., Murphy A.H. "Good" probability assessors. Journal of Applied Meteorology 1968, 7:751-758.
88. Hagedorn R., Smith L.A. Communicating the value of probabilistic forecasts with weather roulette. Meteorological Applications 2009, 16:143-155.
89. Mason I.B. A model for assessment of weather forecasts. Australian Meteorological Magazine 1982, 30:291-303.
90. Swets J.A. The relative operating characteristic in psychology. Science 1973, 182:990-1000.
91. Swets J.A. ROC analysis applied to the evaluation of medical imaging techniques. Investigative Radiology 1979, 14:109-121.
92. Marzban C. The ROC curve and the area under it as performance measures. Weather and Forecasting 2004, 19:1106-1114.
93. Casati B., Wilson L.J., Stephenson D.B., Nurmi P., Ghelli A., Pocernich M., Damrath U., Ebert E.E., Brown B.G., Mason S. Forecast verification: current status and future directions. Meteorological Applications 2008, 15:3-18.
94. Wilson L.J. Comments on "probabilistic predictions of precipitation using the ECMWF ensemble prediction system.". Weather and Forecasting 2000, 15:361-364.
95. Glahn H.R. Discussion of "verification concepts in forecast verification: a practitioner's guide in atmospheric science". Weather and Forecasting 2004, 19:769-775.
96. Jolliffe I.T., Stephenson D.B. Comments on "discussion of verification concepts in forecast verification: a practitioner's guide in atmospheric science. Weather and Forecasting 2005, 20:796-800.
97. Kharin V.V., Zwiers F.W. On the ROC score of probability forecasts. Journal of Climate 2003, 16:4145-4150.
98. Jolliffe I.T. The impenetrable hedge: a note on propriety, equitability, and consistency. Meteorological Applications 2008, 15:25-29.
99. Gneiting T., Raftery A.E. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association 2007, 102:359-378.
100. Brocker J., Smith L.A. Scoring probabilistic forecasts: the importance of being proper. Weather and Forecasting 2007, 22:382-388.
101. Murphy A.H. Hedging and skill scores for probability forecasts. Journal of Applied Meteorology 1973, 12:215-223.
102. Epstein E.S. A scoring system for probability forecasts of ranked categories. Journal of Applied Meteorology 1969, 8:985-987.
103. Murphy A.H. A note on the ranked probability score. Journal of Applied Meteorology 1971, 10:155-156.
104. Staël von Holstein C.-A.S., Murphy A.H. The family of quadratic scoring rules. Monthly Weather Review 1978, 106:917-924.
105. O'Lenic E.A., Unger D.A., Halpert M.S., Pelman K.S. Developments in operational long-range climate prediction at CPC. Weather and Forecasting 2008, 23:496-515.
106. Daan H. Sensitivity of verification scores to the classification of the predictand. Monthly Weather Review 1985, 113:1384-1392.
107. Hersbach H. Decomposition of the continuous ranked probability score for ensemble prediction systems. Weather and Forecasting 2000, 15:559-570.
108. Mason S.J. Understanding forecast verification statistics. Meteorological Applications 2008, 15:31-40.
109. Pinson P., McSharry P., Madsen H. Reliability diagrams for non-parametric density forecasts of continuous variables: accounting for serial correlation. Quarterly Journal of the Royal Meteorological Society 2010, 136:77-90.
110. Wilks D.S. Resampling hypothesis tests for autocorrelated fields. Journal of Climate 1997, 10:65-82.
111. Déqué M. Continuous variables 2003, I.T. Jolliffe and D.B. Stephenson, In, eds., Forecast Verification. Wiley, 97-119.
112. Wilmott C.J., Ackleson S.G., Davis R.E., Feddema J.J., Klink K.M., Legates D.R., O'Donnell J., Rowe C.M. Statistics for the evaluation and comparison of models. Journal of Geophysical Research 1985, 90:8995-9005.
113. Roulston M.S., Smith L.A. Combining dynamical and statistical ensembles. Tellus 2003, 55A:16-30.
114. Bradley A.A., Schwartz S.S., Hashino T. Sampling uncertainty and confidence intervals for the Brier score and Brier Skill score. Weather and Forecasting 2008, 23:992-1006.
115. Mason S.J., Graham N.E. Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: statistical significance and interpretation. Quarterly Journal of the Royal Meteorological Society 2002, 128:2145-2166.
116. Thornes J.E., Stephenson D.B. How to judge the quality and value of weather forecast products. Meteorological Applications 2001, 8:307-314.
117. Agresti A. An Introduction to Categorical Data Analysis 1996, 290. Wiley.
118. Radok U. Chance behavior of skill scores. Monthly Weather Review 1988, 116:489-494.
119. Marzban C., Sandgathe S. Cluster analysis for object-oriented verification fields: a variation. Monthly Weather Review 2008, 136:1013-1025.
120. Agresti A., Coull B.A. Approximate is better than "exact" for interval estimation of binomial proportions. American Statistician 1998, 52:119-126.
121. Seaman R., Mason I., Woodcock F. Confidence intervals for some performance measures of yes-no forecasts. Australian Meteorological Magazine 1996, 45:49-53.
122. Candille G., Talagrand O. Impact of observational error on the validation of ensemble prediction systems. Quarterly Journal of the Royal Meteorological Society 2008, 134:959-971.
123. Saetra O., Hersbach H., Bidlot J.-R., Richardson D.S. Effects of observation errors on the statistics for ensemble spread and reliability. Monthly Weather Review 2004, 132:1487-1501.
124. Hamill T.M. Interpretation of rank histograms for verifying ensemble forecasts. Monthly Weather Review 2001, 129:550-560.
125. Anderson J.L. A method for producing and evaluating probabilistic forecasts from ensemble model integrations. Journal of Climate 1996, 9:1518-1530.
126. Ciach G.J., Krajewski W.F. On the estimation of radar rainfall error variance. Advances in Water Resources 1999, 22:585-595.
127. Bowler N.E. Explicitly accounting for observation error in categorical verification of forecasts. Monthly Weather Review 2006, 134:1600-1606.
128. Richardson D.S. Economic value and skill. Forecast Verification 2003, 165-187. Wiley. I.T. Jolliffe, D.B. Stephenson (Eds.).
129. Mylne K.R. Decision-making from probability forecasts based on forecast value. Meteorological Applications 2002, 9:307-315.
130. Semazzi F.H.M., Mera R.J. An extended procedure for implementing the relative operating characteristic graphical method. Journal of Applied Meteorology and Climatology 2006, 45:1215-1223.
131. Richardson D.S. Skill and economic value of the ECMWF ensemble prediction system. Quarterly Journal of the Royal Meteorological Society 2000, 126:649-667.
132. Richardson D.S. Measures of skill and value of ensemble predictions systems, their interrelationship and the effect of ensemble size. Quarterly Journal of the Royal Meteorological Society 2001, 127:2473-2489.
133. Murphy A.H. A note on the utility of probabilistic predictions and the probability score in the cost-loss ratio situation. Journal of Applied Meteorology 1966, 5:534-537.
134. Roebber P.J., Bosart L.F. The complex relationship between forecast skill and forecast value: a real-world analysis. Weather and Forecasting 1996, 11:544-559.
135. Liljas E., Murphy A.H. Anders Angstrom and his early papers on probability forecasting and the use/value of weather forecasts. Bulletin of the American Meteorological Society 1994, 75:1227-1236.
136. Thompson J.C. Economic gains from scientific advances and operational improvements in meteorological prediction. Journal of Applied Meteorology 1962, 1:13-17.
137. Murphy A.H. The value of climatological, categorical, and probabilistic forecasts in the cost-loss ratio situation. Monthly Weather Review 1977, 105:803-816.
138. Katz R.W., Murphy A.H. Forecast value: prototype decision-making models. Economic Value of Weather and Climate Forecasts 1997, 183-217. Cambridge University Press. R.W. Katz, A.H. Murphy (Eds.).
139. Wilks D.S. Forecast value: prescriptive decision studies. Economic Value of Weather and Climate Forecasts 1997, 109-145. Cambridge University Press. R.W. Katz, A.H. Murphy (Eds.).
140. Winkler R.L., Murphy A.H. Decision analysis. Probability, Statistics and Decision Making in the Atmospheric Sciences 1985, 493-524. Westview. A.H. Murphy, R.W. Katz (Eds.).
141. Johnson S.R., Holt M.T. The value of weather information. Economic Value of Weather and Climate Forecasts 1997, 75-107. Cambridge University Press. R.W. Katz, A.H. Murphy (Eds.).
142. Clemen R.T. Making Hard Decisions: an Introduction to Decision Analysis 1996, 664. Duxbury.
143. Stephenson D.B., Doblas-Reyes F.J. Statistical methods for interpreting Monte- Carlo ensemble forecasts. Tellus 2000, 52A:300-322.
144. Judd K., Smith L.A., Weisheimer A. How good is an ensemble at capturing truth? Using bounding boxes for forecast evaluation. Quarterly Journal of the Royal Meteorological Society 2007, 133:1309-1325.
145. Weisheimer A., Smith L.A., Judd K. A new view of seasonal forecast skill: bounding boxes from the DEMETER ensemble forecasts. Tellus 2005, 57A:265-279.
146. Gneiting T., Stanberry L.I., Grimit E.P., Held L., Johnson N.A. Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds. Test 2008, 17:211-235.
147. Smith L.A. Disentangling uncertainty and error: on the predictability of nonlinear systems. Nonlinear Dynamics and Statistics 2001, 31-64. Birkhauser. A.I. Mees (Ed.).
148. Gombos D., Hansen J.A., Du J., McQueen J. Theory and applications of the minimum spanning tree rank histogram. Monthly Weather Review 2007, 135:1490-1505.
149. Smith L.A., Hansen J.A. Extending the limits of ensemble forecast verification with the minimum spanning tree. Monthly Weather Review 2004, 132:1522-1528.
150. Gneiting T., Raftery A.E., Westveld A.H., Goldman T. Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Monthly Weather Review 2005, 133:1098-1118.
151. Elmore K.L. Alternatives to the chi-square test for evaluating rank histograms from ensemble forecasts. Weather and Forecasting 2005, 20:789-795.
152. Jolliffe I.T., Primo C. Evaluating rank histograms using decompositions of the chi- square test statistic. Monthly Weather Review 2008, 136:2133-2139.
153. Ferro C.A.T., Richardson D.S., Weigel A.P. On the effect of ensemble size on the discrete and continuous ranked probability scores. Meteorological Applications 2008, 15:19-24.
154. Hamill T.M., Colucci S.J. Evaluation of Eta-RSM ensemble probabilistic precipitation forecasts. Monthly Weather Review 1998, 126:711-724.
155. Talagrand O., Vautard R., Strauss B. Evaluation of probabilistic prediction systems. Proceedings, ECMWF Workshop on Predictability 1997, 1-25. ECMWF.
156. Legg T.P., Mylne K.R., Woodcock C. Use of medium-range ensembles at the Met Office I: PREVIN-a system for the production of probabilistic forecast information from the ECMWF EPS. Meteorological Applications 2002, 9:255-271.
157. Atger F. The skill of ensemble prediction systems. Monthly Weather Review 1999, 127:1941-1953.
158. Wilks D.S. Smoothing forecast ensembles with fitted probability distributions. Quarterly Journal of the Royal Meteorological Society 2002, 128:2821-2836.
159. Hannachi A., O'Neill A. Atmospheric multiple equilibria and non-Gaussian behavior in model simulations. Quarterly Journal of the Royal Meteorological Society 2001, 127:939-958.
160. Anderson J.L. The impact of dynamical constraints on the selection of initial conditions for ensemble predictions: low-order perfect model results. Monthly Weather Review 1997, 125:2969-2983.
161. Matheson J.E., Winkler R.L. Scoring rules for continuous probability distributions. Management Science 1976, 22:1087-1096.
162. Unger D.A. A method to estimate the continuous ranked probability score. Preprints, 9th Conference on Probability and Statistics in the Atmospheric Sciences 1985, 206-213. American Meteorological Society.
163. Baker D.G. Verification of fixed-width, credible interval temperature forecasts. Bulletin of the American Meteorological Society 1981, 62:616-619.
164. Gordon N.D. Comments on "verification of fixed-width credible interval temperature forecasts.". Bulletin of the American Meteorological Society 1982, 63:325.
165. Winkler R.L. A decision-theoretic approach to interval estimation. Journal of the American Statistical Association 1972, 67:187-191.
166. Winkler R.L., Murphy A.H. The use of probabilities in forecasts of maximum and minimum temperatures. Meteorological Magazine 1979, 108:317-329.
167. Gober M., Zsoter E., Richardson D.S. Could a perfect model ever satisfy a naive forecaster? On grid box mean versus point verification. Meteorological Applications 2008, 15:359-365.
168. Osborn T.J., Hulme M. Development of a relationship between station and grid-box rainday frequencies for climate model evaluation. Journal of Climate 1997, 10:1885-1908.
169. Tustison B., Harris D., Foufoula-Georgiou E. Scale issues in verification of precipitation forecasts. Journal of Geophysical Research 2001, D106:11775-11784.
170. Teweles S., Wobus H.B. Verification of prognostic charts. Bulletin of the American Meteorological Society 1954, 35:455-463.
171. Murphy A.H., Epstein E.S. Skill scores and correlation coefficients in model verification. Monthly Weather Review 1989, 117:572-581.
172. Gilleland E., Ahijevych D.A., Brown B.G., Ebert E.E. Verifying forecasts spatially. Bulletin of the American Meteorological Society 2010, 91:1365-1373.
173. Ahijevych D., Gilleland E., Brown B.G., Ebert E.E. Application of spatial verification methods to idealized and NWP-gridded precipitation forecasts. Weather and Forecasting 2009, 24:1485-1497.
174. Hoffman R.N., Liu Z., Louis J.-F., Grassotti C. Distortion representation of forecast errors. Monthly Weather Review 1995, 123:2758-2770.
175. Livezey R.E., Hoopingarner J.D., Huang J. Verification of official monthly mean 700-hPa height forecasts: an update. Weather and Forecasting 1995, 10:512-527.
176. Taylor K.E. Summarizing multiple aspects of model performance in a single diagram. Journal of Geophysical Research 2001, D106:7183-7192.
177. Namias J. The annual course of month-to-month persistence in climatic anomalies. Bulletin of the American Meteorological Society 1952, 33:279-285.
178. Miyakoda K., Hembree G.D., Strikler R.F., Shulman I. Cumulative results of extended forecast experiments. I: Model performance for winter cases. Monthly Weather Review 1972, 100:836-855.
179. Murphy A.H. The coefficients of correlation and determination as measures of performance in forecast verification. Weather and Forecasting 1995, 10:681-688.
180. Hollingsworth A., Arpe K., Tiedtke M., Capaldo M., Savijärvi H. The performance of a medium range forecast model in winter-impact of physical parameterizations. Monthly Weather Review 1980, 108:1736-1773.
181. Ebert E.E. Fuzzy verification of high-resolution gridded forecasts: a review and proposed framework. Meteorological Applications 2008, 15:51-64.
182. Briggs W.M., Levine R.A. Wavelets and field forecast verification. Monthly Weather Review 1997, 125:1329-1341.
183. Casati B. New developments of the intensity-scale technique within the spatial verification methods intercomparison project. Weather and Forecasting 2010, 25:113-143.
184. Denis B., CÔté J., Laprise R. Spectral decomposition of two-dimensional atmospheric fields on limited-area domains using the discrete cosine transform (DCT). Monthly Weather Review 2002, 130:1812-1829.
185. de Elia R., Laprise R., Denis B. Forecasting skill limits of nested, limited-area models: A perfect-model approach. Monthly Weather Review 2002, 130:2006-2023.
186. Davis C.A., Brown B.G., Bullock R., Halley-Gotway J. The method for object-based diagnostic evaluation (MODE) applied to numerical forecasts from the 2005 NSSL/SPC spring program. Weather and Forecasting 2009, 24:1252-1267.
187. Ebert E.E., Gallus W.A. Toward better understanding of the contiguous rain area (CRA) method for spatial forecast verification. Weather and Forecasting 2009, 24:1401-1415.
188. Wernli H., Hofmann C., Zimmer M. Spatial forecast verification methods intercomparison project: application of the SAL technique. Weather and Forecasting 2009, 24:1472-1484.
189. Brier G.W. Verification of forecasts expressed in terms of probabilities. Monthly Weather Review 1950, 78:1-3.
190. Hamill T.M. Hypothesis tests for evaluating numerical precipitation forecasts. Weather and Forecasting 1999, 14:155-167.
191. Joliffe I.T. Uncertainty and inference for verification measures. Weather and Forecasting 2007, 22:637-650.
192. Coles S., Heffernan J., Tawn J. Dependence measures for extreme value analyses. Extremes 1999, 2:339-365.