Inferential, nonparametric statistics to assess the quality of probabilistic forecast systems

Monthly Weather Review

Volumn 135, Issue 2, 2007, Pages 351-362

  Maia, Aline De H N ^a   Meinke, Holger ^b,d   Lennox, Sarah ^b   Stone, Roger ^b,c  

^a EMBRAPA CERRADOS   (Brazil)

^b DEPARTMENT OF AGRICULTURE AND FISHERIES   (Australia)

^c UNIVERSITY OF SOUTHERN QUEENSLAND   (Australia)

^d WAGENINGEN UNIVERSITY   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

CLIMATOLOGY; DECISION MAKING; ERROR ANALYSIS; MONTE CARLO METHODS; PROBABILITY; RAIN; RISK MANAGEMENT;

KOLMOGOROV-SMIRNOV METHODS; KRUSKAL-WALLIS METHODS; NOMINAL SIGNIFICANCE LEVELS; RANK PROBABILITY SKILL SCORE;

WEATHER FORECASTING;

CLIMATE PREDICTION; PROBABILITY; SOUTHERN OSCILLATION; SPATIAL ANALYSIS; STATISTICAL ANALYSIS;

EID: 33847348534     PISSN: 00270644     EISSN: None     Source Type: Journal    
DOI: 10.1175/MWR3291.1     Document Type: Article

References (49)

1. Allen, M. R., and D. A. Stainforth, 2002: Towards objective probabilistic climate forecasting. Nature, 419, 228.
2. Clarke, A. J., and L. Shu, 2000: Biennial winds in the far western equatorial Pacific phase-locking El Niño to the seasonal cycle. Geophys. Res. Lett., 27, 771-774.
3. Clarke, A. J., and S. van Gorder, 2003: Improving El Niño prediction using a space-time integration of Indo-Pacific winds and equatorial Pacific upper ocean heat content. Geophys. Res. Lett., 30, 1399, doi:10.1029/2002GL016673.
4. Conover, W. J., 1980: Practical Nonparametric Statistics. 3d ed. John Wiley and Sons, 584 pp.
5. De Jager, J. M., A. B. Potgieter, and W. J. van der Berg, 1998: Framework for forecasting the extent and severity of drought in maize in the Free State province of South Africa. Agric. Syst., 57, 351-365.
6. Donald, A., H. Meinke, B. Power, A. H. N. Maia, M. C. Wheeler, N. White, R. C. Stone, and J. Ribbe, 2006: Near-global impact of the Madden-Julian Oscillation on rainfall. Geophys. Res. Lett., 33, L09704, doi:10.1029/ 2005GL025155.
7. Easterling, D. R., and T. C. Peterson, 1995: A new method for detecting and adjusting for undocumented discontinuities in climatological time series. Int. J. Climatol., 15, 369-377.
8. Easterling, D. R., T. C. Peterson, and T. R. Karl, 1996: On the development and use of homogenized climate datasets. J. Climate, 9, 1429-1434.
9. Epstein, E. S., 1969: A scoring system for probability forecasts of ranked categories. J. Appl. Meteor., 8, 985-987.
10. Ferreyra, R. A., G. P. Podestá, C. D. Messina, D. Letson, J. Dardanelli, E. Guevara, and S. Meira, 2001: A linked-modeling framework to estimate maize production risk associated with ENSO-related climate variability in Argentina. Agric. For. Meteor., 107, 177-192.
11. Goddard, L., and M. Dilley, 2005: El Niño: Catastrophe or opportunity. J. Climate, 18, 651-665.
12. Goddard, L., A. G. Barnston, and S. J. Mason, 2003: Evaluation of the IRI's "Net Assessment" seasonal climate forecasts 1997-2001. Bull. Amer. Meteor. Soc., 84, 1761-1781.
13. Hammer, G. L., N. Nicholls, and C. Mitchell, Eds., 2000: Applications of Seasonal Climate Forecasting in Agriculture and Natural Ecosystems: The Australian Experience. Kluwer Academic, 469 pp.
14. Hartmann, H. C., T. C. Pagano, S. Sorooshian, and R. Bales, 2002: Confidence builders: Evaluating seasonal climate forecasts from a user perspective. Bull. Amer. Meteor. Soc., 83, 683-698.
15. Hill, H. S. J., J. Park, J. W. Mjelde, W. Rosenthal, H. A. Love, and S. W. Fuller, 2000: Comparing the value of southern oscillation index-based climate forecast methods for Canadian and U.S. wheat producers. Agric. For. Meteor., 100, 261-272.
16. Hill, H. S. J., J. W. Mjelde, H. A. Love, D. J. Rubas, S. W. Fuller, W. Rosenthal, and G. Hammer, 2004: Implications of seasonal climate forecasts on world wheat trade: A stochastic, dynamic analysis. Can. J. Agric. Econ., 52, 289-312.
17. Hoffman, F. O., and S. Kaplan, 1999: Beyond the domain of direct observation: How to specify a probability distribution that represents the "State of Knowledge" about uncertain inputs. Risk Anal., 19, 131-134.
18. Jeffrey, S. J., J. O. Carter, K. M. Moodie, and A. R. Beswick, 2001: Using spatial interpolation to construct a comprehensive archive of Australian climate data. Environ. Model. Software, 16, 309-330.
19. Jolliffe, I. T., 2004: Estimation of uncertainty in verification measures. Proc. Int. Verification Methods Workshop, Montreal, QC, Canada, World Meteorological Organization/World Climate Research Programme. [Available online at http://www. bom.gov.au/bmrc/wefor/staff/ eee/verif/Workshop2004/presentations/1.1_Jolliffe.pdf.]
20. Jørgensen, B., 1987: Exponential dispersion models. J. Roy. Stat. Soc., 49B, 127-162.
21. Maia, A. H. N., H. Meinke, and S. Lennox, 2004: Assessment of probabilistic forecast "skill" using p-values. Proc. Fourth Int. Crop Science Congress, Brisbane, Australia, The Regional Institute Ltd., CD-ROM, 2.6. [Available online at http://www.cropscience.org.au.]
22. Manly, B. F. J., 1997: Randomization, Bootstrap and Monte Carlo Methods in Biology. Chapman and Hall, 399 pp.
23. Mason, S. J., 2004: On using "climatology" as a reference strategy in the Brier and ranked probability skill scores. Mon. Wea. Rev., 132, 1891-1895.
24. Meinke, H., and Z. Hochman, 2000: Using seasonal climate forecasts to manage dryland crops in northern Australia. Applications of Seasonal Climate Forecasting in Agricultural and Natural Ecosystems: The Australian Experience, G. L. Hammer, N. Nicholls, and C. Mitchell, Eds., Kluwer Academic, 149-165.
25. Meinke, H., and R. C. Stone, 2005: Seasonal and inter-annual climate forecasting: The new tool for increasing preparedness to climate variability and change in agricultural planning and operations. Climatic Change, 70, 221-253.
26. Meinke, H., L. Donald, P. deVoil, B. Power, W. Baethgen, M. Howden, R. Allan, and B. Bates, 2004: How predictable is the climate and how can we use it in managing cropping risks? Proc. Fourth Int. Crop Science Congress, Brisbane, Australia, The Regional Institute Ltd., CD-ROM, 2.7. [Available online at http://www.cropscience.org.au.]
27. Meinke, H., R. Nelson, P. Kokic, R. Stone, R. Selvaraju, and W. Baethgen, 2006: Actionable climate knowledge - From analysis to synthesis. Climate Res., 33, 101-110.
28. Messina, C. D., J. W. Hansen, and A. J. Hall, 1999: Land allocation conditioned on ENSO phases in the Pampas of Argentina. Agric. Syst., 60, 197-212.
29. Murphy, A. H., 1993: What is a good forecast? An essay on the nature of goodness in weather forecasting. Wea. Forecasting, 8, 281-293.
30. Nelson, R. A., D. P. Holzworth, G. L. Hammer, and P. T. Hayman, 2002: Infusing the use of seasonal climate forecasting into crop management practice in north east Australia using discussion support software. Agric. Syst., 74, 393-414.
31. New, M. G., M. Hulme, and P. D. Jones, 2000: Representing twentieth-century space-time climate variability. Part II: Development of 1901-96 monthly grids of terrestrial surface climate. J. Climate, 13, 2217-2238.
32. Nicholls, N., 2001: The insignificance of significance testing. Bull. Amer. Meteor. Soc., 82, 981-986.
33. Peterson, T. C., and D. R. Easterling, 1994: Creation of homogeneous composite climatological reference series. Int. J. Climatol., 14, 671-679.
34. Podestá, G., and Coauthors, 2002: Use of ENSO-related climate information in agricultural decision making in Argentina: A pilot experience. Agric. Syst., 74, 371-392.
35. Potgieter, A. B., Y. Everingham, and G. L. Hammer, 2003: Measuring quality of a commodity forecasting from a system that incorporates seasonal climate forecasts. Int. J. Remote Sens., 23, 1195-1210.
36. Potts, J. M., C. K. Folland, I. T. Jolliffe, and D. Sexton, 1996: Revised "LEPS" scores for assessing climate model simulations and long-range forecasts. J. Climate, 9, 34-53.
37. Rosnell, R. L., and R. Rosenthal, 1989: Statistical procedures and the justification of knowledge and psychological science. Amer. Psychol., 44, 1276-1284.
38. Selvaraju, R., H. Meinke, and J. Hansen, 2004: Climate information contributes to better water management of irrigated cropping systems in southern India. Proc. Fourth Int. Crop Science Congress, Brisbane, Australia, The Regional Institute Ltd., CD-ROM, 2.6. [Available online at http://www.cropscience.org.au.]
39. Singels, A., and A. B. Potgieter, 1997: A technique to evaluate ENSO-based maize production strategies. South Afr. J. Plant Soil, 14, 93-97.
40. Sivakumar, M. V. K., R. Gommes, and W. Baier, 2000: Agrometeorology and sustainable agriculture. Agric. For. Meteor., 103, 11-26.
41. Stokes, M. E., C. S. David, and G. G. Koch, 2000: Categorical Data Analysis using the SAS® System. 2d ed. SAS Institute, 626 pp.
42. Stone, R. C., 1992: SOI phase relationship with rainfall probabilities over Australia. Ph.D. thesis, The University of Queensland, 410 pp.
43. Stone, R. C., G. L. Hammer, and T. Marcussen, 1996: Prediction of global rainfall probabilities using phases of the Southern Oscillation index. Nature, 384, 252-255.
44. Stone, R. C., I. Smith, and P. McIntosh, 2000: Statistical methods for deriving seasonal climate forecasts from GCM's. Applications of Seasonal Climate Forecasting in Agricultural and Natural Ecosystems, G. L. Hammer, N. Nichols, and C. Mitchell, Eds., Kluwer Academic, 135-147.
45. Tweedie, M. C. K., 1984: An index which distinguishes between some important exponential families. Statistics: Applications and New Directions, J. K. Ghosh and J. Roy, Eds., Indian Statistical Institute, 579-604.
46. Von Storch, H., and F. W. Zwiers, 1999: Statistical Analysis in Climate Research. Cambridge University Press, 484 pp.
47. Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences. Academic Press, 467 pp.
48. WMO, cited 2005: Proceedings of the meeting of the CCI OPAG3 expert team on verification. [Available, online at http://www. wmo.ch/web/wcp/ clips2001/html/Report_ETVerification_080205_2.pdf.]
49. Zhang, H., and T. Casey, 2000: Verification of categorical forecasts. Wea. Forecasting, 15, 80-89.