Use of the 'odds ratio' for diagnosing forecast skill

Weather and Forecasting

Volumn 15, Issue 2, 2000, Pages 221-232

  Stephenson, David B ^a,b  

^a INSTITUT DE MATHÉMATIQUES DE TOULOUSE   (France)

^b UNIVERSITY OF READING   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

MATHEMATICAL MODELS; NUMERICAL METHODS; PROBABILITY DISTRIBUTIONS; RELIABILITY; SAMPLING; STATISTICAL TESTS;

FORECAST SKILL; ODDS RATIO; PEIRCE SKILL; SKILL SCORE;

WEATHER FORECASTING;

MODELING; WEATHER FORECASTING;

EID: 0033924470     PISSN: 08828156     EISSN: None     Source Type: Journal    
DOI: 10.1175/1520-0434(2000)015<0221:UOTORF>2.0.CO;2     Document Type: Article

References (41)

1. Agresti, A., 1996: An Introduction to Categorical Data Analysis. John Wiley and Sons, 290 pp.
2. _, and B. A. Coull, 1998: Approximation is better than "exact" for interval estimation of binomial proportions. Amer. Stat., 52, 1-7.
3. Bishop, Y. M. M., S. E. Fienberg, and P. W. Holland, 1975: Discrete Multivariate Analysis: Theory and Practice, The MIT Press, 557 pp.
4. Doolittle, M. H., 1885: The verification of predictions. Amer. Meteor. J., 2, 327-329.
5. _, 1888: Association ratios, Bull. Philos. Soc. Wash., 10, 83-96.
6. Finley, J. P., 1884: Tornado predictions. Amer. Meteor. J., 1, 85-88.
7. Fisher, R. A., 1990: Statistical Methods, Experimental Design, and Scientific Inference, Oxford University Press, 155 pp.
8. Flueck, J. A., 1987: A study of some measures of forecast verification. Preprints. 10th Conf. on Probability and Statistics in Atmospheric Sciences, Edmonton, AB, Canada, Amer. Meteor. Soc., 69-73.
9. Gandin, L. S., and A. H. Murphy, 1992: Equitable skill scores for categorical forecasts. Mon. Wea. Rev., 120, 2709-2712.
10. Gilbert, G. F., 1884: Finley's tornado predictions. Amer. Meteor. J., 1, 166-172.
11. Goodman, L. A., and W. H. Kruskal, 1979: Measures of Association for Cross Classifications. Springer-Verlag, 146 pp.
12. Green, D. M., and J. A. Swets, 1966: Signal Detection Theory and Psychophysics. Wiley, 155 pp. (Reprinted by Robert E. Krieger Publishing Co., 1974.)
13. Haussen, A. W., and W. J. A. Kuipers, 1965: On the relationship between the frequency of rain and various meteorological parameters. Meded. Verh., 81, 2-15.
14. Harvey, L. O., Jr., K. R. Hammond, C. M. Lusk, and E. F. Moss, 1992: The application of signal detection theory to weather forecasting behavior. Mon. Wea. Rev., 120, 863-883.
15. Heidke, P., 1926: Berechnung des Erfolges und der Güte der Windstärkevorhersagen im Sturmwarnungdienst (Calculation of the success and goodness of strong wind forecasts in the storm warning service). Geogr. Ann. Stockholm, 8, 301-349.
16. Macmillan, N. A., and C. D. Creelman, 1991: Detection Theory: A User's Guide, Cambridge University Press, 155 pp.
17. Mason, I. B., 1980: Decision-theoretic evaluation of probabilistic predictions. Proc. WMO Symp. on Probabilistic and Statistical Methods in Weather Forecasting, Nice, France, WMO, 219-228.
18. _, 1982: A model for the assessment of weather forecasts. Aust. Meteor. Mag., 30, 291-303.
19. _, 1989: Dependence of the critical success index on sample climate and threshold probability. Aust. Meteor. Mag., 37, 75-81.
20. _, 1997: The weather forecast as a statistical decision: An outline of signal detection theory and ROC analysis in assessment of forecast quality. Proc. Forecast and Warning Verification Workshop, Melbourne, Australia, Australian Bureau of Meteorology, 1-10.
21. Matthews, R. A. J., 1996: Base-rate errors and rain forecasts. Nature, 382, 766.
22. Murphy, A. H., 1991: Forecast verification: Its complexity and dimensionality. Mon. Wea. Rev., 119, 1590-1601.
23. _, 1993: What is a good forecast? An essay on the nature of goodness in weather forecasting. Wea. Forecasting, 8, 281-293.
24. _, 1996: The Finley affair: A signal event in the history of forecast verification. Wea. Forecasting, 11, 3-20.
25. _, and H. Daan, 1985: Forecast evaluation. Probability, Statistics, and Decision Making in the Atmospheric Sciences, A. H. Murphy and R. W. Katz, Eds., Westview Press, 379-437.
26. _, and R. L. Winkler, 1987: A general framework for forecast verification. Mon. Wea. Rev., 115, 1330-1338.
27. Peirce, C. S., 1884: The numerical measure of the success of predictions. Science, 4, 453-454.
28. Pickup, M. N., 1982: A consideration of the effect of 500mb cyclonicity on the success of some thunderstorm forecasting techniques. Meteor. Mag., 111, 87-97.
29. Potts, J. M., C. K. Folland, I. T. Jolliffe, and D. Sexton, 1996: Revised "LEPS" scores for assessing climate model simulations and long-range forecasts. J. Climate, 9, 34-53.
30. Schaefer, J. T., 1990: The critical success index as an indicator of warning skill. Wea. Forecasting, 5, 570-575.
31. Seaman, R., I. Mason, and F. Woodcock, 1996: Confidence intervals for some performance measures of yes/no forecasts. Aust. Meteor. Mag., 45, 49-53.
32. Stanski, H. R., L. J. Wilson, and W. R. Burrows, 1989: Survey of common verification methods in meteorology, WMO/TD-No. 358, World Meteorological Organization, Geneva, Switzerland, 114 pp.
33. Stephenson, D. B., 1997: Correlation of spatial climate/weather maps and the advantages of using the Mahalanobis metric in predictions. Tellus, 49A, 513-527.
34. Swets, J. A., 1973: The relative operating characteristic in psychology. Science, 182, 990-1000.
35. _, 1986: Indices of discrimination or diagnostic accuracy: Their ROCs and implied models. Pxychol. Bull., 99, 100-117.
36. _, 1988: Measuring the accuracy of diagnostic systems. Science, 240, 1285-1293.
37. _, and R. M. Pickett, 1982: Evaluation of Diagnostic Systems: Methods from Signal Detection Theory. Academic Press, 253 pp.
38. Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences. Academic Press, 465 pp.
39. Woodcock, F., 1976: The evaluation of yes/no forecasts for scientific and administrative purposes. Mon. Wea. Rev., 104, 1209-1214.
40. Yates, F., 1984: Tests of significance for 2 × 2 contingency tables. J. Roy. Stat. Soc., 147A, 426-463.
41. Yule, G. U., 1900: On the association of attributes in statistics. Philos. Trans. Roy. Soc. London, 194A, 257-319.