Extremal dependence indices: Improved Verification measures for deterministic forecasts of rare binary events

Weather and Forecasting

Volumn 26, Issue 5, 2011, Pages 699-713

  Ferro, Christopher A T ^a   Stephenson, David B ^a  

^a UNIVERSITY OF EXETER   (United Kingdom)

Author keywords

Extreme events; Forecast verification

Indexed keywords

BASE RATE; DETERMINISTIC FORECASTS; EXTREMAL; EXTREME EVENTS; PERFORMANCE MEASURE; RARE EVENT; RELATIVE OPERATING CHARACTERISTICS;

FENCES;

FORECASTING;

EXTREME EVENT; MEASUREMENT METHOD; WEATHER FORECASTING;

EID: 81855161303     PISSN: 08828156     EISSN: None     Source Type: Journal    
DOI: 10.1175/WAF-D-10-05030.1     Document Type: Article

References (33)

1. Brill, K. F., 2009: A general analytic method for assessing sensitivity to bias of performance measures for dichotomous forecasts. Wea. Forecasting, 24, 307-318.
2. Coles, S., J. Heffernan, and J. Tawn, 1999: Dependence measures for extreme value analyses. Extremes, 2, 339-365.
3. Davies, T., M. J. P. Cullen, A. J. Malcolm, M. H. Mawson, A. Staniforth, A. A. White, and N. Wood, 2005: A new dynamical core for theMet Office's global and regional modelling of the atmosphere. Quart. J. Roy.Meteor. Soc., 131, 1759-1782.
4. Davison, A. C., and D. V. Hinkley, 1997: Bootstrap Methods and Their Application. Cambridge University Press, 582 pp.
5. Ferro, C. A. T., 2007: A probability model for verifying deterministic forecasts of extreme events. Wea. Forecasting, 22, 1089-1100.
6. Gandin, L. S., and A. H. Murphy, 1992: Equitable skill scores for categorical forecasts. Mon. Wea. Rev., 120, 361-370.
7. Ghelli, A., and C. Primo, 2009: On the use of the extreme dependency score to investigate the performance of an NWP model for rare events. Meteor. Appl., 16, 537-544.
8. Göber, M., C. A. Wilson, S. F. Milton, and D. B. Stephenson, 2004: Fair play in the verification of operational quantitative precipitation forecasts. J. Hydrol., 288, 225-236.
9. Hamill, T. M., and J. Juras, 2006: Measuring forecast skill: Is it real skill or is it the varying climatology? Quart. J. Roy. Meteor. Soc., 132, 2905-2923.
10. Harvey, L. O., Jr., K. R. Hammond, C. M. Lusk, and E. F. Mross, 1992: The application of signal detection theory to weather forecasting behavior. Mon. Wea. Rev., 120, 863-883.
11. Hogan, R. J., E. J. O'Connor, and A. J. Illingworth, 2009: Verification of cloud-fraction forecasts. Quart. J. Roy. Meteor. Soc., 135, 1494-1511.
12. Hogan, R. J., C. A. T. Ferro, I. T. Jolliffe, and D. B. Stephenson, 2010: Equitability revisited: Why the "equitable threat score" is not equitable. Wea. Forecasting, 25, 710-726.
13. Hubálek, Z., 1982: Coefficients of association and similarity, based on binary (presence-absence) data: An evaluation. Biol. Rev. Cambridge Philos. Soc., 57, 669-689.
14. Jolliffe, I. T., 2008: The impenetrable hedge: A note on proprety, equitability and consistency. Meteor. Appl., 15, 25-29.
15. Ledford, A.W., and J. A. Tawn, 1996: Statistics for near independence in multivariate extreme values. Biometrika, 83, 169-187.
16. Marzban, C., 1998: Scalar measures of performance in rare-event situations. Wea. Forecasting, 13, 753-763.
17. Marzban, C., 2004: The ROC curve and the area under it as performance measures. Wea. Forecasting, 19, 1106-1144.
18. Mason, I. B., 1982: A model for assessment of weather forecasts. Aust. Meteor. Mag., 30, 291-303.
19. Mason, I. B, 2003: Binary events. Forecast Verification: A Practitioner's Guide inAtmospheric Science, I. T. Jolliffe andD. B. Stephenson, Eds., John Wiley and Sons, 37-76.
20. Mason, S. J., and N. E. Graham, 1999: Conditional probabilities, relative operating characteristics, and relative operating levels. Wea. Forecasting, 14, 713-725.
21. Mason, S. J., and N. E. Graham, 2002: Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation. Quart. J. Roy. Meteor. Soc., 128, 2145-2166.
22. Murphy, A. H., and H. Daan, 1985: Forecast evaluation. Probability, Statistics and Decision Making in the Atmospheric Sciences, A. H. Murphy and R. W. Katz, Eds., Westview Press, 379-437.
23. Primo, C., and A. Ghelli, 2009: The affect of the base rate on the extreme dependency score. Meteor. Appl., 16, 533-535.
24. Ramos, A., and A. Ledford, 2009: A new class of models for bivariate joint tails. J. Roy. Stat. Soc., 71B, 219-241.
25. Segers, J., and B. Vandewalle, 2004: Statistics of multivariate extremes. Statistics of Extremes: Theory and Applications, J. Beirlant et al., Eds., John Wiley and Sons, 297-368.
26. Stephenson, D. B., 2000: Use of the "odds ratio" for diagnosing forecast skill. Wea. Forecasting, 15, 221-232.
27. Stephenson, D. B, B. Casati, C. A. T. Ferro, and C. A. Wilson, 2008: The extreme dependency score: A non-vanishing measure for forecasts of rare events. Meteor. Appl., 15, 41-50.
28. Swets, J. A., 1986: Form of empirical ROCs in discrimination and diagnostic tasks: Implications for theory and measurement of performance. Psychol. Bull., 99, 181-198.
29. Swets, J. A, 1988: Measuring the accuracy of diagnostic systems. Science, 240, 1285-1293.
30. Swets, J. A, 1996: Signal Detection Theory and ROC Analysis in Psychology and Diagnostics: Collected Papers. Lawrence Erlbaum, 328 pp.
31. Woodcock, F., 1976: The evaluation of yes/no forecasts for scientific and administrative purposes. Mon. Wea. Rev., 104, 1209-1214.
32. Yule, G. U., 1900: On the association of attributes in statistics: With illustrations from the material of the Childhood Society, &c. Philos. Trans. Roy. Soc. London, 194A, 257-319.
33. Yule, G. U., 1912: On the methods of measuring association between two attributes. J. Roy. Stat. Soc., 75, 579-652.