Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation

Quarterly Journal of the Royal Meteorological Society

Volumn 128, Issue 584 PART B, 2002, Pages 2145-2166

  Mason, S J ^a   Graham, N E ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Forecast verification; Mann Whitney U test; Probabilistic forecasts; Signal detection theory; Student's t test

Indexed keywords

APPROXIMATION THEORY; PROBABILITY DISTRIBUTIONS; STATISTICAL METHODS; WEATHER FORECASTING;

SIGNAL DETECTION THEORY;

CLIMATOLOGY;

QUALITY CONTROL; STATISTICAL ANALYSIS; WEATHER FORECASTING;

EID: 0036630603     PISSN: 00359009     EISSN: None     Source Type: Journal    
DOI: 10.1256/003590002320603584     Document Type: Article

References (84)

1. The area above the ordinal dominance graph and the area below the receiver operating characteristics graph
2. Advances in statistical methodology for diagnostic medicine in the 1980s
3. An approximation to the Wilcoxon-Mann-Whitney distribution
4. Accuracy and potential economic value of categorical and probabilistic forecasts of discrete events
5. Impact of ensemble size on ensemble prediction
6. The impact of model resolution and ensemble size on the performance of an ensemble prediction system
7. Probabilistic predictions of precipitation using the ECMWF ensemble prediction system
8. Advances in statistical methodology for the evaluation of diagnostic and laboratory tests
9. Signal detectability: The use of ROC curves and their analyses
10. An evaluation of methods for estimating the area under the receiver operating characteristics (ROC) curve
11. Rank tests for one sample, two samples, and k samples without the assumption of a continuous distribution function
12. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach
13. Algorithm AS 62. A generator for the sampling distribution of the Mann-Whitney U statistic
14. A method of sampling inspection
15. The paradigms of quality: Evolution and revolution in the history of the discipline
16. Maximum likelihood estimation of parameters of signal-detection theory and determination of confidence intervals-rating-method data
17. Targeted ensemble prediction for northern Europe and parts of the North Atlantic Ocean
18. A generalization of the one-sided two-sample Kolmogorov-Smirnov statistic for evaluating diagnostic tests
19. Experimental predictions of wet season precipitation in northeastern Brazil
20. An assessment of seasonal predictability using atmospheric general circulation models
21. Some aspects of ROC curve-fitting: Normal and logistic models
22. The robustness of the 'binormal' assumptions used in fitting ROC curves
23. The meaning and use of the area under the receiver operating characteristic (ROC) curve
24. A method of comparing the areas under receiver operating characteristic curves from the same cases
25. An efficient, minimal-storage procedure for calculating the Mann-Whitney U, generalized U and similar distributions
26. The application of signal detection theory to weather forecasting behavior
27. The area under the ROC curve and its competitors
28. A class of statistics with asymptotically normal distribution
29. Nonparametric and semiparametric estimation of the receiver operating characteristic curve
30. The Wilcoxon, ties, and the computer
31. Ranking the effect of different features on the classification of discrete valued data
32. Comparing the areas under more than two independent ROC curves
33. On a test whether one or two random variables is stochastically larger than the other
34. Statistical analysis with receiver operating characteristic curves
35. On reducing probability forecasts to yes/no forecasts
36. A model for assessment of weather forecasts
37. Conditional probabilities, relative operating characteristics, and relative operating levels
38. The IRI seasonal climate prediction system and the 1997/98 El Niño event
39. Basic principles of ROC analysis
40. Statistical significance tests for binormal ROC curves
41. A new approach for testing the significance of differences between ROC curves measured from correlated data
42. Quantitative precipitation forecasts over the United States by the ECMWF ensemble prediction system
43. A general framework for forecast verification
44. A comparison of current measures of the accuracy of feeling-of-knowing predictions
45. ROC curves and measures of discrimination accuracy: A reply to Swets
46. Some procedures for calculating the distributions of nonparametric test statistics
47. On the problem of the most efficient tests of statistical hypotheses
48. Algorithm AS 55. The generalized Mann-Whitney U-statistic
49. A probability and decision-model analysis of PROVOST seasonal multi-model ensemble integrations
50. The theory of signal detectability
51. Skill and relative economic value of the ECMWF ensemble prediction system
52. What is the best index of detectability?
53. Statistical comparison of ROC curves from multiple readers
54. The relative operating characteristics in psychology
55. ROC analysis applied to the evaluation of medical imaging techniques
56. Indices of discrimination of diagnostic accuracy: Their ROCs and implied models
57. Measuring the accuracy of diagnostic systems
58. Deferred decision in human signal detection: A preliminary experiment
59. Decision processes in perception
60. Better decisions through science
61. A dynamical approach to seasonal prediction of Atlantic tropical cyclone activity
62. Evaluation of a short-range multimodel ensemble system
63. A skill score based on economic value for probability forecasts
64. Comments on 'probabilistic predictions of precipitation using the ECMWF ensemble prediction system'
65. A comparison of three radar-based severe-storm-detection algorithms on Colorado high plains thunderstorms
66. Verification of categorical probability forecasts
67. The economic value of ensemble-based forecasts