"The stippling shows statistically significant grid points": How research results are routinely overstated and overinterpreted, and what to do about it

Bulletin of the American Meteorological Society

Volumn 97, Issue 12, 2016, Pages 2263-2273

  Wilks, D S ^a  

^a Institute for the Study of the Continents   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EARTH ATMOSPHERE; METEOROLOGY;

FALSE DISCOVERY RATE; GRID POINTS; MULTIPLE HYPOTHESIS TESTS; NOCV1; RESEARCH RESULTS;

STATISTICAL TESTS;

EID: 84988965822     PISSN: 00030007     EISSN: None     Source Type: Journal    
DOI: 10.1175/BAMS-D-15-00267.1     Document Type: Review

References (21)

1. Ambaum, M. H. P., 2010: Significance tests in climate science. J. Climate, 23, 5927-5932, doi: 10.1175/2010JCLI3746.1.
2. Benjamini, Y., and Y. Hochberg, 1995: Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Stat. Soc., 57B, 289-300.
3. Hartmann, D. L., and Coauthors, 2013: Observations: Atmosphere and surface. Climate Change 2013: The Physical Science Basis, T. F. Stocker et al., Eds., Cambridge University Press, 159-254.
4. Jolliffe, I. T., 2004: P stands for⋯ Weather, 59, 77-79, doi: 10.1256/wea.132.03.
5. Katz, R. W., 1982: Statistical evaluation of climate experiments with general circulation models: A parametric time series modeling approach. J. Atmos. Sci., 39, 1446-1455, doi: 10.1175/1520-0469(1982)039<1446:SEOCEW>2.0.CO;2.
6. -, 2002: Sir Gilbert Walker and a connection between El Niño and statistics. Stat. Sci., 17, 97-112, doi: 10.1214/ss/1023799000.
7. -, and B. G. Brown, 1991: The problem of multiplicity in research on teleconnections. Int. J. Climatol., 11, 505-513, doi: 10.1002/joc.3370110504.
8. Livezey, R. E., and W. Y. Chen, 1983: Statistical field significance and its determination by Monte Carlo techniques. Mon. Wea. Rev., 111, 46-59, doi: 10.1175/1520-0493(1983)111<0046:SFSAID>2.0.CO;2.
9. Mielke, P. W., K. J. Berry, and G. W. Brier, 1981: Application of multi-response permutation procedures for examining seasonal changes in monthly mean sea level pressure patterns. Mon. Wea. Rev., 109, 120-126, doi: 10.1175/1520-0493(1981)109<0120:AOMRPP>2.0.CO;2.
10. Polyak, I., 1996: Computational Statistics in Climatology. Oxford University Press, 358 pp.
11. Storey, J. D., and R. Tibshirani, 2003: Statistical significance for genomewide studies. Proc. Natl. Acad. Sci. USA, 100, 9440-9445, doi: 10.1073/pnas.1530509100.
12. Ventura, V., C. J. Paciorek, and J. S. Risbey, 2004: Controlling the proportion of falsely rejected hypotheses when conducting multiple tests with climatological data. J. Climate, 17, 4343-4356, doi: 10.1175/3199.1.
13. von Storch, H., 1982: A remark on Chervin-Schneider's algorithm to test significance of climate experiments with GCM's. J. Atmos. Sci., 39, 187-189, doi: 10.1175/15200469(1982)039<0187:AROCSA>2.0.CO;2.
14. -, and F. W. Zwiers, 1999: Statistical Analysis in Climate Research. Cambridge University Press, 484 pp.
15. Vose, R. S., R. L. Schmoyer, P. M. Steurer, T. C. Peterson, R. Heim, T. R. Karl, and J. Eischeid, 1992: The Global Historical Climatology Network: Long-term monthly temperature, precipitation, sea level pressure, and station pressure data. Environmental Sciences Division Publ. 3912, ORNL/CDIAC-53, NDP-041, 325 pp., doi: 10.3334/CDIAC/cli.ndp041.
16. Walker, G. T., 1914: Correlation in seasonal variations of weather. III. On the criterion for the reality of relationships or periodicities. Mem. Indian Meteor. Dept., 21 (9), 13-15.
17. Wilks, D. S., 1997: Resampling hypothesis tests for autocorrelated fields. J. Climate, 10, 65-83, doi: 10.1175/1520-0442(1997)010<0065:RHTFAF>2.0.CO;2.
18. -, 2006: On "field significance" and the false discovery rate. J. Appl. Meteor. Climatol., 45, 1181-1189, doi: 10.1175/JAM2404.1.
19. -, 2011: Statistical Methods in the Atmospheric Sciences. 3rd ed. Academic Press, 676 pp.
20. Zwiers, F. W., 1987: Statistical considerations for climate experiments. Part II: Multivariate tests. J. Climate Appl. Meteor., 26, 477-487, doi: 10.1175/1520-0450(1987)026<0477:SCFCEP>2.0.CO;2.
21. -, and H. J. Thiébaux, 1987: Statistical considerations for climate experiments. Part I: Scalar tests. J. Climate Appl. Meteor., 26, 465-476, doi: 10.1175/15200450(1987)026<0464:SCFCEP>2.0.CO;2.