Inference by eye: Reading the overlap of independent confidence intervals

Statistics in Medicine

Volumn 28, Issue 2, 2009, Pages 205-220

  Cumming, Geoff ^a  

^a LA TROBE UNIVERSITY   (Australia)

Author keywords

Confidence intervals; Error bars; Inference by eye; Overlap; P values; Statistical graphics

Indexed keywords

ACCURACY; ANALYSIS OF VARIANCE; ARTICLE; CONFIDENCE INTERVAL; CONTROLLED STUDY; CORRELATION COEFFICIENT; INFERENCE BY EYE; INFERENTIAL STATISTICS; INTERMETHOD COMPARISON; SAMPLE SIZE; STATISTICAL SIGNIFICANCE;

BIAS (EPIDEMIOLOGY); CONFIDENCE INTERVALS; DATA DISPLAY; DATA INTERPRETATION, STATISTICAL;

EID: 61749094802     PISSN: 02776715     EISSN: 10970258     Source Type: Journal    
DOI: 10.1002/sim.3471     Document Type: Article

References (39)

1. Belia S, Fidler F, Williams J, Cumming G. Researchers misunderstand confidence intervals and standard error bars. Psychological Methods 2005; 10:389-396. DOI: 10.1037/1082-989X.10.4.389.
2. Schenker N, Gentleman JF. On judging the significance of differences by examining the overlap between confidence intervals. The American Statistician 2001; 55:182-186. DOI: 10.1198/ 000313001317097960.
3. Austin PC, Hux JE. A brief note on overlapping confidence intervals. Journal of Vascular Surgery 2002; 36:194-195. DOI: 10.1067/ mva.2002.125015.
4. Wolfe R, Hanley J. If we're so different why do we keep overlapping? When 1 plus 1 doesn't make 2. Canadian Medical Association Journal 2002; 166:65-66.
5. Cumming G, Finch S. Inference by eye: confidence intervals and how to read pictures of data. American Psychologist 2005; 60:170-180. DOI: 10.1037/0003-066X.60.2.170.
6. Welch BL. The significance of the difference between two means when the population variances are unequal. Biometrika 1938; 29:350-362. DOI: 10.1093/biomet/29.3-4.350.
7. Satterthwaite FE. An approximate distribution of estimates of variance components. Biometrics Bulletin 1946; 2:110-114. DOI: 10.2307/3002019.
8. Browne RH. On visual assessment of the significance of a mean difference. Biometrics 1979; 35:657-665. DOI: 10.2307/ 2530259.
9. Simpson GG, Roe A, Lewontin RC. Quantitative Zoology (revised edn). Harcourt, Brace & World: New York, 1960.
10. Payton ME, Greenstone MH, Schenker N. Overlapping confidence intervals or standard error intervals: what do they mean in terms of statistical significance? Journal of Insect Science 2003; 3:34-39. Available at insectscience.org/3.34.
11. Payton ME, Miller AE, Raun WR. Testing statistical hypotheses using standard error bars and confidence intervals. Communications in Soil Science and Plant Analysis 2000; 31:547-551.
12. Bulpitt CJ. Confidence intervals. The Lancet 1987; i:494-497. DOI: 10.1016/S0140-6736(87)92100-3.
13. Cole SR, Blair RC. Overlapping confidence intervals. Journal of the American Academy of Dermatology 1999; 41:1051-1052.
14. Wheeler MW, Park RM, Bailer AJ. Comparing median lethal concentration values using confidence interval overlap or ratio tests. Environmental Toxicology and Chemistry 2006; 25:1441-1444. DOI: 10.1897/05-320R.1.
15. Shaughnessy JJ, Zechmeister EB, Zechmeister JS. Research Methods in Psychology (7th edn). McGraw-Hill: New York, 2006.
16. Bigby M, Gadenne A. Understanding and evaluating clinical trials. Journal of the American Academy of Dermatology 1996; 34:555-590. DOI: 10.1016/S0190-9622(96)80053-3.
17. Bigby M, Gadenne A. Reply. Journal of the American Academy of Dermatology 1997; 37:804-805. DOI: 10.1016/ S0190-9622(97)70127-0.
18. Rahlfs VW. Understanding and evaluating clinical trials. Journal of the American Academy of Dermatology 1997; 37:803-804. DOI: 10.1016/S0190-9622(97)70126-9.
19. Ryan GW, Leadbetter SD. On the misuse of confidence intervals for two means in testing for the significance of the difference between the means. Journal of Modern Applied Statistical Methods 2002; 1:473-478.
20. Saville DJ. Basic statistics and the inconsistency of multiple comparison procedures. Canadian Journal of Experimental Psychology 2003; 57:167-175.
21. Goldstein H, Healy MJR. The graphical presentation of a collection of means. Journal of the Royal Statistical Society A 1995; 158:175-177. DOI: 10.2307/2983411.
22. Burghardt GM. Comparative prey-attack studies in newborn snakes of the genus Thamnophis. Behaviour 1969; 33:77-114. DOI: 10.1163/ 156853969X00332.
23. Moses LE. Graphical methods in statistical analysis. Annual Review of Public Health 1987; 8:309-353. DOI: 10.1146/ annurev.pu.08.050187.001521.
24. Barr DR. Using confidence intervals to test hypotheses. Journal of Quality Technology 1969; 1:256-258.
25. Nelson LS. Evaluating overlapping confidence intervals. Journal of Quality Technology 1989; 21:140-141.
26. Tryon WW. Evaluating statistical difference, equivalence, and indeterminacy using inferential confidence intervals: an integrated alternative method of conducting null hypothesis statistical tests. Psychological Methods 2001; 6:371-386. DOI: 10.1037/ 1082-989X.6.4.371.
27. Julious SA. Using confidence intervals around individual means to assess statistical significance between two means. Pharmaceutical Statistics 2004; 3:217-222. DOI: 10.1002/pst.126.
28. Sall J. Graphical comparison of means. Statistical Computing and Graphics Newsletter of the American Statistical Association 1992; 3(1):27-32.
29. Sall J, Creighton L, Lehman A. JMP Start Statistics: A Guide to Statistics and Data Analysis Using JMP and JMP IN Software (3rd edn). Thomson Brooks/Cole: Pacific Grove, CA, 2005.
30. International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. Annals of Internal Medicine 1997; 126:36-47.
31. Newcombe RG, Altman DG. Proportions and their differences. In Statistics with Confidence: Confidence Intervals and Statistical Guidelines (2nd edn), Altman DG, Machin D, Bryant TN, Gardner MJ (eds). British Medical Journal Books: London, 2000; 45-56.
32. Newcombe RG. Two-sided confidence intervals for the single proportion: comparison of seven methods. Statistics in Medicine 1998; 17:857-872. DOI: 10.1002/ (SICI)1097-0258(19980430)17:8<857::AID-SIM777>3.0.CO;2-E.
33. Newcombe RG. Interval estimation for the difference between independent proportions: comparison of eleven methods. Statistics in Medicine 1998; 17:873-890. DOI: 10.1002/ (SICI)1097-0258(19980430)17:8<873::AID-SIM779>3.0.CO;2-I.
34. Howell DC. Statistical Methods for Psychology (5th edn). Duxbury: Pacific Grove, CA, 2002.
35. American Psychological Association. Publication Manual of the American Psychological Association (5th edn). Author: Washington, DC, 2001.
36. American Educational Research Association. Standards for reporting on empirical social science research in AERA publications. Educational Researcher 2006; 35:33-40. DOI: 10.3102/0013189X035006033.
37. Fidler F, Cumming G, Burgman M, Thomason N. Statistical reform in medicine, psychology and ecology. Journal of Socio-Economics 2004; 33:615-630. DOI: 10.1016/j.socec.2004.09.035.
38. Cumming G. Replication and p intervals: p values predict the future only vaguely, but confidence intervals do much better. Perspectives on Psychological Science 2008; 3:286-300. DOI: 10.1111/j.1745-6924.2008.00079.x.
39. Cumming G, Fidler F, Leonard M, Kalinowski P, Christiansen A, Kleinig A, Lo J, McMenamin N, Wilson S. Statistical reform in psychology: is anything changing? Psychological Science 2007; 18:230-232. DOI: 10.1111/j.1467-9280.2007.01881.x.