The New Statistics: Why and How

Psychological Science

Volumn 25, Issue 1, 2014, Pages 7-29

  Cumming, Geoff ^a  

^a LA TROBE UNIVERSITY   (Australia)

Author keywords

estimation; meta analysis; replication; research integrity; research methods; statistical analysis; the new statistics

Indexed keywords

HUMAN; MEDICAL RESEARCH; PSYCHOLOGY; STANDARDS; STATISTICAL ANALYSIS; STATISTICS;

BIOMEDICAL RESEARCH; DATA INTERPRETATION, STATISTICAL; HUMANS; PSYCHOLOGY; STATISTICS AS TOPIC;

EID: 84891940182     PISSN: 09567976     EISSN: 14679280     Source Type: Journal    
DOI: 10.1177/0956797613504966     Document Type: Article

References (84)

1. Altman D. G.,Machin D.,Bryant T. N.,Gardner M. J.Statistics with confidence: Confidence intervals and statistical guidelines. 2000). Statistics with confidence: Confidence intervals and statistical guidelines (2nd ed.). London, England: BMJ Books.London, England: BMJ Books; 2000:.
2. Publication manual of the American Psychological Association. 2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author.Washington, DC: Author; 2010:.
3. AndersonD. (1997). A few quotes regarding hypothesis testing. Retrieved from tiny.cc/nhstquotes.
4. Baguley T.Serious stats: A guide to advanced statistics for the behavioral sciences. Houndmills, England: Palgrave Macmillan; 2012:.
5. Beck A. T.,Steer R. A.,Ball R.,Ranieri W. F.Comparison of Beck Depression Inventories -IA and -II in psychiatric outpatients.Journal of Personality Assessment. 1996;67:588-597.
6. BeckerL. A. (1999). Effect size calculators. Retrieved from tiny.cc/beckerescalc.
7. Blouin D. C.,Riopelle A. J.On confidence intervals for within-subjects designs.Psychological Methods. 2005;10:397-412.
8. Bonett D. G.Meta-analytic interval estimation for standardized and unstandardized mean differences.Psychological Methods. 2009;14:225-238.
9. Borenstein M.,Hedges L. V.,Higgins J. P. T.,Rothstein H. R.Introduction to meta-analysis. Chichester, England: Wiley; 2009:.
10. Cohen J.The statistical power of abnormal social psychological research: A review.Journal of Abnormal and Social Psychology. 1962;65:145-153.
11. Cohen J.Statistical power analysis for the behavioral sciences. 1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.Hillsdale, NJ: Erlbaum; 1988:.
12. Cooper H.,Patall E. A.The relative benefits of meta-analysis conducted with individual participant data versus aggregated data.Psychological Methods. 2009;14:165-176.
13. Cooper H. M.Research synthesis and meta-analysis: A step-by-step approach. 2010). Research synthesis and meta-analysis: A step-by-step approach (4th ed.). Thousand Oaks, CA: Sage.Thousand Oaks, CA: Sage; 2010:.
14. Cooper H. M.Reporting research in psychology: How to meet journal article reporting standards. Washington, DC: American Psychological Association; 2011:.
15. CoulsonM.HealeyM.FidlerF.CummingG. (2010). Confidence intervals permit, but don't guarantee, better inference than statistical significance testing. Frontiers in Quantitative Psychology and Measurement, 1, 26. Retrieved from http://www.frontiersin.org/Journal/10.3389/fpsyg.2010.00026/full.
16. CummingG. (2006). Meta-analysis: Pictures that explain how experimental findings can be integrated. Retrieved from https://www.stat.auckland.ac.nz/~iase/publications/17/C105.pdf.
17. Cumming G.Inference by eye: Pictures of confidence intervals and thinking about levels of confidence.Teaching Statistics. 2007;29:89-93.
18. Cumming G.Replication and p intervals: p values predict the future only vaguely, but confidence intervals do much better.Perspectives on Psychological Science. 2008;3:286-300.
19. Cumming G.Inference by eye: Reading the overlap of independent confidence intervals.Statistics in Medicine. 2009;28:205-220.
20. Cumming G.Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. New York, NY: Routledge; 2012:.
21. CummingG. (2013). The new statistics: Estimation for better research. Retrieved from www.thenewstatistics.com.
22. Cumming G.,Fidler F.Confidence intervals: Better answers to better questions.Zeitschrift für Psychologie/Journal of Psychology. 2009;217:15-26.
23. Cumming G.,Fidler F.,Kalinowski P.,Lai J.The statistical recommendations of the American Psychological Association Publication Manual: Effect sizes, confidence intervals, and meta-analysis.Australian Journal of Psychology. 2012;64:138-146.
24. Cumming G.,Fidler F.,Vaux D. L.Error bars in experimental biology.Journal of Cell Biology. 2007;177:7-11.
25. Cumming G.,Finch S.A primer on the understanding, use and calculation of confidence intervals that are based on central and noncentral distributions.Educational and Psychological Measurement. 2001;61:532-574.
26. Cumming G.,Finch S.Inference by eye: Confidence intervals, and how to read pictures of data.American Psychologist. 2005;60:170-180.
27. Cumming G.,Maillardet R.Confidence intervals and replication: Where will the next mean fall?.Psychological Methods. 2006;11:217-227.
28. Cumming G.,Williams J.,Fidler F.Replication, and researchers' understanding of confidence intervals and standard error bars.Understanding Statistics. 2004;3:299-311.
29. Dawkins R.The ancestor's tale: A pilgrimage to the dawn of life. London, England: Weidenfeld & Nicolson; 2004:.
30. Douglas H.Value-free science? Ideals and illusions. Kincaid H.Dupre J.Wylie A., ed. Oxford, England: Oxford University Press; 2007:120-139.
31. Ellis P. D.The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results. Cambridge, England: Cambridge University Press; 2010:.
32. EllisP. D. (n.d.). Effect size FAQs: Research that matters, results that make sense. Retrieved from tiny.cc/ellissite.
33. Erceg-Hurn D. M.,Mirosevich V. M.Modern robust statistical methods: An easy way to maximize the accuracy and power of your research.American Psychologist. 2008;63:591-601.
34. Faulkner C.,Fidler F.,Cumming G.The value of RCT evidence depends on the quality of statistical analysis.Behaviour Research and Therapy. 2008;46:270-281.
35. FidlerF. (2005). From statistical significance to effect estimation: Statistical reform in psychology, medicine and ecology (Doctoral dissertation). Retrieved from tiny.cc/fionasphd.
36. Fidler F.,Cumming G.Handbook of psychology: Vol. 2. Research methods in psychology. 2013). Effect size estimation and confidence intervals. In SchinkaJ. A.VelicerW. F. (Eds.), Handbook of psychology: Vol. 2. Research methods in psychology (2nd ed., pp. 142-163). Hoboken, NJ: Wiley.Schinka J. A.Velicer W. F., ed. Hoboken, NJ: Wiley; 2013:142-163.
37. Fidler F.,Faulkner S.,Cumming G.Evidence-based outcome research: A practical guide to conducting randomized controlled trials for psychosocial interventions. Nezu A. M.Nezu C. M., ed. New York, NY: Oxford University Press; 2008:315-334.
38. Fidler F.,Loftus G.Why figures with error bars should replace p values: Some conceptual arguments and empirical demonstrations.Zeitschrift für Psychologie/Journal of Psychology. 2009;217:27-37.
39. Fidler F.,Thomason N.,Cumming G.,Finch S.,Leeman J.Editors can lead researchers to confidence intervals, but can't make them think: Statistical reform lessons from medicine.Psychological Science. 2004;15:119-126.
40. Fiedler K.,Kutner F.,Krueger J. I.The long way from α-error control to validity proper: Problems with a short-sighted false-positive debate.Perspectives on Psychological Science. 2012;7:661-669.
41. Finch S.,Cumming G.Putting research in context: Understanding confidence intervals from one or more studies.Journal of Pediatric Psychology. 2009;34:903-916.
42. Finch S.,Cumming G.,Williams J.,Palmer L.,Griffith E.,Alders C.,Goodman O.Reform of statistical inference in psychology: The case of.Memory & Cognition. Behavior Research Methods, Instruments, & Computers. 2004;36:312-324.
43. Fritz C. O.,Morris P. E.,Richler J. J.Effect size estimates: Current use, calculations, and interpretation.Journal of Experimental Psychology: General. 2012;141:2-18.
44. GerlancD.KirbyK. (2013). bootES: Bootstrap effect sizes. Retrieved from tiny.cc/bootes.
45. Greenland S.,Schlesselman J. J.,Criqui M. H.The fallacy of employing standardized regression coefficients and correlations as measures of effect.American Journal of Epidemiology. 1986;123:203-208.
46. Grissom R. J.,Kim J. J.Effect sizes for research: Univariate and multivariate applications. 2012). Effect sizes for research: Univariate and multivariate applications (2nd ed.). New York, NY: Routledge.New York, NY: Routledge; 2012:.
47. Hoenig J. M.,Heisey D. M.The abuse of power: The pervasive fallacy of power calculations for data analysis.The American Statistician. 2001;55:19-24.
48. Uniform requirements for manuscripts submitted to biomedical journals.Annals of Internal Medicine. 1988;108:258-265.
49. IoannidisJ. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2, e124. Retrieved from http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124.
50. Kelley K.Methods for the behavioral, educational, and social sciences: An R package.Behavior Research Methods. 2007;39:979-984.
51. KelleyK. (n.d.). MBESS: An R package. Retrieved from tiny.cc/mbess.
52. Kirby K. N.,Gerlanc D.BootES: An R package for bootstrap confidence intervals on effect sizes.Behavior Research Methods. 2013;:.
53. Kirk R. E.Handbook of research methods in experimental psychology. Davis S. F., ed. Malden, MA: Blackwell; 2003:83-105.
54. Kline R. B.Beyond significance testing: Reforming data analysis methods in behavioral research. Washington, DC: APA Books; 2004:.
55. KlineR. B. (2013a). Beyond significance testing (2nd edition): Reader and instructor resource guide. Retrieved from tiny.cc/rklinesite.
56. Kline R. B.Beyond significance testing: Statistics reform in the behavioral sciences. 2013b). Beyond significance testing: Statistics reform in the behavioral sciences (2nd ed.). Washington, DC: APA Books.Washington, DC: APA Books; 2013b:.
57. Kruschke J. K.Doing Bayesian data analysis: A tutorial with R and BUGS. New York, NY: Academic Press; 2010:.
58. Masicampo E. J.,Lalande D. R.A peculiar prevalence of p values just below.05.The Quarterly Journal of Experimental Psychology. 2012;65:2271-2279.
59. Masson M. E. J.,Loftus G. R.Using confidence intervals for graphically based data interpretation.Canadian Journal of Experimental Psychology/Revue Canadienne de Psychologie Expérimentale. 2003;57:203-220.
60. Maxwell S. E.The persistence of underpowered studies in psychological research: Causes, consequences, and remedies.Psychological Methods. 2004;9:147-163.
61. Maxwell S. E.,Kelley K.,Rausch J. R.Sample-size planning for statistical power and accuracy in parameter estimation.Annual Review of Psychology. 2008;59:537-563.
62. McCarthy M. A.Bayesian methods for ecology. Cambridge, England: Cambridge University Press; 2007:.
63. Panter A. T.,Sterba S. K.Handbook of ethics in quantitative methodology. New York, NY: Routledge; 2011:.
64. Pashler H.,Wagenmakers E. J.Editors' introduction to the special section on replicability in psychological science: A crisis of confidence?.Perspectives on Psychological Science. 2012;7:528-530.
65. Psychonomic Society. (2012). 2012 Psychonomic Society guidelines on statistical issues. Retrieved from tiny.cc/psychonomicstats.
66. Rodgers J. L.The epistemology of mathematical and statistical modeling: A quiet methodological revolution.American Psychologist. 2010;65:1-12.
67. Rosenthal R.,Rosnow R. L.,Rubin D. B.Contrasts and effect sizes in behavioral research: A correlational approach. Cambridge, England: Cambridge University Press; 2000:.
68. Schmidt F. L.,Hunter J. E.What if there were no significance tests?. Harlow L. L.Mulaik S. A.Steiger J. H., ed. Mahwah, NJ: Erlbaum; 1997:37-64.
69. Schmidt F. L.,Oh I.-S.,Hayes T. L.Fixed- versus random-effects models in meta-analysis: Model properties and an empirical comparison of differences in results.British Journal of Mathematical and Statistical Psychology. 2009;62:97-128.
70. Simmons J. P.,Nelson L. D.,Simonsohn U.False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant.Psychological Science. 2011;22:1359-1366.
71. Smithson M.Confidence intervals. Thousand Oaks, CA: Sage; 2003:.
72. SmithsonM. (n.d.). Scripts and software for noncentral confidence interval and power calculations. Retrieved from tiny.cc/mikecis.
73. SoperD. (2006-2013). Statistics calculators Version 3.0. Retrieved from tiny.cc/sopercalc.
74. Spellman B. A.Introduction to the special section on research practices.Perspectives on Psychological Science. 2012;7:655-656.
75. SPSP Task Force on Publication and Research Practices. (in press). Improving the dependability of research in personality and social psychology: Recommendations for research and educational practice. Personality and Social Psychology Review.
76. Steiger J. H.Beyond the F test: Effect size confidence intervals and tests of close fit in the analysis of variance and contrast analysis.Psychological Methods. 2004;9:164-182.
77. Tabachnick B. G.,Fidell L. S.Using multivariate statistics. 2007). Using multivariate statistics (5th ed.). Boston, MA: Pearson.Boston, MA: Pearson; 2007:.
78. Tukey J. W.Exploratory data analysis. Reading, MA: Addison-Wesley; 1977:.
79. van Deemter K.Not exactly: In praise of vagueness. Oxford, England: Oxford University Press; 2010:.
80. Velicer W. F.,Cumming G.,Fava J. L.,Rossi J. S.,Prochaska J. O.,Johnson J.Theory testing using quantitative predictions of effect size.Applied Psychology: An International Review. 2008;57:589-608.
81. Wagenmakers E. J.,Wetzels R.,Borsboom D.,van der Maas H. J. L.,Kievit R. A.An agenda for purely confirmatory research.Perspectives on Psychological Science. 2012;7:632-638.
82. Wilcox R.Modern statistics for the social and behavioral sciences: A practical introduction. Boca Raton, FL: CRC Press; 2011:.
83. Wilkinson L.Statistical methods in psychology journals: Guidelines and explanations.American Psychologist. 1999;54:594-604.
84. WilsonD. B. (n.d.). Practical meta-analysis effect size calculator. Retrieved from tiny.cc/campbellcalc.