The new statistics in psychology - The status quo and future of data analysis;Die new statistics in der psychologie: Status quo und zukunft der datenanalyse

Zeitschrift fur Entwicklungspsychologie und Padagogische Psychologie

Volumn 50, Issue 1, 2018, Pages 3-18

  Schäfer, Thomas ^a  

^a CHEMNITZ UNIVERSITY OF TECHNOLOGY   (Germany)

Author keywords

Confidence interval; Effect size; Methods; Power; Replication; Statistics

Indexed keywords

ARTICLE; AWARENESS; CALCULATION; CHECKLIST; DATA ANALYSIS; EFFECT SIZE; HUMAN; HUMAN EXPERIMENT; LANDSCAPE; PSYCHOLOGY; PUBLICATION; SIMULATION; STATISTICS; THUMB;

EID: 85051378800     PISSN: 00498637     EISSN: 21906262     Source Type: Journal    
DOI: 10.1026/0049-8637/a000184     Document Type: Article

References (64)

1. American Psychological Association (2001). Publication manual of the American Psychological Association (5th ed.). Washington, DC: American Psychological Association.
2. American Psychological Association (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association.
3. Arbuckle, J. L. (2006). Amos (Version 7.0) [Computer Program]. Chicago: SpSS.
4. Bakker, M., Dijk, A. van & Wicherts, J. M. (2012). The rules of the game called psychological science. Perspectives on Psychological Science, 7, 543-554.
5. Bakker, M., and Hartgerink, C. H., Wicherts, J. M. & Maas, H. L. van der (2016). Researchers' intuitions about power in psychological research. Psychological Science, 27, 1069-1077.
6. Bates, D., Maechler, M., Bolker, B., Walker, S., Christensen, R. H. B., Singmann, H. et al. (2014). Package 'lme4'. Vienna: R foundation for statistical computing.
7. Berkson, J. (1938). Some difficulties of interpretation encountered in the application of the chi-square test. Journal of the American Statistical Association, 33, 526-536.
8. Bezeau, S. & Graves, R. (2001). Statistical power and effect sizes of clinical neuropsychology research. Journal of Clinical and Experimental Neuropsychology, 23, 399-406.
9. Borenstein, M., and Hedges, L. V., Higgins, J. & Rothstein, H. R. (2010). A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods, 1, 97-111.
10. Brandt, M. J., Ijzerman, H., Dijksterhuis, A., Farach, F. J., Geller, J., Giner-Sorolla, R. et al. (2014). The replication recipe: What makes for a convincing replication? Journal of Experimental Social Psychology, 50, 217-224.
11. Callahan, J. L. & Reio Jr, T. G. (2006). Making subjective judgments in quantitative studies: The importance of using effect sizes and confidence intervals. Human Resource Development Quarterly, 17, 159-173.
12. Cohen, J. (1962). The statistical power of abnormal social psychological research: a review. Journal of Abnormal and Social Psychology, 65, 145-153.
13. Cohen, J. (1969). Statistical power analysis for the behavioral sciences. New York, NY: Academic Press.
14. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York, NY: Academic Press.
15. Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304-1312.
16. Cohen, J. (1994). The earth is round (p <.05). American Psychologist, 49, 997-1003.
17. Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. New York, NY: Routledge.
18. Cumming, G. (2013). The new statistics: Estimation for better research. Retrieved from www.thenewstatistics.com
19. Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25, 7-29.
20. Cumming, G. & Finch, S. (2005). Inference by eye: Confidence intervals, and how to read pictures of data. American Psychologist, 60, 170-180. https://doi.org/10.1037/0003-066X.60.2.170
21. Dienes, Z. (2016). How Bayes factors change scientific practice. Journal of Mathematical Psychology, 72, 78-89.
22. Ellis, P. D. (2010). The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results. Cambridge, United Kingdom: Cambridge University Press.
23. Fidler, F., Cumming, G., Thomason, N., Pannuzzo, D., Smith, J., Fyffe, P., et al (2005). Toward improved statistical reporting in the journal of consulting and clinical psychology. Journal of Consulting and Clinical Psychology, 73, 136-143.
24. Fidler, F. & Loftus, G. R. (2009). Why figures with error bars should replace p values: Some conceptual arguments and empirical demonstrations. Zeitschrift für Psychologie, 217, 27-37.
25. Fisher, R. A. (1956). Statistical Methods and Scientific Inference. Edinburgh: Oliver & Boyd.
26. Fraley, R. C. & Vazire, S. (2014). The N-pact factor: evaluating the quality of empirical journals with respect to sample size and statistical power. PloS ONE, 9, e109019.
27. Fritz, A., Scherndl, T. & Kühberger, A. (2012). A comprehensive review of reporting practices in psychological journals: Are effect sizes really enough? Theory & Psychology, 23, 98-122.
28. Fritz, C. O., Morris, P. E. & Richler, J. J. (2012). Effect size estimates: Current use, calculations, and interpretation. Journal of Experimental Psychology: General, 141, 2-18. https://doi.org/10.1037/a0024338
29. Gelman, A. & Loken, E. (2014). The Statistical Crisis in Science. Data-dependent analysis - a "garden of forking paths" - explains why many statistically significant comparisons don't hold up. American Scientist, 102, 460.
30. Gigerenzer, G. (1993). The superego, the ego, and the i d in statistical reasoning. In G. Keren & C. Lewis (Eds.), A handbook for data analysis in the behavioral sciences: Methodological issues (pp. 311-339). Hillsdale, NJ: Erlbaum.
31. Gigerenzer, G. (2004). Mindless statistics. The Journal of Socio-Economics, 33, 587-606.
32. Haller, H. & Krauss, S. (2002). Misinterpretations of significance: A problem students share with their teachers. Methods of Psychological Research, 7, 1-20.
33. Hammond, G. (1996). The objections to null hypothesis testing as a means of analysing psychological data. Australian Journal of Psychology, 48, 104-106.
34. Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2, e124. Retrieved from http://www.plosmedicine.org/article/info. https://doi.org/10.1371/journal.pmed.0020124
35. Ioannidis, J. P. A. (2014). How to make more published research true. PLoS medicine, 11, e1001747.
36. Ioannidis, J. P. A. (2016). Why most clinical research is not useful. PLoS medicine, 13, e1002049.
37. Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational and psychological measurement, 56, 746-759.
38. Kirk, R. E. (2007). Effect magnitude: a different focus. Journal of statistical planning and inference, 137, 1634-1646.
39. Krueger, J. (2001). Null hypothesis significance testing: On the survival of a flawed method. American Psychologist, 56, 16.
40. Mackinnon, S. (2013). Increasing statistical power in psychological research without increasing sample size. Retrieved January 6, 2017 from http://osc.centerforopenscience.org/category/content5.html
41. Margraf, J. (2015). Zur Lage der Psychologie. Psychologische Rundschau, 66, 1-30.
42. Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of consulting and clinical Psychology, 46, 806.
43. Munafò, M. R., Nosek, B. A., Bishop, D. V., Button, K. S., Chambers, C. D. Sert, N. P. du et al. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1, 0021.
44. Nasser-Abu Alhija, F. & Levy, A. (2009). Effect size reporting practices in published articles. Educational and Psychological Measurement, 69, 245-265.
45. Neyman, J. & Pearson, E. S. (1933). On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London, 231, 289-337.
46. Nickerson, R. S. (2000). Null hypothesis significance testing: a review of an old and continuing controversy. Psychological Methods, 5, 241-301.
47. Onwuegbuzie, A. J. & Levin, J. R. (2003). Without supporting statistical evidence, where would reported measures of substantive importance lead? To no good effect. Journal of Modern Applied Statistical Methods, 2, 12.
48. Open Science Collaboration. (2012). An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. Perspectives on Psychological Science, 7, 657-660.
49. Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349 (6251), aac4716.
50. Osborne, J. W. (2008). Sweating the small stuff in educational psychology: how effect size and power reporting failed to change from 1969 to 1999, and what that means for the future of changing practices. Educational Psychology, 28, 151-160.
51. Ottenbacher, K. J. (1996). The power of replications and replications of power. The American Statistician, 50, 271-275.
52. Roberts, S. & Pashler, H. (2000). How persuasive is a good fit? A comment on theory testing. Psychological Review, 107, 358.
53. Schäfer, T. (2017). Die Gefährdung des wissenschaftlichen Fortschritts durch adaptives Verhalten von Nachwuchswissenschaftler-innen. Kommentar zum Beitrag von Rentzsch, Harzer & Wolter (2017). Psychologische Rundschau, 68.
54. Schmidt, F. L. & Oh, I. S. (2016). The crisis of confidence in research findings in psychology: Is lack of replication the real Problem? Or is it something else? Archives of Scientific Psychology, 4, 32-37.
55. Schmidt, F. L., Oh, I. S. & Hayes, T. L. (2009). Fixed-versus random-effects models in meta-analysis: Model properties and an empirical comparison of differences in results. British Journal of Mathematical and Statistical Psychology, 62, 97-128. https://doi.org/10.1348/000711007X255327
56. Schönbrodt, F., Gollwitzer, M. & Abele-Brehm, A. (2017). Der Umgang mit Forschungsdaten im Fach Psychologie: Konkretisierung der DFG-Leitlinien. Psychologische Rundschau, 68, 20-35.
57. Sedlmeier, P. & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105, 309-316.
58. Simmons, J. P., Nelson, L. D. & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility indata collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359-1366. https://doi.org/10.1177/0956797611417632
59. Simonsohn, U. (2015). Small telescopes: Detectability and the evaluation of replication results. Psychological Science, 26, 559-569.
60. Spellman, B. A. (2012). Introduction to the special section on research practices. Perspectives on Psychological Science, 7, 655-656. https://doi.org/10.1177/1745691612465075
61. Tressoldi, P. E. (2012). Replication unreliability in psychology: elusive phenomena or "elusive" statistical power? Frontiers in Psychology, 3, 218.
62. Tukey, J. W. (1991). The philosophy of multiple comparisons. Statistical Science, 6, 100-116.
63. Verhagen, J. & Wagenmakers, E.-J. (2014). A Bayesian Test to Quantify the Success or Failure of a Replication Attempt. Journal of Experimental Psychology: General, 143, 1457-1475.
64. Wagenmakers, E.-J., Wetzels, R., Borsboom, D., Maas, H. J. L. van der & Kievit, R. A. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7, 632-638. https://doi.org/10.1177/1745691612463078