Result-blind peer reviews and editorial decisions: A missing pillar of scientific culture

European Psychologist

Volumn 18, Issue 4, 2013, Pages 286-294

  Greve, Werner ^a   Bröder, Arndt ^b   Erdfelder, Edgar ^b  

^a UNIVERSITY OF HILDESHEIM   (Germany)

^b UNIVERSITY OF MANNHEIM   (Germany)

Author keywords

Methodology; Publication bias; Replication

Indexed keywords

EID: 84890302661     PISSN: 10169040     EISSN: None     Source Type: Journal    
DOI: 10.1027/1016-9040/a000144     Document Type: Review

References (44)

1. Bakker, M., & Wicherts, J. M. (2011). The (mis)reporting of statistical results in psychology journals. Behavior Research Methods, 43, 666-678.
2. Bones, A. (2012). We knew the future all along: Scientific hypothesizing is much more accurate than other forms of precognition-A satire in one part. Perspectives on Psychological Science, 7, 307-309.
3. Bredenkamp, J. (1969). Über die Anwendung des Signifikanztests bei theorietestenden Experimenten [Application of significance tests in theory-testing experiments]. Psychologische Beiträge, 11, 275-285.
4. Bredenkamp, J. (1972). Der Signifikanztest in der psychologischen Forschung [The test of significance in psychological research]. Frankfurt, Germany: Akademische Verlagsgesellschaft.
5. Cohen, J. (1962). Statistical power of abnormal-social psychological research: A review. Journal of Abnormal Psychology, 65, 145-153.
6. Cohen, J. (1988). Statistical power analysis for the social sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
7. Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997-1003.
8. Crandall, R. (1990). Improving editorial procedures. American Psychologist, 45, 665-666.
9. Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 629-34.
10. Erdfelder, E. (2010). Experimental Psychology: A note on statistical analysis. Experimental Psychology, 57, 1-4.
11. Fritz, A., Scherndl, T., & Kühberger, A. (2013). A comprehensive review of reporting practices in psychology journals: Are effect sizes really enough? Theory & Psychology, 23, 98-122. doi: 10.1177/0959354312436870
12. Gigerenzer, G., Krauss, S., & Vitouch, O. (2004). The null ritual: What you always wanted to know about significance testing but were afraid to ask. In D. Kaplan (Ed.), The SAGE handbook of quantitative methodology for the social sciences (pp. 391-408). Thousand Oaks, CA: Sage.
13. Gleser, L. J., & Olkin, I. (1996). Models for estimating the number of unpublished studies. Statistics in Medicine, 15, 2493-2507.
14. Greenwald, A. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin, 82, 1-20.
15. Grob, A. (2010). Letter from the new editor-in-chief. European Psychologist, 15, 1-2.
16. Hunt, E. (1990). The traditional editorial statement. Journal of Experimental Psychology: General, 119, 3-4.
17. Iverson, G. F., Lee, M. D., & Wagenmakers, E.-J. (2009). p(rep) misestimates the probability of replication. Psychonomic Bulletin & Review, 16, 424-429.
18. Iyengar, S., & Greenhouse, J. B. (1988). Selection models and the file drawer problem. Statistical Science, 3, 109-135.
19. Killeen, P. R. (2005). An alternative to null-hypothesis significance tests. Psychological Science, 16, 345-353.
20. Kupfersmid, J. (1988). Improving what is published: a model in search for an editor. American Psychologist, 43, 635-642.
21. Lau, J., Ioannidis, J. P. A., Norma, T., Schmid, C. H., & Olkin, I. (2006). The case of the misleading funnel plot. British Medical Journal, 333, 597-600.
22. Lehrer, J. (2010). The truth wears off. Is there something wrong with the scientific method? The New Yorker, December 13, 2010.
23. Losee, J. (2001). A historical introduction to the philosophy of science (4th ed.). Oxford, UK: Oxford University Press.
24. Mayor, J. (2010). Are scientists nearsighted gamblers? The misleading nature of impact factors. Frontiers in Psychology, 1, 1-2.
25. Palmer, A. R. (1999). Detecting biases in meta-analyses: A case study of fluctuating asymmetry and sexual selection. The American Naturalist, 154, 220-233.
26. Popper, K. R. (1959). The logic of scientific discovery. London, UK: Routledge.
27. Renkewitz, F., Fuchs, H. M., & Fiedler, S. (2011). Is there evidence of publication bias in JDM research? Judgment and Decision Making, 6, 870-881.
28. Roediger, H. L. III (2012). Psychology's woes and a partial cure: The value of replication. APS Observer, 25, 27-29.
29. Rosenthal, R. (1978). Combining results of independent studies. Psychological Bulletin, 85, 185-193.
30. Rosenthal, R. (1979). The "file drawer problem" and tolerance for null results. Psychological Bulletin, 86, 638-641.
31. Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. J. (2009). Bayesian t-tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16, 225-237.
32. Scargle, J. D. (2000). Publication bias: The "file-drawer" problem in scientific inference. Journal of Scientific Exploration, 14, 91-106.
33. Schimmack, U. (2012). The ironic effect of significant results on the credibility of multiple study articles. Psychological Methods 17, 551-566. doi: 10.1037/a0029487
34. Schooler, J. (2011). Unpublished results hide the decline effect. Nature, 470, 437.
35. Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 107, 309-316.
36. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359-1366.
37. Spellman, B. A. (2012). Introduction to the Special Section: Data, data, everywhere. especially in my file drawer. Perspectives on Psychological Science, 7, 58-59.
38. Sterling, T. D. (1959). Publication decisions and their possible effects on inferences drawn from tests of significance - or vice versa. Journal of the American Statistical Association, 54, 30-34.
39. Terrin, N., Schmid, C. H., & Lau, J. (2005). In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias. Journal of Clinical Epidemiology, 58, 894-901.
40. Thompson, B. (1994). The pivotal role of replication in psychology: Empirically evaluating the replicability of sample results. Journal of Personality, 62, 157-176.
41. Tversky, A., & Kahneman, D. (1982). Judgment under uncertainty: Heuristics and biases. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 3-20). Cambridge, UK: Cambridge University Press.
42. Walster, G. W., & Cleary, T. A. (1970). A proposal for a new editorial policy in the social sciences. The American Statistician, 16, 16-19.
43. Wilkinson, L., The Task Force on Statistical Inference (1999). Statistical Methods in Psychology Journals: Guidelines and Explanations. American Psychologist, 54, 594-604.
44. Yong, E. (October 3, 2012). Nobel laureate challenges psychologists to clean up their act. Nature News and Comment, http://www.nature.com/news/nobel- laureate-challengespsychologists-to-clean-up-their-act-1.11535 (retrieved 05 October 2012).