Meta-analyses are no substitute for registered replications: a skeptical perspective on religious priming

Frontiers in Psychology

Volumn 6, Issue , 2015, Pages

  van Elk, Michiel ^a   Matzke, Dora ^a   Gronau, Quentin F ^a   Guan, Maime ^b   Vandekerckhove, Joachim ^b   Wagenmakers, Eric Jan ^a  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Bayesian Bias Correction; meta analysis; PET PEESE; preregistration; prosocial behavior; religious priming

Indexed keywords

EID: 85118970278     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2015.01365     Document Type: Article

References (31)

1. Barber T. X. (1976). Pitfalls in Human Research: Ten Pivotal Points. New York: Pergamon Press Inc.
2. Bargh J. A. (2006). What have we been priming all these years? On the development, mechanisms, and ecology of nonconscious social behavior. Eur. J. Soc. Psychol. 36 147–168. 10.1002/ejsp.336
3. Chambers C. D. (2013). Registered reports: a new publishing initiative at Cortex. Cortex 49 609–610. 10.1016/j.cortex.2012.12.016
4. De Groot A. D. (1956/2014). The meaning of “significance” for different types of research. [translated and annotated by Eric-Jan Wagenmakers, Denny Borsboom, Verhagen, Rogier Kievit, Marjan Bakker, Angelique Cramer, Dora Matzke, Don Mellenbergh, and Han L. J. van der Maas]. Acta Psychol. 148 188–194. 10.1016/j.actpsy.2014.02.001
5. Eagly A. H. (2009). The his and hers of prosocial behavior: an examination of the social psychology of gender. Am. Psychol. 64 644. 10.1037/0003-066X.64.8.644
6. Egger M. Smith G. D. Schneider M. Minder C. (1997). Bias in meta-analysis detected by a simple, graphical test. Br. Med. J. 315 629–634. 10.1136/bmj.315.7109.629
7. Francis G. (2012). Publication bias and the failure of replication in experimental psychology. Psychon. Bull. Rev. 19 975–991. 10.3758/s13423-012-0322-y
8. Francis G. (2013). Replication, statistical consistency, and publication bias. J. Math. Psychol. 57 153–169. 10.1016/j.jmp.2013.02.003
9. Galen L. W. (2012). Does religious belief promote prosociality? A critical examination. Psychol. Bull. 138 876–906. 10.1037/a0028251
10. Gomes C. McCullough M. E. (2015). The effects of implicit religious primes on dictator game allocations: a preregistered replication experiment. J. Exp. Psychol. 10.1037/xge0000027 [Epub ahead of print].
11. Guan M. Vandekerckhove J. (2015). A Bayesian approach to mitigation of publication bias. Psychon. Bull. Rev. 10.3758/s13423-015-0868-6 [Epub ahead of print].
12. Hoeting J. A. Madigan D. Raftery A. E. Volinsky C. T. (1999). Bayesian model averaging: a tutorial. Statist. Sci. 14 382–417.
13. John L. K. Loewenstein G. Prelec D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychol. Sci. 23 524–532. 10.1177/0956797611430953
14. Matzke D. Nieuwenhuis S. van Rijn H. Slagter H. A. van der Molen M. W. Wagenmakers E.-J. (2015). The effect of horizontal eye movements on free recall: a preregistered adversarial collaboration. J. Exp. Psychol. 144 1–15. 10.1037/xge0000038
15. Morey R. D. (2013). The consistency test does not–and cannot–deliver what is advertised: a comment on Francis (2013). J. Math. Psychol. 57 180–183. 10.1016/j.jmp.2013.03.004
16. Norenzayan A. Shariff A. F. Gervais W. M. Willard A. K. McNamara R. A. Slingerland E. et al. (2014). The cultural evolution of prosocial religions. Behav. Brain Sci. 1–86. 10.1017/s0140525x14001356
17. Nosek B. A. Lakens D. (2014). Registered reports: a method to increase the credibility of published results. Soc. Psychol. 45 137–141. 10.1027/1864-9335/a000192
18. Piff P. K. Kraus M. W. Côté S. Cheng B. H. Keltner D. (2010). Having less, giving more: the influence of social class on prosocial behavior. J. Pers. Soc. Psychol. 99 771–784. 10.1037/a0020092
19. Podsakoff P. M. MacKenzie S. B. Lee J. Y. Podsakoff N. P. (2003). Common method biases in behavioral research: a critical review of the literature and recommended remedies. J. Appl. Psychol. 88 879–903. 10.1037/0021-9010.88.5.879
20. Pyysiäinen I. Hauser M. (2010). The origins of religion: evolved adaptation or by-product? Trends Cogn. Sci. 14 104–109. 10.1016/j.tics.2009.12.007
21. Schimmack U. (2012). The ironic effect of significant results on the credibility of multiple-study articles. Psychol. Methods 17 551–566. 10.1037/a0029487
22. Shariff A. F. Norenzayan A. (2007). God is watching you – priming god concepts increases prosocial behavior in an anonymous economic game. Psychol. Sci. 18 803–809. 10.1111/j.1467-9280.2007.01983.x
23. Shariff A. F. Willard A. Andersen T. Norenzayan A. (2015). Religious priming: a meta-analysis with a focus on prosociality. Pers. Soc. Psychol. Rev. 10.1177/1088868314568811 [Epub ahead of print].
24. Simons D. J. Holcombe A. O. Spellman B. A. (2014). An introduction to registered replication reports at perspectives on psychological science. Perspect. Psychol. Sci. 9 552–555. 10.1177/1745691614543974
25. Simonsohn U. (2013). It really just does not follow, comments on. J. Math. Psychol. 57 174–176. 10.1016/j.jmp.2013.03.006
26. Simonsohn U. Nelson L. D. Simmons J. P. (2014). P-curve: a key to the file-drawer. J. Exp. Psychol. 143 534–547. 10.1037/a0033242
27. Stanley T. D. (2005). Beyond publication bias. J. Econ. Surv. 19 309–345. 10.1111/j.0950-0804.2005.00250.x
28. Stanley T. D. Doucouliagos H. (2007). Identifying and correcting publication selection bias in the efficiency-wage literature: Heckman meta-regression. Econ. Series 11 32.
29. Stanley T. D. Doucouliagos H. (2014). Meta-regression approximations to reduce publication selection bias. Res. Synth. Methods 5 60–78. 10.1002/jrsm.1095
30. Van Elk M. Rutjens B. T. van der Pligt J. van Harreveld F. (2014). Priming of supernatural agent concepts and agency detection. Religion Brain Behav. 1–30. 10.1080/2153599X.2014.933444
31. Wagenmakers E. J. Wetzels R. Borsboom D. Kievit R. van der Maas H. L. J. (2015). “A skeptical eye on psi,” in Extrasensory Perception: Support, Skepticism, and Science eds May E. Marwaha S. (Santa Barbara: ABC-CLIO).