Correction: What meta-analyses reveal about the replicability of psychological research [Psychological Bulletin, 144, 12, (2018), (1325-1346)] DOI: 10.1037/bul0000169;What meta-analyses reveal about the replicability of psychological research

Psychological Bulletin

Volumn 144, Issue 12, 2018, Pages 1325-1346

  Stanley, T D ^a   Carter, Evan C ^b   Doucouliagos, Hristos ^c  

^a DEAKIN UNIVERSITY   (Australia)

^b U S ARMY RESEARCH LABORATORY   (United States)

^c DEAKIN UNIVERSITY   (Australia)

Author keywords

Bias; Heterogeneity; Meta analysis; Power; Replicability

Indexed keywords

BEHAVIORAL RESEARCH; HUMAN; META ANALYSIS; META ANALYSIS (TOPIC); METHODOLOGY; PSYCHOLOGY; PUBLISHING; REPRODUCIBILITY; STATISTICAL ANALYSIS;

BEHAVIORAL RESEARCH; DATA INTERPRETATION, STATISTICAL; HUMANS; META-ANALYSIS AS TOPIC; PSYCHOLOGY; PUBLICATION BIAS; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN;

EID: 85054858346     PISSN: 00332909     EISSN: None     Source Type: Journal    
DOI: 10.1037/bul0000203     Document Type: Erratum

References (142)

1. Abramowitz, M., & Stegun, I. A. (Eds.). (1964). Handbook of mathematical functions with formulas, graphs and mathematical tables. Washington, DC: U.S. Department of Commerce
2. American Psychological Association. (2010). Manual of the american psychological association (6th ed.). Washington, DC: Author
3. Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J., Fiedler, K., . . . Wicherts, J. M. (2013). Recommendations for increasing replicability in psychology. European Journal of Personality, 27, 108-119. http://dx.doi.org/10.1002/per.1919
4. Baas, M., Nijstad, B. A., Boot, N. C., & De Dreu, C. K. W. (2016). Mad genius revisited: Vulnerability to psychopathology, biobehavioral approach-avoidance, and creativity. Psychological Bulletin, 142, 668-692. http://dx.doi.org/10.1037/bul0000049
5. Baker, M. (2015, August 27). Over half of psychology studies fail reproducibility test. Nature: International Weekly Journal of Science. Advance online publication. http://dx.doi.org/10.1038/nature.2015.18248
6. Bakker, M., Hartgerink, C. H., Wicherts, J. M., & van der Maas, H. L. (2016). Researchers' intuitions about power in psychological research. Psychological Science, 27, 1069-1077. http://dx.doi.org/10.1177/0956 797616647519
7. Bakker, M., van Dijk, A., & Wicherts, J. M. (2012). The rules of the game called psychological science. Perspectives on Psychological Science, 7, 543-554. http://dx.doi.org/10.1177/1745691612459060
8. Balliet, D., Li, N. P., Macfarlan, S. J., & Van Vugt, M. (2011). Sex differences in cooperation: A meta-analytic review of social dilemmas. Psychological Bulletin, 137, 881-909. http://dx.doi.org/10.1037/ a0025354
9. Balliet, D., Mulder, L. B., & Van Lange, P. A. M. (2011). Reward, punishment, and cooperation: A meta-analysis. Psychological Bulletin, 137, 594-615. http://dx.doi.org/10.1037/a0023489
10. Balliet, D., Wu, J., & De Dreu, C. K. W. (2014). Ingroup favoritism in cooperation: A meta-analysis. Psychological Bulletin, 140, 1556-1581. http://dx.doi.org/10.1037/a0037737
11. Bohannon, J. (2015). Many psychology papers fail replication test. Science, 349, 910-911. http://dx.doi.org/10.1126/science.349.6251.910
12. Braver, S. L., Thoemmes, F. J., & Rosenthal, R. (2014). Continuously cumulating meta-analysis and replicability. Perspectives on Psychological Science, 9, 333-342. http://dx.doi.org/10.1177/1745691614529796
13. Brewin, C. R. (2014). Episodic memory, perceptual memory, and their interaction: Foundations for a theory of posttraumatic stress disorder. Psychological Bulletin, 140, 69-97. http://dx.doi.org/10.1037/a0033722
14. Bruns, S. B., & Ioannidis, J. P. A. (2016). p-curve and p-hacking in observational research. PLoS ONE, 11, e0149144. http://dx.doi.org/10 .1371/journal.pone.0149144
15. Button, K. S., Ioannidis, J. P., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S., & Munafò, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14, 365-376. http://dx.doi.org/10.1038/nrn3475
16. Byron, K., & Khazanchi, S. (2012). Rewards and creative performance: A meta-analytic test of theoretically derived hypotheses. Psychological Bulletin, 138, 809-830. http://dx.doi.org/10.1037/a0027652
17. Cerasoli, C. P., Nicklin, J. M., & Ford, M. T. (2014). Intrinsic motivation and extrinsic incentives jointly predict performance: A 40-year metaanalysis. Psychological Bulletin, 140, 980-1008. http://dx.doi.org/10 .1037/a0035661
18. Chaplin, T. M., & Aldao, A. (2013). Gender differences in emotion expression in children: A meta-analytic review. Psychological Bulletin, 139, 735-765. http://dx.doi.org/10.1037/a0030737
19. Cohen, J. (1962). The statistical power of abnormal-social psychological research: A review. The Journal of Abnormal and Social Psychology, 65, 145-153. http://dx.doi.org/10.1037/h0045186
20. Cohen, J. (1965). Some statistical issues in psychological research. In B. B. Wolman (Ed.), Handbook of clinical psychology (pp. 95-121). New York, NY: McGraw-Hill
21. Cohen, J. (1977). Statistical power analysis for the behavioral sciences (2nd ed.). New York, NY: Academic Press
22. Cooper, H. M., & Hedges, L. V. (Eds.). (1994). Handbook of research synthesis. New York, NY: Russell Sage Foundation
23. Defoe, I. N., Dubas, J. S., Figner, B., & van Aken, M. A. (2015). A meta-analysis on age differences in risky decision making: Adolescents versus children and adults. Psychological Bulletin, 141, 48-84. http:// dx.doi.org/10.1037/a0038088
24. Degner, J., & Dalege, J. (2013). The apple does not fall far from the tree, or does it? A meta-analysis of parent-child similarity in intergroup attitudes. Psychological Bulletin, 139, 1270-1304. http://dx.doi.org/10 .1037/a0031436
25. Dreber, A., Pfeiffer, T., Almenberg, J., Isaksson, S., Wilson, B., Chen, Y., . . . Johannesson, M. (2015). Using prediction markets to estimate the reproducibility of scientific research. Proceedings of the National Academy of Sciences of the United States of America, 112, 15343-15347. http://dx.doi.org/10.1073/pnas.1516179112
26. Eastwick, P. W., Luchies, L. B., Finkel, E. J., & Hunt, L. L. (2014). The predictive validity of ideal partner preferences: A review and metaanalysis. Psychological Bulletin, 140, 623-665. http://dx.doi.org/10 .1037/a0032432
27. Eerland, A., Sherrill, A. M., Magliano, J. P., Zwaan, R. A., Arnal, J. D., Aucoin, P. A., . . . Prenoveau, J. M. (2016). Registered replication report: Hart & Albarracín (2011). Perspectives on Psychological Science, 11, 158-171. http://dx.doi.org/10.1177/1745691615605826
28. Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 629-634. http://dx.doi.org/10.1136/bmj.315.7109.629
29. Else-Quest, N. M., Higgins, A., Allison, C., & Morton, L. C. (2012). Gender differences in self-conscious emotional experience: A metaanalysis. Psychological Bulletin, 138, 947-981. http://dx.doi.org/10 .1037/a0027930
30. Erica, C. Y., Sprenger, A. M., Thomas, R. P., & Dougherty, M. R. (2014). When decision heuristics and science collide. Psychonomic Bulletin & Review, 21, 268-282. http://dx.doi.org/10.3758/s13423-013-0495-z
31. Fabrigar, L. R., & Wegener, D. T. (2016). Conceptualizing and evaluating the replication of research results. Journal of Experimental Social Psychology, 66, 68-80. http://dx.doi.org/10.1016/j.jesp.2015.07.009
32. Fairbairn, C. E., & Sayette, M. A. (2014). A social-attributional analysis of alcohol response. Psychological Bulletin, 140, 1361-1382. http://dx .doi.org/10.1037/a0037563
33. Fanelli, D. (2010). "Positive" results increase down the hierarchy of the sciences. PLoS ONE, 5, e10068. http://dx.doi.org/10.1371/journal.pone .0010068
34. Fanelli, D., Costas, R., & Ioannidis, J. P. (2017). Meta-assessment of bias in science. Proceedings of the National Academy of Sciences of the United States of America, 201618569, 114, 3714-3719. http://dx.doi .org/10.1073/pnas.1618569114
35. Fiedler, K., & Schwarz, N. (2016). Questionable research practices revisited. Social Psychological and Personality Science, 7, 45-52. http://dx .doi.org/10.1177/1948550615612150
36. Fischer, P., Krueger, J. I., Greitemeyer, T., Vogrincic, C., Kastenmüller, A., Frey, D., . . . Kainbacher, M. (2011). The bystander-effect: A meta-analytic review on bystander intervention in dangerous and nondangerous emergencies. Psychological Bulletin, 137, 517-537. http://dx .doi.org/10.1037/a0023304
37. Fox, M. C., Ericsson, K. A., & Best, R. (2011). Do procedures for verbal reporting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods. Psychological Bulletin, 137, 316-344. http://dx.doi.org/10.1037/a0021663
38. Fox, N. A., Bakermans-Kranenburg, M. J., Yoo, K. H., Bowman, L. C., Cannon, E. N., Vanderwert, R. E., . . . van IJzendoorn, M. H. (2016). Assessing human mirror activity with EEG mu rhythm: A meta-analysis. Psychological Bulletin, 142, 291-313. http://dx.doi.org/10.1037/ bul0000031
39. Fraley, R. C., & Vazire, S. (2014). The N-pact factor: Evaluating the quality of empirical journals with respect to sample size and statistical power. PLoS ONE, 9, e109019. http://dx.doi.org/10.1371/journal.pone .0109019
40. Franco, A., Malhotra, N., & Simonovits, G. (2016). Underreporting in psychology experiments: Evidence from a study registry. Social Psychological & Personality Science, 7, 8-12. http://dx.doi.org/10.1177/ 1948550615598377
41. Freund, P. A., & Kasten, N. (2012). How smart do you think you are? A meta-analysis on the validity of self-estimates of cognitive ability. Psychological Bulletin, 138, 296-321. http://dx.doi.org/10.1037/a0026556
42. Gelman, A., & Carlin, J. (2014). Beyond power calculations: Assessing Type S (sign) and Type M (magnitude) errors. Perspectives on Psychological Science, 9, 641-651. http://dx.doi.org/10.1177/174569 1614551642
43. Grijalva, E., Newman, D. A., Tay, L., Donnellan, M. B., Harms, P. D., Robins, R. W., & Yan, T. (2015). Gender differences in narcissism: A meta-analytic review. Psychological Bulletin, 141, 261-310. http://dx .doi.org/10.1037/a0038231
44. Haedt-Matt, A. A., & Keel, P. K. (2011). Revisiting the affect regulation model of binge eating: A meta-analysis of studies using ecological momentary assessment. Psychological Bulletin, 137, 660-681. http://dx .doi.org/10.1037/a0023660
45. Hagger, M. S., Chatzisarantis, N. L. D., Alberts, H., Anggono, C. O., Batailler, C., Birt, A. R., . . . Zwienenberg, M. (2016). A multi-lab preregistered replication of the ego-depletion effect. Perspectives on Psychological Science, 11, 546-573. http://dx.doi.org/10.1177/174 5691616652873
46. Harkin, B., Webb, T. L., Chang, B. P., Prestwich, A., Conner, M., Kellar, I., . . . Sheeran, P. (2016). Does monitoring goal progress promote goal attainment? A meta-analysis of the experimental evidence. Psychological Bulletin, 142, 198-229. http://dx.doi.org/10.1037/bul0000025
47. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press
48. Henmi, M., & Copas, J. B. (2010). Confidence intervals for random effects meta-analysis and robustness to publication bias. Statistics in Medicine, 29, 2969-2983. http://dx.doi.org/10.1002/sim.4029
49. Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21, 1539-1558. http://dx.doi.org/ 10.1002/sim.1186
50. Hoenig, J. M., & Heisey, D. M. (2001). The abuse of power: The pervasive fallacy of power calculations for data analysis. The American Statistician, 55, 19-24. http://dx.doi.org/10.1198/000313001300339897
51. Houben, M., Van Den Noortgate, W., & Kuppens, P. (2015). The relation between short-term emotion dynamics and psychological well-being: A meta-analysis. Psychological Bulletin, 141, 901-930. http://dx.doi.org/ 10.1037/a0038822
52. Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2, e124. http://dx.doi.org/10.1371/journal.pmed .0020124
53. Ioannidis, J. P. A., Hozo, I., & Djulbegovic, B. (2013). Optimal type I and type II error pairs when the available sample size is fixed. Journal of Clinical Epidemiology, 66, 903-910.e2. http://dx.doi.org/10.1016/j .jclinepi.2013.03.002
54. Ioannidis, J. P. A., Stanley, T. D., & Doucouliagos, C. (2017). The power of bias in economics research. Economic Journal, 127, F236-F265. http://dx.doi.org/10.1111/ecoj.12461
55. John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23, 524-532. http://dx.doi.org/10.1177/ 0956797611430953
56. Johnsen, T. J., & Friborg, O. (2015). The effects of cognitive behavioral therapy as an anti-depressive treatment is falling: A meta-analysis. Psychological Bulletin, 141, 747-768. http://dx.doi.org/10.1037/ bul0000015
57. Karlin, B., Zinger, J. F., & Ford, R. (2015). The effects of feedback on energy conservation: A meta-analysis. Psychological Bulletin, 141, 1205-1227. http://dx.doi.org/10.1037/a0039650
58. Kim, S., Thibodeau, R., & Jorgensen, R. S. (2011). Shame, guilt, and depressive symptoms: A meta-analytic review. Psychological Bulletin, 137, 68-96. http://dx.doi.org/10.1037/a0021466
59. Klahr, A. M., & Burt, S. A. (2014). Elucidating the etiology of individual differences in parenting: A meta-analysis of behavioral genetic research. Psychological Bulletin, 140, 544-586. http://dx.doi.org/10.1037/ a0034205
60. Klein, R. A., Vianello, M., Hasselman, F., Alper, S., Aveyard, M., Axt, J. R., & Nosek, B. A. (2015). Many Labs 2: Investigating variation in replicability across sample and setting. Retrieved from http:// projectimplicit.net/nosek/ML2protocol.pdf
61. Koenig, A. M., Eagly, A. H., Mitchell, A. A., & Ristikari, T. (2011). Are leader stereotypes masculine? A meta-analysis of three research paradigms. Psychological Bulletin, 137, 616-642. http://dx.doi.org/10.1037/ a0023557
62. Kredlow, M. A., Unger, L. D., & Otto, M. W. (2016). Harnessing reconsolidation to weaken fear and appetitive memories: A metaanalysis of post-retrieval extinction effects. Psychological Bulletin, 142, 314-336. http://dx.doi.org/10.1037/bul0000034
63. Kühberger, A., Fritz, A., & Scherndl, T. (2014). Publication bias in psychology: A diagnosis based on the correlation between effect size and sample size. PLoS ONE, 9(9), e105825. http://dx.doi.org/10.1371/ journal.pone.0105825
64. Kuykendall, L., Tay, L., & Ng, V. (2015). Leisure engagement and subjective well-being: A meta-analysis. Psychological Bulletin, 141, 364-403. http://dx.doi.org/10.1037/a0038508
65. Landau, M. J., Kay, A. C., & Whitson, J. A. (2015). Compensatory control and the appeal of a structured world. Psychological Bulletin, 141, 694-722. http://dx.doi.org/10.1037/a0038703
66. LeBel, E. P., Borsboom, D., Giner-Sorolla, R., Hasselman, F., Peters, K. R., Ratliff, K. A., & Smith, C. T. (2013). PsychDisclosure.org: Grassroots support for reforming reporting standards in psychology. Perspectives on Psychological Science, 8, 424-432. http://dx.doi.org/ 10.1177/1745691613491437
67. LeBel, E. P., Vanpaemel, W., McCarthy, R. J., Earp, B. D., & Elson, M. (2017). A unified framework to quantify the trustworthiness of empirical research. Advances in Methods and Practices in Psychological Science. Retrieved from https://osf.io/preprints/psyarxiv/uwmr88/23/2017
68. Lee, E.-S., Park, T.-Y., & Koo, B. (2015). Identifying organizational identification as a basis for attitudes and behaviors: A meta-analytic review. Psychological Bulletin, 141, 1049-1080. http://dx.doi.org/10 .1037/bul0000012
69. Lindsay, D. S. (2015). Replication in psychological science. Psychological Science, 26, 1827-1832. http://dx.doi.org/10.1177/0956797615616374
70. Lui, P. P. (2015). Intergenerational cultural conflict, mental health, and educational outcomes among Asian and Latino/a Americans: Qualitative and meta-analytic review. Psychological Bulletin, 141, 404-446. http:// dx.doi.org/10.1037/a0038449
71. Lull, R. B., & Bushman, B. J. (2015). Do sex and violence sell? A meta-analytic review of the effects of sexual and violent media and ad content on memory, attitudes, and buying intentions. Psychological Bulletin, 141, 1022-1048. http://dx.doi.org/10.1037/bul0000018
72. Makel, M. C., Plucker, J. A., & Hegarty, B. (2012). Replications in psychology research: How often do they really occur? Perspectives on Psychological Science, 7, 537-542. http://dx.doi.org/10.1177/ 1745691612460688
73. Maxwell, S. E. (2004). The persistence of underpowered studies in psychological research: Causes, consequences, and remedies. Psychological Methods, 9, 147-163. http://dx.doi.org/10.1037/1082-989X.9.2.147
74. Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015). Is psychology suffering from a replication crisis? What does "failure to replicate" really mean? American Psychologist, 70, 487-498. http://dx.doi.org/10 .1037/a0039400
75. Mazei, J., Hüffmeier, J., Freund, P. A., Stuhlmacher, A. F., Bilke, L., & Hertel, G. (2015). A meta-analysis on gender differences in negotiation outcomes and their moderators. Psychological Bulletin, 141, 85-104. http://dx.doi.org/10.1037/a0038184
76. McShane, B. B., & Böckenholt, U. (2016). Planning sample sizes when effect sizes are uncertain: The power-calibrated effect size approach. Psychological Methods, 21, 47-60. http://dx.doi.org/10.1037/met 0000036
77. McShane, B. B., Böckenholt, U., & Hansen, K. T. (2016). Adjusting for publication bias in meta-analysis: An evaluation of selection methods and some cautionary notes. Perspectives on Psychological Science, 11, 730-749. http://dx.doi.org/10.1177/1745691616662243
78. Melby-Lervåg, M., & Lervåg, A. (2014). Reading comprehension and its underlying components in second-language learners: A meta-analysis of studies comparing first-and second-language learners. Psychological Bulletin, 140, 409-433. http://dx.doi.org/10.1037/a0033890
79. Melby-Lervåg, M., Lyster, S. A., & Hulme, C. (2012). Phonological skills and their role in learning to read: A meta-analytic review. Psychological Bulletin, 138, 322-352. http://dx.doi.org/10.1037/a0026744
80. Mendelson, J. L., Gates, J. A., & Lerner, M. D. (2016). Friendship in school-age boys with autism spectrum disorders: A meta-analytic summary and developmental, process-based model. Psychological Bulletin, 142, 601-622. http://dx.doi.org/10.1037/bul0000041
81. Mol, S. E., & Bus, A. G. (2011). To read or not to read: A meta-analysis of print exposure from infancy to early adulthood. Psychological Bulletin, 137, 267-296. http://dx.doi.org/10.1037/a0021890
82. Moreno, S. G., Sutton, A. J., Ades, A. E., Stanley, T. D., Abrams, K. R., Peters, J. L., & Cooper, N. J. (2009). Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study. BMC Medical Research Methodology, 9, 2. http://dx.doi .org/10.1186/1471-2288-9-2
83. North, M. S., & Fiske, S. T. (2015). Modern attitudes toward older adults in the aging world: A cross-cultural meta-analysis. Psychological Bulletin, 141, 993-1021. http://dx.doi.org/10.1037/a0039469
84. Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716-aac4716. http://dx .doi.org/10.1126/science.aac4716
85. Orquin, J. L., & Kurzban, R. (2016). A meta-analysis of blood glucose effects on human decision making. Psychological Bulletin, 142, 546-567. http://dx.doi.org/10.1037/bul0000035
86. Ottaviani, C., Thayer, J. F., Verkuil, B., Lonigro, A., Medea, B., Couyoumdjian, A., & Brosschot, J. F. (2016). Physiological concomitants of perseverative cognition: A systematic review and meta-analysis. Psychological Bulletin, 142, 231-259. http://dx.doi.org/10.1037/bul 0000036
87. Pahlke, E., Hyde, J. S., & Allison, C. M. (2014). The effects of single-sex compared with coeducational schooling on students' performance and attitudes: A meta-analysis. Psychological Bulletin, 140, 1042-1072. http://dx.doi.org/10.1037/a0035740
88. Pan, S. C., & Rickard, T. C. (2015). Sleep and motor learning: Is there room for consolidation? Psychological Bulletin, 141, 812-834. http:// dx.doi.org/10.1037/bul0000009
89. Pashler, H., & Harris, C. R. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7, 531-536. http://dx.doi.org/10.1177/1745691612463401
90. Pashler, H., & Wagenmakers, E. J. (2012). Editors' introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science, 7, 528-530. http:// dx.doi.org/10.1177/1745691612465253
91. Patil, P., & Leek, J. T. (2015). Reporting of 36% of studies replicate in the media. Retrieved from https://github.com/jtleek/replication_paper/blob/ gh-pages/in_the_media.md
92. Patil, P., Peng, R. D., & Leek, J. T. (2016). What should researchers expect when they replicate studies? A statistical view of replicability in psychological science. Perspectives on Psychological Science, 11, 539-544. http://dx.doi.org/10.1177/1745691616646366
93. Phillips, W. J., Fletcher, J. M., Marks, A. D. G., & Hine, D. W. (2016). Thinking styles and decision making: A meta-analysis. Psychological Bulletin, 142, 260-290. http://dx.doi.org/10.1037/bul0000027
94. Pigott, T. (2012). Advances in meta-analysis. New York, NY: Springer. http://dx.doi.org/10.1007/978-1-4614-2278-5
95. Pool, E., Brosch, T., Delplanque, S., & Sander, D. (2016). Attentional bias for positive emotional stimuli: A meta-analytic investigation. Psychological Bulletin, 142, 79-106. http://dx.doi.org/10.1037/bul0000026
96. Poole, C., & Greenland, S. (1999). Random-effects meta-analyses are not always conservative. American Journal of Epidemiology, 150, 469-475. http://dx.doi.org/10.1093/oxfordjournals.aje.a010035
97. Popper, K. (1959). The logic of scientific discovery. New York, NY: Basic Books
98. Psychonomic Society. (2012). New statistical guidelines for journals of the psychonomic society. Retrieved from http://www.psychonomic.org/page/ statisticalguideline
99. Randall, J. G., Oswald, F. L., & Beier, M. E. (2014). Mind-wandering, cognition, and performance: A theory-driven meta-analysis of attention regulation. Psychological Bulletin, 140, 1411-1431. http://dx.doi.org/10 .1037/a0037428
100. Richard, F. D., Bond, C. F., & Stokes-Zoota, J. J. (2003). One hundred years of social psychology quantitatively described. Review of General Psychology, 7, 331-363. http://dx.doi.org/10.1037/1089-2680.7.4.331
101. Rossi, J. S. (1990). Statistical power of psychological research: What have we gained in 20 years? Journal of Consulting and Clinical Psychology, 58, 646-656. http://dx.doi.org/10.1037/0022-006X.58.5.646
102. Sambrook, T. D., & Goslin, J. (2015). A neural reward prediction error revealed by a meta-analysis of ERPs using great grand averages. Psychological Bulletin, 141, 213-235. http://dx.doi.org/10.1037/bul 0000006
103. Scargle, J. D. (2000). Publication bias: The "File-Drawer" problem in scientific inference. Journal of Scientific Exploration, 14, 91-106
104. Schmidt, F. L., & Hunter, J. E. (2015). Methods of meta-analysis: Correcting error and bias in research findings (3rd ed.). Los Angeles, CA: Sage. http://dx.doi.org/10.4135/9781483398105
105. Schmidt, F. L., & Oh, I.-S. (2016). The crisis of confidence in research findings in psychology: Is lack of replication the real problem? Or something else? Archives of Scientific Psychology, 4, 32-37. http://dx .doi.org/10.1037/arc0000029
106. Sedlmeier, P., Eberth, J., Schwarz, M., Zimmermann, D., Haarig, F., Jaeger, S., & Kunze, S. (2012). The psychological effects of meditation: A meta-analysis. Psychological Bulletin, 138, 1139-1171. http://dx.doi .org/10.1037/a0028168
107. Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105, 309-316. http://dx.doi.org/10.1037/0033-2909.105.2.309
108. Sheeran, P., Harris, P. R., & Epton, T. (2014). Does heightening risk appraisals change people's intentions and behavior? A meta-analysis of experimental studies. Psychological Bulletin, 140, 511-543. http://dx .doi.org/10.1037/a0033065
109. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359-1366. http://dx.doi.org/10.1177/0956797611417632
110. Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). P-curve: A key to the file-drawer. Journal of Experimental Psychology: General, 143, 534-547. http://dx.doi.org/10.1037/a0033242
111. Smith, S. F., & Lilienfeld, S. O. (2015). The response modulation hypothesis of psychopathy: A meta-analytic and narrative analysis. Psychological Bulletin, 141, 1145-1177. http://dx.doi.org/10.1037/ bul0000024
112. Soderberg, C. K., Callahan, S. P., Kochersberger, A. O., Amit, E., & Ledgerwood, A. (2015). The effects of psychological distance on abstraction: Two meta-analyses. Psychological Bulletin, 141, 525-548. http://dx.doi.org/10.1037/bul0000005
113. Stanley, T. D. (2005). Beyond publication bias. Journal of Economic Surveys, 19, 309-347
114. Stanley, T. D. (2008). Meta-regression methods for detecting and estimating empirical effects in the presence of publication selection. Oxford Bulletin of Economics and Statistics, 70, 103-127
115. Stanley, T. D. (2017). Limitations of PET-PEESE and other meta-analysis methods. Social Psychological and Personality Science, 8, 581-591. http://dx.doi.org/10.1177/1948550617693062
116. Stanley, T. D., & Doucouliagos, H. (2012). Meta-regression analysis in economics and business. Oxford, United Kingdom: Routledge
117. Stanley, T. D., & Doucouliagos, H. (2014). Meta-regression approximations to reduce publication selection bias. Research Synthesis Methods, 5, 60-78. http://dx.doi.org/10.1002/jrsm.1095
118. Stanley, T. D., & Doucouliagos, H. (2015). Neither fixed nor random: Weighted least squares meta-analysis. Statistics in Medicine, 34, 2116-2127. http://dx.doi.org/10.1002/sim.6481
119. Stanley, T. D., & Doucouliagos, H. (2017). Neither fixed nor random: Weighted least squares meta-regression. Research Synthesis Methods, 8, 19-42. http://dx.doi.org/10.1002/jrsm.1211
120. Stanley, T. D., Doucouliagos, H., & Ioannidis, J. P. (2017). Finding the power to reduce publication bias. Statistics in Medicine, 36, 1580-1598
121. Stanley, T. D., Doucouliagos, H., & Jarrell, S. B. (2008). Meta-regression analysis as the socio-economics of economics research. Journal of Socio-Economics, 37, 276-292. http://dx.doi.org/10.1016/j.socec.2006 .12.030
122. Stanley, D. J., & Spence, J. R. (2014). Expectations for replications: Are yours realistic? Perspectives on Psychological Science, 9, 305-318. http://dx.doi.org/10.1177/1745691614528518
123. Sterling, T. D. (1959). Publication decisions and their possible effects on inferences drawn from tests of significance: Or vice versa. Journal of the American Statistical Association, 54, 30-34
124. Sterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995). Publication decisions revisited: The effect of the outcome of statistical tests on the decision to publish and vice versa. The American Statistician, 49, 108-112
125. Sutton, A. J., Song, F., Gilbody, S. M., & Abrams, K. R. (2000). Modelling publication bias in meta-analysis: A review. Statistical Methods in Medical Research, 9, 421-445. http://dx.doi.org/10.1177/0962280 20000900503
126. Tannenbaum, M. B., Hepler, J., Zimmerman, R. S., Saul, L., Jacobs, S., Wilson, K., & Albarracín, D. (2015). Appealing to fear: A meta-analysis of fear appeal effectiveness and theories. Psychological Bulletin, 141, 1178-1204. http://dx.doi.org/10.1037/a0039729
127. Toosi, N. R., Babbitt, L. G., Ambady, N., & Sommers, S. R. (2012). Dyadic interracial interactions: A meta-analysis. Psychological Bulletin, 138, 1-27. http://dx.doi.org/10.1037/a0025767
128. Tressoldi, P. E., & Giofré, D. (2015). The pervasive avoidance of prospective statistical power: Major consequences and practical solutions. Frontiers in Psychology, 6, 726. http://dx.doi.org/10.3389/fpsyg.2015.00726
129. Vachon, D. D., Lynam, D. R., & Johnson, J. A. (2014). The (non)relation between empathy and aggression: Surprising results from a metaanalysis. Psychological Bulletin, 140, 751-773. http://dx.doi.org/10 .1037/a0035236
130. van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2016). Conducting meta-analyses based on p values: Reservations and recommendations for applying p-uniform and p-curve. Perspectives on Psychological Science, 11, 713-729. http://dx.doi.org/10.1177/17456 91616650874
131. Van Bavel, J. J., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. (2016). Contextual sensitivity in scientific reproducibility. PNAS Proceedings of the National Academy of Sciences of the United States of America, 113, 6454-6459. http://dx.doi.org/10.1073/pnas.1521897113
132. van Erp, S., Verhagen, J., Grasman, R. P. P. P., & Wagenmakers, E.-J. (2017). Estimates of between-study heterogeneity for 705 meta-analyses reported in psychological bulletin from 1990-2013. Journal of Open Psychology Data, 5, 4. http://dx.doi.org/10.5334/jopd.33
133. Verhage, M. L., Schuengel, C., Madigan, S., Fearon, R. M. P., Oosterman, M., Cassibba, R., . . . van IJzendoorn, M. H. (2016). Narrowing the transmission gap: A synthesis of three decades of research on intergenerational transmission of attachment. Psychological Bulletin, 142, 337-366. http://dx.doi.org/10.1037/bul0000038
134. von Stumm, S., & Ackerman, P. L. (2013). Investment and intellect: A review and meta-analysis. Psychological Bulletin, 139, 841-869. http:// dx.doi.org/10.1037/a0030746
135. Voyer, D., & Voyer, S. D. (2014). Gender differences in scholastic achievement: A meta-analysis. Psychological Bulletin, 140, 1174-1204. http://dx.doi.org/10.1037/a0036620
136. Vukasović, T., & Bratko, D. (2015). Heritability of personality: A metaanalysis of behavior genetic studies. Psychological Bulletin, 141, 769-785. http://dx.doi.org/10.1037/bul0000017
137. Wanberg, C. R., Kanfer, R., Hamann, D. J., & Zhang, Z. (2016). Age and reemployment success after job loss: An integrative model and metaanalysis. Psychological Bulletin, 142, 400-426. http://dx.doi.org/10 .1037/bul0000019
138. Weingarten, E., Chen, Q., McAdams, M., Yi, J., Hepler, J., & Albarracín, D. (2016). From primed concepts to action: A meta-analysis of the behavioral effects of incidentally presented words. Psychological Bulletin, 142, 472-497. http://dx.doi.org/10.1037/bul0000030
139. Wicherts, J. M., Veldkamp, C. L., Augusteijn, H. E., Bakker, M., van Aert, R. C., & van Assen, M. A. (2016). Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking. Frontiers in Psychology, 7, 1832. http://dx.doi.org/10 .3389/fpsyg.2016.01832
140. Williams, M. J., & Tiedens, L. Z. (2016). The subtle suspension of backlash: A meta-analysis of penalties for women's implicit and explicit dominance behavior. Psychological Bulletin, 142, 165-197. http://dx.doi .org/10.1037/bul0000039
141. Winer, E. S., & Salem, T. (2016). Reward devaluation: Dot-probe metaanalytic evidence of avoidance of positive information in depressed persons. Psychological Bulletin, 142, 18-78. http://dx.doi.org/10.1037/ bul0000022
142. Yuan, K., & Maxwell, S. (2005). On the post hoc power in testing mean differences. Journal of Educational and Behavioral Statistics, 30, 141-167. http://dx.doi.org/10.3102/10769986030002141