Reliable gains? Evidence for substantially underpowered designs in studies of working memory training transfer to fluid intelligence

Frontiers in Psychology

Volumn 6, Issue JAN, 2015, Pages

  Bogg, Tim ^a   Lasecki, Leanne ^a  

^a WAYNE STATE UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

Cognitive training intervention; Fluid intelligence; Meta analysis; Small study effects; Statistical power; Transfer effects

Indexed keywords

EID: 84926640970     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2015.00589     Document Type: Article

References (29)

1. Au, J., Sheehan, E., Tsai, N., Duncan, G. J., Buschkuehl, M., and Jaeggi, S. M. (2014). Improving fluid intelligence with training on working memory: a meta-analysis. Psychon. Bull. Rev. doi: 10.3758/s13423-014-0699-x [Epub ahead of print].
2. Baddeley, A. D., and Logie, R. (1999). "Working memory: the multicomponent model," in Models of Working Memory: Mechanisms of Active Maintenance and Executive Control, eds A. Miyake and P. Shah (Cambridge, MA: Cambridge University Press), 28-62.
3. Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B., Flint, J., Robinson, E. S. J., et al. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 14, 365-376. doi: 10.1038/nrn3475.
4. Carter, E. C., and McCullough, M. E. (2014). Publication bias and the limited strength model of self-control: has the evidence for ego depletion been overestimated? Front. Psychol. 5:823. doi: 10.3389/fpsyg.2014.00823.
5. Cowan, N. (1999). "An embedded-process model of working memory," in Models of Working Memory: Mechanisms of Active Maintenance and Executive Control, eds A. Miyake and P. Shah (Cambridge, MA: Cambridge University Press), 62-101. doi: 10.1017/CBO9781139174909.006.
6. Egger, M., and Sterne, J. A. C. (2005). "Regression methods to detect publication and other bias in meta-analysis," in Publication Bias and Meta-Analysis: Prevention, Assessments and Adjustments, eds H. Rothstein, A. Sutton, and M. Borenstein (Chichester: John Wiley and Sons), 99-109.
7. Hedges, L. V., and Pigott, T. D. (2001). The power of statistical tests in meta-analysis. Psychol. Methods 6, 203-217. doi: 10.1037/1082-989X.6.3.203.
8. Higgins, J. P., Thompson, S. G., Deeks, J. J., and Altman, D. G. (2003). Measuring inconsistency in meta-analyses. BMJ 327, 557-560. doi: 10.1136/bmj.327.7414.557.
9. Ioannidis, J. P. (2005). Why most published research findings are false. PLoS Med. 2:e124. doi: 10.1371/journal.pmed.0020124.
10. Ioannidis, J. P. A. (2008). Why most discovered true associations are inflated. Epidemiology 19, 640-648. doi: 10.1097/EDE.0b013e31818131e7.
11. Ioannidis, J. P., and Trikalinos, T. A. (2005). Early extreme contradictory estimates may appear in published research: the Proteus phenomenon in molecular genetics research and randomized trials. J. Clin. Epidemiol. 58, 543-549. doi: 10.1016/j.jclinepi.2004.10.019.
12. Ioannidis, J. P., and Trikalinos, T. A. (2007). An exploratory test for an excess of significant findings. Clin. Trials 4, 245-253. doi: 10.1177/1740774507079441.
13. Jaeggi, S. M., Buschkuehl, M., Jonides, J., and Perrig, W. J. (2008). Improving fluid intelligence with training on working memory. Proc. Natl. Acad. Sci. U.S.A. 105, 6829-6833. doi: 10.1073/pnas.0801268105.
14. Jaeggi, S. M., Buschkuehl, M., Shah, P., and Jonides, J. (2014). The role of individual differences in cognitive training and transfer. Mem. Cogn. 42, 464-480. doi: 10.3758/s13421-013-0364-z.
15. Moreno, S. G., Sutton, A. J., Turner, E. H., Abrams, K. R., Cooper, N. J., Palmer, T. M., et al. (2009). Novel methods to deal with publication biases: secondary analysis of antidepressant trials in the FDA trial registry database and related journal publications. BMJ 339:b2981. doi: 10.1136/bmj.b2981.
16. Morrison, A. B., and Chein, J. M. (2011). Does working memory training work? The promise and challenges of enhancing cognition by training memory. Pychon. Bull. Rev. 18, 46-60. doi: 10.3758/s13423-010-0034-0.
17. Nisbett, R. E., Aronson, J., Blair, C., Dickens, W., Flynn, J., Halpern, D. F., et al. (2012). Intelligence: new findings and theoretical developments. Am. Psychol. 67, 130-159. doi: 10.1037/a0026699.
18. Pereira, T. V., and Ioannidis, J. P. A. (2011). Statistically significant meta-analyses of clinical trials have modest credibility and inflated effects. J. Clin. Epidemiol. 64, 1060-1069. doi: 10.1016/j.jclinepi.2010.12.012.
19. Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., and Rushton, L. (2008). Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. J. Clin. Epidemiol. 61, 991-996. doi: 10.1016/j. jclinepi.2007.11.010.
20. Rücker, G., Schwarzer, G., Carpenter, J. R., Binder, H., and Schumacher, M. (2011). Treatment-effect estimates adjusted for small-study effects via a limit meta-analysis. Biostatistics 12, 122-142. doi: 10.1093/biostatistics/kxq046.
21. Schmidt, F. L. (2013). "Meta-analysis," in Comprehensive Handbook of Psychology: Research Methods in Psychology, Vol. 2, eds J. Shinka and W. Velicer (Hoboken, NJ: Wiley), 571-594.
22. Stanley, T. D. (2005). Beyond publication bias. J. Econ. Surv. 19, 309-345. doi: 10.1111/j.0950-0804.2005.00250.x.
23. Stanley, T. D. (2008). Meta-regression methods for detecting and estimating empirical effects in the presence of publication selection. Oxf. Bull. Econ. Stat. 70, 103-127. doi: 10.1111/j.1468-0084.2007.00487.x.
24. Stanley, T. D., and Doucouliagos, C. (2007). Identifying and correcting publication selection bias in the efficiency-wage literature: Heckman meta-regression. Econ. Ser. 11.
25. Stanley, T. D., and Doucouliagos, C. (2013). Meta-regression approximations to reduce publication selection bias. Res. Synth. Methods 5, 60-78. doi: 10.1002/jrsm.1095.
26. Thompson, T. W., Waskom, M. L., Garel, K. L., Cardenas-Iniguez, C., Reynolds, G. O., Winter, R., et al. (2013). Failure of working memory training to enhance cognition or intelligence. PLoS ONE 8:e63614. doi: 10.1371/journal.pone.0063614.
27. Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., and Rosenthal, R. (2008). Selective publication of antidepressant trials and its influence on apparent efficacy. N. Engl. J. Med. 358, 252-260. doi: 10.1056/NEJMsa065779.
28. Unsworth, N., and Engle, R. W. (2007). The nature of individual differences in working memory capacity: active maintenance in primary memory and controlled search from secondary memory. Psychol. Rev. 114, 104-132. doi: 10.1037/0033-295X.114.1.104.
29. Valentine, J. C., Pigott, T. D., and Rothstein, H. R. (2010). How many studies do you need? A primer on statistical power for meta-analysis. J. Educ. Behav. Stat. 35, 215-247. doi: 10.3102/10769986093469.