Reliable gains? Evidence for substantially underpowered designs in studies of working memory training transfer to fluid intelligence

Frontiers in Psychology

Volumn 5, Issue , 2015, Pages

  Bogg, Tim ^a   Lasecki, Leanne ^a  

^a WAYNE STATE UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

cognitive training intervention; fluid intelligence; meta analysis; small study effects; statistical power; transfer effects

Indexed keywords

EID: 84940976573     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2014.01589     Document Type: Review

References (29)

1. Au J. Sheehan E. Tsai N. Duncan G. J. Buschkuehl M. Jaeggi S. M. (2014). Improving fluid intelligence with training on working memory: a meta-analysis. Psychon. Bull. Rev. 10.3758/s13423-014-0699-x [Epub ahead of print].
2. Baddeley A. D. Logie R. (1999). “Working memory: the multicomponent model,” in Models of Working Memory: Mechanisms of Active Maintenance and Executive Control, eds Miyake A. Shah P. (Cambridge, MA: Cambridge University Press), 28–62.
3. Button K. S. Ioannidis J. P. A. Mokrysz C. Nosek B. Flint J. Robinson E. S. J. et al. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 14 365–376. 10.1038/nrn3475
4. Carter E. C. McCullough M. E. (2014). Publication bias and the limited strength model of self-control: has the evidence for ego depletion been overestimated? Front. Psychol. 5:823. 10.3389/fpsyg.2014.00823
5. Cowan N. (1999). “An embedded-process model of working memory,” in Models of Working Memory: Mechanisms of Active Maintenance and Executive Control, eds Miyake A. Shah P. (Cambridge, MA: Cambridge University Press), 62–101. 10.1017/CBO9781139174909.006
6. Egger M. Sterne J. A. C. (2005). “Regression methods to detect publication and other bias in meta-analysis,” in Publication Bias and Meta-Analysis: Prevention, Assessments and Adjustments, eds Rothstein H. Sutton A. Borenstein M. (Chichester: John Wiley and Sons), 99–109.
7. Hedges L. V. Pigott T. D. (2001). The power of statistical tests in meta-analysis. Psychol. Methods 6 203–217. 10.1037/1082-989X.6.3.203
8. Higgins J. P. Thompson S. G. Deeks J. J. Altman D. G. (2003). Measuring inconsistency in meta-analyses. BMJ 327 557–560. 10.1136/bmj.327.7414.557
9. Ioannidis J. P. (2005). Why most published research findings are false. PLoS Med. 2:e124. 10.1371/journal.pmed.0020124
10. Ioannidis J. P. A. (2008). Why most discovered true associations are inflated. Epidemiology 19 640–648. 10.1097/EDE.0b013e31818131e7
11. Ioannidis J. P. Trikalinos T. A. (2005). Early extreme contradictory estimates may appear in published research: the Proteus phenomenon in molecular genetics research and randomized trials. J. Clin. Epidemiol. 58 543–549. 10.1016/j.jclinepi.2004.10.019
12. Ioannidis J. P. Trikalinos T. A. (2007). An exploratory test for an excess of significant findings. Clin. Trials 4 245–253. 10.1177/1740774507079441
13. Jaeggi S. M. Buschkuehl M. Jonides J. Perrig W. J. (2008). Improving fluid intelligence with training on working memory. Proc. Natl. Acad. Sci. U.S.A. 105 6829–6833. 10.1073/pnas.0801268105
14. Jaeggi S. M. Buschkuehl M. Shah P. Jonides J. (2014). The role of individual differences in cognitive training and transfer. Mem. Cogn. 42 464–480. 10.3758/s13421-013-0364-z
15. Moreno S. G. Sutton A. J. Turner E. H. Abrams K. R. Cooper N. J. Palmer T. M. et al. (2009). Novel methods to deal with publication biases: secondary analysis of antidepressant trials in the FDA trial registry database and related journal publications. BMJ 339:b2981. 10.1136/bmj.b2981
16. Morrison A. B. Chein J. M. (2011). Does working memory training work? The promise and challenges of enhancing cognition by training memory. Pychon. Bull. Rev. 18 46–60. 10.3758/s13423-010-0034-0
17. Nisbett R. E. Aronson J. Blair C. Dickens W. Flynn J. Halpern D. F. et al. (2012). Intelligence: new findings and theoretical developments. Am. Psychol. 67 130–159. 10.1037/a0026699
18. Pereira T. V. Ioannidis J. P. A. (2011). Statistically significant meta-analyses of clinical trials have modest credibility and inflated effects. J. Clin. Epidemiol. 64 1060–1069. 10.1016/j.jclinepi.2010.12.012
19. Peters J. L. Sutton A. J. Jones D. R. Abrams K. R. Rushton L. (2008). Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. J. Clin. Epidemiol. 61 991–996. 10.1016/j.jclinepi.2007.11.010
20. Rücker G. Schwarzer G. Carpenter J. R. Binder H. Schumacher M. (2011). Treatment-effect estimates adjusted for small-study effects via a limit meta-analysis. Biostatistics 12 122–142. 10.1093/biostatistics/kxq046
21. Schmidt F. L. (2013). “Meta-analysis,” in Comprehensive Handbook of Psychology: Research Methods in Psychology, Vol. 2 eds Shinka J. Velicer W. (Hoboken, NJ: Wiley), 571–594.
22. Stanley T. D. (2005). Beyond publication bias. J. Econ. Surv. 19 309–345. 10.1111/j.0950-0804.2005.00250.x
23. Stanley T. D. (2008). Meta-regression methods for detecting and estimating empirical effects in the presence of publication selection. Oxf. Bull. Econ. Stat. 70 103–127. 10.1111/j.1468-0084.2007.00487.x
24. Stanley T. D. Doucouliagos C. (2007). Identifying and correcting publication selection bias in the efficiency-wage literature: Heckman meta-regression. Econ. Ser. 11.
25. Stanley T. D. Doucouliagos C. (2013). Meta-regression approximations to reduce publication selection bias. Res. Synth. Methods 5 60–78. 10.1002/jrsm.1095
26. Thompson T. W. Waskom M. L. Garel K. L. Cardenas-Iniguez C. Reynolds G. O. Winter R. et al. (2013). Failure of working memory training to enhance cognition or intelligence. PLoS ONE 8:e63614. 10.1371/journal.pone.0063614
27. Turner E. H. Matthews A. M. Linardatos E. Tell R. A. Rosenthal R. (2008). Selective publication of antidepressant trials and its influence on apparent efficacy. N. Engl. J. Med. 358 252–260. 10.1056/NEJMsa065779
28. Unsworth N. Engle R. W. (2007). The nature of individual differences in working memory capacity: active maintenance in primary memory and controlled search from secondary memory. Psychol. Rev. 114 104–132. 10.1037/0033-295X.114.1.104
29. Valentine J. C. Pigott T. D. Rothstein H. R. (2010). How many studies do you need? A primer on statistical power for meta-analysis. J. Educ. Behav. Stat. 35 215–247. 10.3102/10769986093469