Sample size calculations in human electrophysiology (EEG and ERP) studies: A systematic review and recommendations for increased rigor

International Journal of Psychophysiology

Volumn 111, Issue , 2017, Pages 33-41

  Larson, Michael J ^a   Carbine, Kaylie A ^a  

^a BRIGHAM YOUNG UNIVERSITY   (United States)

Author keywords

EEG; Electroencephalogram; ERP; Event related potential; Rigor; Sample size

Indexed keywords

ARTICLE; CALCULATION; EFFECT SIZE; ELECTROENCEPHALOGRAM; ELECTROENCEPHALOGRAPHY; EVENT RELATED POTENTIAL; HUMAN; INTERRATER RELIABILITY; POWER ANALYSIS; RELIABILITY; SAMPLE SIZE; SYSTEMATIC REVIEW; VARIANCE; ELECTROPHYSIOLOGY; EVOKED RESPONSE; METHODOLOGY; PHYSIOLOGY; PROCEDURES; STANDARDS;

ELECTROENCEPHALOGRAPHY; ELECTROPHYSIOLOGY; EVOKED POTENTIALS; HUMANS; RESEARCH DESIGN; SAMPLE SIZE;

EID: 84977098360     PISSN: 01678760     EISSN: 18727697     Source Type: Journal    
DOI: 10.1016/j.ijpsycho.2016.06.015     Document Type: Article

References (61)

1. Abdulatif, M., Mukhtar, A., Obayah, G., Pitfalls in reporting sample size calculation in randomized controlled trials published in leading anaesthesia journals: a systematic review. Br. J. Anaesth. 115 (2015), 699–707.
2. Aberson, C.L., Applied Power Analysis for the Behavioral Sciences. 2010, Routledge, New York, NY.
3. Altman, D.G., Practical Statistics for Medical Research. 1991, Chapman and Hall, London.
4. Ameis, S.H., Catani, M., Altered white matter connectivity as a neural substrate for social impairment in Autism Spectrum Disorder. Cortex 62 (2015), 158–181.
5. Baldwin, S.A., Improving the rigor of psychophysiology research. Int. J. Psychophysiol. 111 (2017), 5–16.
6. Barch, D.M., Yarkoni, T., Introduction to the special issue on reliability and replication in cognitive and affective neuroscience research. Cogn. Affect. Behav. Neurosci. 13 (2013), 687–689.
7. Bezeau, S., Graves, R., Statistical power and effect sizes of clinical neuropsychology research. J. Clin. Exp. Neuropsychol. 23 (2001), 399–406.
8. Button, K.S., Ioannidis, J.P.A., Mokrysz, C., Nosek, B.A., Flint, J., Robinson, E.S.J., Munafò, M.R., Power failure: why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 14 (2013), 365–376.
9. Cashen, L.H., Geiger, S.W., Statistical power and the testing of null hypotheses: a review of contemporary management research and recommendations for future studies. Organ. Res. Methods 7 (2004), 151–167.
10. Chan, A.W., Altman, D.G., Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. Br. Med. J. 330 (2005), 753–758.
11. Charles, P., Giraudeau, B., Dechartres, A., Baron, G., Ravaud, P., Reporting of sample size calculation in randomised controlled trials: review. Br. Med. J. 338 (2009), 1256–1259.
12. Chiu, P.H., Deldin, P.J., Neural evidence for enhanced error detection in major depressive disorder. Am. J. Psychiatry 164 (2007), 608–616.
13. Clayson, P.E., Baldwin, S.A., Larson, M.J., How does noise affect amplitude and latency measurement of event-related potentials (ERPs)? A methodological critique and simulation study. Psychophysiology 50 (2013), 174–186.
14. Cohen, J., Statistical Power Analysis for the Behavioral Sciences. second ed., 1988, Erlbaum, Hillsdale, NJ.
15. Cohen, M.X., Rigor and replication in time-frequency analyses of cognitive electrophysiology data. Int. J. Psychophysiol. 111 (2017), 80–87.
16. Elms, A.C., The crisis of confidence in social psychology. Am. Psychol. 30 (1975), 967–976.
17. Faul, F., Erdfelder, E., Lang, A.G., Buchner, A., G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav. Res. Methods 39 (2007), 175–191.
18. Georgiadi, E., Liotti, M., Nixon, N.L., Liddle, P.F., Electrophysiological evidence for abnormal error monitoring in recurrent major depressive disorder. Psychophysiology 48 (2011), 1192–1202.
19. Guo, Y., Logan, H.L., Glueck, D.H., Muller, K.E., Selecting a sample size for studies with repeated measures. BMC Med. Res. Methodol. 13 (2013), 1–8.
20. Guo, Q., Thabane, L., Hall, G., McKinnon, M., Goeree, R., Pullenayegum, E., A systematic review of the reporting of sample size calculations and corresponding data components in observational functional magnetic resonance imaging studies. NeuroImage 86 (2014), 172–181.
21. Hoenig, J.M., Heisey, D.M., The abuse of power: the pervasive fallacy of power calculations for data analysis. Am. Stat., 55, 2001, 19.
22. Holmes, A.J., Pizzagalli, D.A., Spatiotemporal dynamics of error processing dysfunctions in major depressive disorder. Arch. Gen. Psychiatry 65 (2008), 179–188.
23. Holmes, A.J., Pizzagalli, D.A., Effects of task-relevant incentives on the electrophysiological correlates of error processing in major depressive disorder. Cogn. Affect. Behav. Neurosci. 10 (2010), 118–128.
24. Ioannidis, J.P., Why most published research findings are false. PLoS Med., 2, 2005, e124.
25. Ioannidis, J.P., Why most discovered true associations are inflated. Epidemiology 19 (2008), 640–648.
26. Ioannidis, J.P., How to make more published research true. PLoS Med., 11, 2014, e1001747.
27. John, L.K., Loewenstein, G., Prelec, D., Measuring the prevalence of questionable research practices with incentives for truth telling. Psychol. Sci. 23 (2012), 524–532.
28. Keil, A., Debener, S., Gratton, G., Junghofer, M., Kappenman, E.S., Luck, S.J., Luu, P., Miller, G.A., Yee, C.M., Committee report: publication guidelines and recommendations for studies using electroencephalography and magnetoencephalography. Psychophysiology 51 (2014), 1–21.
29. Kim, J., Seo, B.S., How to calculate sample size and why. Clin. Orthop. Surg. 5 (2013), 235–242.
30. Kirby, A., Gebski, V., Keech, A.C., Determining the sample size in a clinical trial. Med. J. Aust. 177 (2002), 256–257.
31. Landis, J.R., Koch, G.G., The measurement of observer agreement for categorical data. Biometrics 33 (1977), 159–174.
32. Larson, M.J., Commitment to cutting-edge research with rigor and replication in psychophysiological science. Int. J. Psychophysiol., 102, 2016, ix.
33. Latif, L.A., Amadera, J.E.D., Pimentel, D., Pimentel, T., Fregni, F., Sample size calculation in physical medicine and rehabilitation: a systematic review of reporting, characteristics, and results in randomized controlled trials. Arch. Phys. Med. Rehabil. 92 (2011), 306–315.
34. Lenth, R.V., Some practical guidelines for effective sample size determination. Am. Stat. 55 (2001), 187–193.
35. Levine, M., Ensom, M.H.H., Post hoc power analysis: an idea whose time has passed?. Pharmacotherapy 21 (2001), 405–409.
36. Luck, S.J., Gaspelin, N. How to get statistically significant effects in any ERP experiment (and why you shouldn't). Psychophysiology (in press).
37. Maggard, M.A., O'Connell, J.B., Liu, J.H., Etzioni, D.A., Ko, C.Y., Sample size calculations in surgery: are they done correctly?. Surgery 134 (2003), 275–279.
38. Maxwell, S.E., The persistence of underpowered studies in psychological research: causes, consequences, and remedies. Psychol. Methods 9 (2004), 147–163.
39. Moran, T.P., Schroder, H.S., Kneip, C., Moser, J.S., Meta-analysis and psychophysiology: a tutorial using depression and action-monitoring event-related potentials. Int. J. Psychophysiol. 111 (2017), 17–32.
40. Nakagawa, S., Foster, T.M., The case against retrospective statistical power analyses with an introduction to power analysis. Acta Ethol. 7 (2004), 103–108.
41. Nosek, B.A., Spies, J.R., Motyl, M., Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspect. Psychol. Sci. 7 (2012), 615–631.
42. O'Keefe, D.J., Post hoc power, observed power, a priori power, retrospective power, prospective power, achieved power: sorting out appropriate uses of statistical power analyses. Commun. Methods Measures 1 (2007), 291–299.
43. Olvet, D.M., Klein, D., Hajcak, G., Depression symptom severity and error-related brain activity. Psychiatry Res. 179 (2010), 30–37.
44. Onwuegbuzie, A.J., Leech, N.L., Post hoc power: a concept whose time has come. Underst. Stat. 3 (2004), 201–230.
45. Open Science Collaboration. Estimating the reproducibility of psychological science. Science, 349, 2015.
46. Picton, T.W., Bentin, S., Berg, P., Donchin, E., Hillyard, S.A., Johnson, R. Jr., Miller, G.A., Ritter, W., Ruchkin, D.S., Rugg, M.D., Taylor, M.J., Guidelines for using human event-related potentials to study cognition: recording standards and publication criteria. Psychophysiology 37 (2000), 127–152.
47. Ruchsow, M., Herrnberger, B., Wiesend, C., Gron, G., Spitzer, M., Kiefer, M., The effect of erroneous responses on response monitoring in patients with major depressive disorder: a study with event-related potentials. Psychophysiology 41 (2004), 833–840.
48. Ruchsow, M., Herrnberger, B., Beschoner, P., Gron, G., Spitzer, M., Kiefer, M., Error processing in major depressive disorder: evidence from event-related potentials. J. Psychiatr. Res. 40 (2006), 37–46.
49. Sawyer, A.G., Ball, A.D., Statistical power and effect size in marketing research. J. Mark. Res. 18 (1981), 275–290.
50. Schrijvers, D., De Bruijn, E.R., Maas, Y.J., De Grave, C., Sabbe, B.G., Hulstijn, W., Action monitoring in major depressive disorder with psychomotor retardation. Cortex 44 (2008), 569–579.
51. Schrijvers, D., De Bruijn, E.R., Maas, Y.J., Vancoillie, P., Hulstijn, W., Sabbe, B.G., Action monitoring and depressive symptom reduction in major depressive disorder. Int. J. Psychophysiol. 71 (2009), 218–224.
52. Schulz, K.F., Grimes, D.A., Sample size calculations in randomised trials: mandatory and mystical. Lancet 365 (2005), 1348–1353.
53. Schulz, K.F., Altman, D.G., Moher, D., CONSORT, CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. Open Med. 4 (2010), 60–68.
54. Simmons, J.P., Nelson, L.D., Simonsohn, U., False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol. Sci. 22 (2011), 1359–1366.
55. Sterne, J.A.C., Smith, G.D., Sifting the evidence–what's wrong with significance tests?. Br. Med. J. 332 (2001), 226–231.
56. Vaidyanathan, U., Nelson, L., Patrick, C.J., Clarifying domains of internalizing psychopathology using neurophysiology. Psychol. Med. 42 (2012), 447–459.
57. von Elm, E., Altman, D.G., Egger, M., Pocock, S.J., Gotzsche, P.C., Vandenbrouck, J.P., The Strengthening of Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann. Intern. Med. 147 (2007), 573–577.
58. Wang, H., Chow, S.C., Chen, M., A Bayesian approach on sample size calculation for comparing means. J. Biopharm. Stat. 15 (2005), 799–807.
59. Whitley, E., Ball, J., Statistics review 4: sample size calculations. Crit. Care 6 (2002), 335–341.
60. Wilkinson, L., Statistical methods in psychology journals: guidelines and explanations. Am. Psychol. 54 (1999), 594–604.
61. Zhong, B., How to calculate sample size in a randomized controlled trial. J. Thorac. Dis. 1 (2009), 51–54.