ERP Reliability Analysis (ERA) Toolbox: An open-source toolbox for analyzing the reliability of event-related brain potentials

International Journal of Psychophysiology

Volumn 111, Issue , 2017, Pages 68-79

  Clayson, Peter E ^a   Miller, Gregory A ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Dependability; ERP Reliability Analysis Toolbox; Event related potentials; Generalizability theory; Psychometrics

Indexed keywords

AVERAGING; CONCEPTUAL FRAMEWORK; COVARIANCE; EVENT RELATED POTENTIAL; EVOKED RESPONSE; HUMAN; PSYCHOMETRY; RELIABILITY; SCIENTIST; SOFTWARE; THEORETICAL MODEL; DEVICES; INFORMATION PROCESSING; PHYSIOLOGY; PROCEDURES; PSYCHOPHYSIOLOGY; REPRODUCIBILITY; STANDARDS; STATISTICAL ANALYSIS;

AUTOMATIC DATA PROCESSING; DATA INTERPRETATION, STATISTICAL; EVOKED POTENTIALS; HUMANS; PSYCHOPHYSIOLOGY; REPRODUCIBILITY OF RESULTS;

EID: 85005975930     PISSN: 01678760     EISSN: 18727697     Source Type: Journal    
DOI: 10.1016/j.ijpsycho.2016.10.012     Document Type: Article

References (42)

1. Baldwin, S.A., Larson, M.J., Clayson, P.E., The dependability of electrophysiological measurements of performance monitoring in a clinical sample: a generalizability and decision analysis of the ERN and Pe. Psychophysiology 52 (2015), 790–800, 10.1111/psyp.12401.
2. Brennan, R.L., An essay on the history and future of reliability from the perspective of replications. J. Educ. Meas. 38 (2001), 295–317, 10.1111/j.1745-3984.2001.tb01129.x.
3. Brennan, R.L., Generalizability Theory: Statistics for Social Science and Public Policy. 2001, Springer-Verlag, New York, NY.
4. Brennan, R.L., Coefficients and Indices in Generalizability Theory. CASMA Research Report No. 1 (pp. 1–48). 2003, The University of Iowa: Center for Advanced Studies in Measurement and Assessment.
5. Brennan, R.L., Generalizability theory and classical test theory. Appl. Meas. Educ. 24 (2010), 1–21, 10.1080/08957347.2011.532417.
6. Brennan, R.L., Kane, M.T., An index of dependability for mastery tests. J. Educ. Meas. 14 (1977), 277–289, 10.1111/j.1745-3984.1977.tb00045.x.
7. Carpenter, B., Gelman, A., Hoffman, M., Lee, D., Goodrich, B., Betancourt, M., … Riddell, A. (n.d.). Stan: a probabilistic programming language. J. Stat. Softw. (in press).
8. Clayson, P.E., Miller, G.A., Psychometric considerations in the measurement of event-related brain potentials: Guidelines for measurement and reporting. Int. J. Psychophysiol. 111 (2017), 57–67.
9. Clayson, P. E., & Miller, G. A. (under review). ERP reliability analysis (ERA) toolbox: an open-source toolbox for analyzing the reliability of event-related potentials.
10. Clayson, P.E., Baldwin, S.A., Larson, M.J., How does noise affect amplitude and latency measurement of event-related potentials (ERPs)? A methodological critique and simulation study. Psychophysiology 50 (2013), 174–186, 10.1111/psyp.12001.
11. Cronbach, L.J., Coefficient alpha and the internal structure of tests. Psychometrika 16 (1951), 297–334, 10.1007/BF02310555.
12. Cronbach, L.J., My current thoughts on coefficient alpha and successor procedures. Educ. Psychol. Meas. 64 (2004), 391–418, 10.1177/0013164404266386.
13. Cronbach, L.J., Gleser, G.C., Nanda, H., Rajaratnum, N., The Dependability of Behavioral Measures: Theory of Generalizability for Scores and Profiles. 1972, John Wiley, New York, NY.
14. Eriksen, B.A., Eriksen, C.W., Effects of noise letters upon the identification of a target letter in a non-search task. Percept. Psychophys. 16 (1974), 143–149.
15. Falkenstein, M., Hohnsbein, J., Hoormann, J., Banke, L., Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. Electroencephalogr. Clin. Neurophysiol. 78 (1991), 447–455.
16. Gehring, W.J., Goss, B., Coles, M.G.H., Meyer, D.E., Donchin, E., A neural system for error detection and compensation. Psychol. Sci. 4 (1993), 385–390, 10.1111/j.1467-9280.1993.tb00586.x.
17. Gelman, A., Carlin, J.B., Stern, H.E., Dunson, D.B., Vehtari, A., Rubin, D.B., Bayesian data analysis. 3rd ed, 2013, CRC Press, Boca Raton, FL.
18. Jennings, J.R., Editorial policy on analyses of variance with repeated measures. Psychophysiology 24 (1987), 474–475, 10.1111/j.1469-8986.1987.tb00320.x.
19. Kappenman, E.S., Luck, S.J., The effects of electrode impedance on data quality and statistical significance in ERP recordings. Psychophysiology 47 (2010), 888–904, 10.1111/j.1469-8986.2010.01009.x.
20. Kuder, G.F., Richardson, M.W., The theory of the estimation of test reliability. Psychometrika 2 (1937), 151–160, 10.1007/BF02288391.
21. Larson, M.J., Clayson, P.E., Baldwin, S.A., Performance monitoring following conflict: internal adjustments in cognitive control?. Neuropsychologia 50 (2012), 426–433, 10.1016/j.neuropsychologia.2011.12.021.
22. Larson, M.J., Clayson, P.E., Clawson, A., Making sense of all the conflict: a theoretical review and critique of conflict-related ERPs. Int. J. Psychophysiol. 93 (2014), 283–297, 10.1016/j.ijpsycho.2014.06.007.
23. Lau, B., MatlabProcessManager. 2016 Retrieved from https://github.com/brian-lau/MatlabProcessManager.
24. Logan, G.D., Attention, automaticity, and the ability to stop a speeded choice response. Long, J., Baddeley, A.D., (eds.) Attention and Performance IX, 1981, Erlbaum, Hillsdale, NJ, 205–222.
25. Luck, S.J., An Introduction to the Event-related Potential Technique. 2nd ed., 2014, The MIT Press, Cambridge, MA.
26. Morey, R.D., Hoekstra, R., Rouder, J.N., Lee, M.D., Wagenmakers, E.-J., The fallacy of placing confidence in confidence intervals. Psychon. Bull. Rev. 23 (2016), 103–123, 10.3758/s13423-015-0947-8.
27. Nunnally, J.C., Bernstein, I.H., Psychometric Theory. 3rd ed., 1994, McGraw-Hill, New York, NY.
28. O'Brien, R.M., Generalizability coefficients are reliability coefficients. Qual. Quant. 29 (1995), 421–428, 10.1007/BF01106066.
29. Picton, T.W., Bentin, S., Berg, P., Donchin, E., Hillyard, S.A., Johnson, R.,.. Taylor, M.J., Guidelines for using human event-related potentials to study cognition: recording standards and publication criteria. Psychophysiology 37 (2000), 127–152.
30. Shavelson, R.J., Webb, N.M., Generalizability Theory: A Primer. 1991, SAGE Publications, Inc., Thousand Oaks, CA.
31. Shavelson, R.J., Webb, N.M., Rowley, G.L., Generalizability theory. Am. Psychoanal. 44 (1989), 922–932, 10.1037/0003-066X.44.6.922.
32. Shrout, P.E., Measurement reliability and agreement in psychiatry. Stat. Methods Med. Res. 7 (1998), 301–317, 10.1191/1740774504cn057oa.
33. Shrout, P.E., Fleiss, J.L., Intraclass correlations: uses in assessing rater reliability. Psychol. Bull. 86 (1979), 420–428, 10.1037/0033-2909.86.2.420.
34. Sijtsma, K., On the use, the misuse, and the very limited usefulness of Cronbach's alpha. Psychometrika 74 (2008), 107–120, 10.1007/s11336-008-9101-0.
35. Sijtsma, K., Reliability beyond theory and into practice. Psychometrika 74 (2009), 169–173, 10.1007/s11336-008-9103-y.
36. Stroop, J.R., Studies of interference in serial verbal reactions. J. Exp. Psychol. 18 (1935), 643–662.
37. Strube, M.J., Newman, L.C., Psychometrics. Cacioppo, J.T., Tassinary, L.G., Berntson, G.G., (eds.) Handbook of Psychophysiology, 3rd ed, 2007, Cambridge University Press, New York, NY, 789–811.
38. Team, S.D., CmdStan: the command-line interface to Stan, version 2.10.0. Retrieved from http://mc-stan.org/, 2016.
39. Team, S.D., MatlabStan: the Matlab interface to Stan. Retrieved from http://mc-stan.org/, 2016.
40. Vince, M.A., The intermittency of control movements and the psychological refractory period. Br. J. Psychol. 38 (1948), 149–157, 10.1111/j.2044-8295.1948.tb01150.x.
41. Webb, N.M., Shavelson, R.J., Haertel, E.H., Reliability coefficients and generalizability theory. Handbook of Statistics, 26, 2006, 81–124.
42. Weir, J.P., Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J. Strength Cond. Res. 19 (2005), 231–240, 10.1519/15184.1.