Psychometric considerations in the measurement of event-related brain potentials: Guidelines for measurement and reporting

International Journal of Psychophysiology

Volumn 111, Issue , 2017, Pages 57-67

  Clayson, Peter E ^a   Miller, Gregory A ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Dependability; ERP reliability analysis toolbox; Event related potentials; Guidelines; Psychometrics

Indexed keywords

CALCULATION; CONSENSUS DEVELOPMENT; EVENT RELATED POTENTIAL; EVOKED RESPONSE; INFORMATION PROCESSING; INTERNAL CONSISTENCY; PIPELINE; POPULATION MODEL; PSYCHOMETRY; STATISTICAL ANALYSIS; VALIDITY; ELECTROENCEPHALOGRAPHY; HUMAN; PHYSIOLOGY; PRACTICE GUIDELINE; PSYCHOPHYSIOLOGY; STANDARDS;

ELECTROENCEPHALOGRAPHY; EVOKED POTENTIALS; GUIDELINES AS TOPIC; HUMANS; PSYCHOMETRICS; PSYCHOPHYSIOLOGY;

EID: 84994751949     PISSN: 01678760     EISSN: 18727697     Source Type: Journal    
DOI: 10.1016/j.ijpsycho.2016.09.005     Document Type: Article

References (123)

1. Adcock, R., Collier, D.A., Measurement validity: a shared standard for qualitative and quantitative research. Am. Polit. Sci. Rev. 95 (2001), 529–546, 10.1017/S0003055401003100.
2. Anastasi, A., Psychological Testing. 6th ed., 1997, Macmillan, New York.
3. Baldwin, S.A., Improving the rigor of psychophysiology research. Int. J. Psychophysiol. 111 (2017), 5–16 (in this issue).
4. Baldwin, S.A., Larson, M.J., Clayson, P.E., The dependability of electrophysiological measurements of performance monitoring in a clinical sample: a generalizability and decision analysis of the ERN and Pe. Psychophysiology 52 (2015), 790–800, 10.1111/psyp.12401.
5. Barkley, G.L., Baumgartner, C., MEG and EEG in epilepsy. J. Clin. Neurophysiol. 20 (2003), 163–178.
6. Baugh, F., Correcting effect sizes for score reliability: a reminder that measurement and substantive issues are linked inextricably. Educ. Psychol. Meas. 62 (2002), 254–263.
7. Boutros, N.N., Overall, J., Zouridakis, G., Test-retest reliability of the P50 mid-latency auditory evoked response. Psychiatry Res. 39 (1991), 181–192, 10.1016/0165-1781(91)90086-5.
8. Bramon, E., Rabe-Hesketh, S., Sham, P., Murray, R.M., Frangou, S., Meta-analysis of the P300 and P50 waveforms in schizophrenia. Schizophr. Res. 70 (2004), 315–329, 10.1016/j.schres.2004.01.004.
9. Brand, A., Bradley, M.T., The precision of effect size estimation from published psychological research: surveying confidence intervals. Psychol. Rep. 118 (2016), 154–170, 10.1177/0033294115625265.
10. Brennan, R.L., Generalizability Theory: Statistics for Social Science and Public Policy. 2001, Springer-Verlag, New York, NY.
11. Briesch, A.M., Swaminathan, H., Welsh, M., Chafouleas, S.M., Generalizability theory: a practical guide to study design, implementation, and interpretation. J. Sch. Psychol. 52 (2014), 13–35, 10.1016/j.jsp.2013.11.008.
12. Brunia, C.H., Möcks, J., Van den Berg-Lenssen, M.M., Coelho, M., Correcting ocular artifacts in the EEG: a comparison of several models. J. Psychophysiol. 3 (1989), 1–50.
13. Carp, J., The secret lives of experiments: methods reporting in the fMRI literature. NeuroImage 63 (2012), 289–300, 10.1016/j.neuroimage.2012.07.004.
14. Cassidy, S.M., Robertson, I.H., O'Connell, R.G., Retest reliability of event-related potentials: evidence from a variety of paradigms. Psychophysiology 49 (2012), 659–664, 10.1111/j.1469-8986.2011.01349.x.
15. Chapman, L.J., Chapman, J.P., Disordered Thought in Schizophrenia. 1973, Appleton-Century-Crofts, East Norwalk, CT, US.
16. Chapman, L.J., Chapman, J.P., The measurement of differential deficit. J. Psychiatr. Res., 13, 1978.
17. Chapman, L.J., Chapman, J.P., Commentary on two articles concerning generalized and specific cognitive deficits. J. Abnorm. Psychol. 110 (2001), 31–39, 10.1037//0021-843X.110.1.31.
18. Chmielewski, M., Watson, D., What is being assessed and why it matters: the impact of transient error on trait research. J. Pers. Soc. Psychol. 97 (2009), 186–202, 10.1037/a0015618.
19. Clayson, P.E., Miller, G.A., ERP Reliability Analysis (ERA) Toolbox: An Open-source Toolbox for Analyzing the Reliability of Event-related Potentials. Int. J. Psychophysiol. 111 (2017), 68–79 (in this issue).
20. Clayson, P.E., Baldwin, S.A., Larson, M.J., How does noise affect amplitude and latency measurement of event-related potentials (ERPs)? A methodological critique and simulation study. Psychophysiology 50 (2013), 174–186, 10.1111/psyp.12001.
21. Clementz, B.A., Geyer, M.A., Braff, D.L., P50 suppression among schizophrenia and normal comparison subjects: a methodological analysis. Biol. Psychiatry 41 (1997), 1035–1044, 10.1016/S0006-3223(96)00208-9.
22. Cohen, J., Cohen, P., West, S.G., Aiken, L.S., Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. 3rd ed., 2003, Erlbaum, Hillsdale, NJ.
23. Coles, M.G.H., Rugg, M.D., Rugg, M.D., Coles, M.G.H., (eds.) Event-related brain potentials: an introduction, 1995, Oxford University Press, New York, NY, 1–26.
24. Cook, E.W. III, Miller, G.A., Digital filtering: background and tutorial for psychophysiologists. Psychophysiology 29 (1992), 350–362, 10.1111/j.1469-8986.1992.tb01709.x.
25. Cronbach, L.J., My current thoughts on coefficient alpha and successor procedures. Educ. Psychol. Meas. 64 (2004), 391–418, 10.1177/0013164404266386.
26. Cronbach, L.J., Thorndike, R.L., Educational Measurement. Test Validation. 1971, 443–507.
27. Cronbach, L.J., Gleser, G.C., Nanda, H., Rajaratnum, N., The Dependability of Behavioral Measures: Theory of Generalizability for Scores and Profiles. 1972, John Wiley, New York, NY.
28. Donchin, E., Callaway, E., Cooper, E., Desmedt, R., Publication criteria for studies of evoked potentials (EP) in man. Report of the methodology committee. Desmedt, J.E., (eds.) Progress in Clinical Neurophysiology: Vol 1. Attention, Voluntary Contraction and Event-Related Cerebral Potentials, 1, 1977, Karger, Basel, Switzerland, 1–11.
29. Edgar, J.C., Miller, G.A., Digital Signal Processing. 2016, Mathable, Inc., Champaign, IL.
30. Edgar, J.C., Huang, M.X., Weisend, M.P., Sherwood, A., Miller, G.A., Adler, L.E., Cañive, J.M., Interpreting abnormality: an EEG and MEG study of P50 and the auditory paired-stimulus paradigm. Biol. Psychol. 65 (2003), 1–20, 10.1016/S0301-0511(03)00094-2.
31. Falkenstein, M., Hohnsbein, J., Hoormann, J., Banke, L., Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. Electroencephalogr. Clin. Neurophysiol. 78 (1991), 447–455.
32. Fan, X., Thompson, B., Confidence intervals for effect sizes: confidence intervals about score reliability coefficients, please: an EPM guidelines editorial. Educ. Psychol. Meas. 61 (2001), 517–531, 10.1177/0013164401614001.
33. Foti, D., Kotov, R., Hajcak, G., Psychometric considerations in using error-related brain activity as a biomarker in psychotic disorders. J. Abnorm. Psychol. 122 (2013), 520–531, 10.1037/a0032618.
34. Freedman, R., Adler, L.E., Gerhardt, G.A., Waldo, M., Baker, N., Rose, G.M.,.. Franks, R., Neurobiological studies of sensory gating in schizophrenia. Schizophr. Bull. 13 (1987), 669–678.
35. Freedman, R., Olincy, A., Ross, R.G., Waldo, M.C., Stevens, K.E., Adler, L.E., Leonard, S., The genetics of sensory gating deficits in schizophrenia. Curr. Psychiatry Rep. 5 (2003), 155–161, 10.1007/s11920-003-0032-2.
36. Gehring, W.J., Goss, B., Coles, M.G.H., Meyer, D.E., Donchin, E., A neural system for error detection and compensation. Psychol. Sci. 4 (1993), 385–390, 10.1111/j.1467-9280.1993.tb00586.x.
37. Gelman, A., Carlin, J., Beyond power calculations: assessing type S (sign) and type M (magnitude) errors. Perspect. Psychol. Sci. 9 (2014), 641–651, 10.1177/1745691614551642.
38. Gignac, G.E., Psychometrics and the measurement of emotional intelligence. Parker, A.J.D., Saklofske, H.D., Stough, C., (eds.) Assessing Emotional Intelligence: Theory, Research, and Applications, 2009, Springer US, Boston, MA, 9–40.
39. Glaser, E.M., Ruchkin, D.S., Principles of Neurobiological Signal Analysis. 1976, Academic Press, New York.
40. Golden, R.R., Meehl, P.E., Detection of the schizoid taxon with MMPI indicators. J. Abnorm. Psychol. 88 (1979), 217–233, 10.1037/0021-843X.88.3.217.
41. Gratton, G., Kramer, A.F., Coles, M.G., Donchin, E., Simulation studies of latency measures of components of the event-related brain potential. Psychophysiology 26 (1989), 233–248.
42. Hall, M.-H., Schulze, K., Rijsdijk, F., Picchioni, M., Ettinger, U., Bramon, E.,.. Sham, P., Heritability and reliability of P300, P50 and duration mismatch negativity. Behav. Genet. 36 (2006), 845–857, 10.1007/s10519-006-9091-6.
43. Handy, T.C., Event-related Potentials: A Methods Handbook. 2005, MIT press, Cambridge, MA.
44. Heinrichs, R.W., Meta-analysis and the science of schizophrenia: variant evidence or evidence of variants?. Neurosci. Biobehav. Rev. 28 (2004), 379–394, 10.1016/j.neubiorev.2004.06.003.
45. Huffmeijer, R., Bakermans-Kranenburg, M.J., Alink, L.R.A., van Ijzendoorn, M.H., Reliability of event-related potentials: the influence of number of trials and electrodes. Physiol. Behav. 130 (2010), 13–22, 10.1016/j.physbeh.2014.03.008.
46. Humphreys, L.G., Further comments on reliability and power of significance tests. Appl. Psychol. Meas. 17 (1993), 11–14, 10.1177/014662169301700102.
47. Humphreys, L.G., Drasgow, F., Some comments on the relation between reliability and statistical power. Appl. Psychol. Meas. 13 (1989), 419–425, 10.1177/014662168901300407.
48. Jerger, K., Biggins, C., Fein, G., P50 suppression is not affected by attentional manipulations. Biol. Psychiatry 31 (1992), 365–377, 10.1016/0006-3223(92)90230-W.
49. Kanyongo, G.Y., Brook, G.P., Kyei-Blankson, L., Gocmen, G., Reliability and statistical power: how measurement fallibility affects power and required sample sizes for several parametric and nonparametric statistics. J. Mod. Appl. Stat. Methods, 6, 2007, 9.
50. Kappenman, E.S., Luck, S.J., The effects of electrode impedance on data quality and statistical significance in ERP recordings. Psychophysiology 47 (2010), 888–904, 10.1111/j.1469-8986.2010.01009.x.
51. Kappenman, E.S., Luck, S.J., ERP components: the ups and downs of brainwave recordings. Luck, S.J., Kappenman, E.S., (eds.) The Oxford Handbook of Event-related Potential Components, 2012, Oxford University Press, Inc., New York, NY, 3–30.
52. Kathmann, N., Engel, R.R., Sensory gating in normals and schizophrenics: a failure to find strong P50 suppression in normals. Biol. Psychiatry 27 (1990), 1216–1226, 10.1016/0006-3223(90)90419-3.
53. Keil, A., Debener, S., Gratton, G., Junghöfer, M., Kappenman, E.S., Luck, S.J.,.. Yee, C.M., Committee report: publication guidelines and recommendations for studies using electroencephalography and magnetoencephalography. Psychophysiology 51 (2014), 1–21.
54. Kendler, K.S., “A gene for…”: the nature of gene action in psychiatric disorders. Am. J. Psychiatr. 162 (2005), 1243–1252, 10.1176/appi.ajp.162.7.1243.
55. Kozak, M.J., Cuthbert, B.N., The NIMH research domain criteria initiative: background, issues, and pragmatics. Psychophysiology 53 (2016), 286–297, 10.1111/psyp.12518.
56. Lamberti, J.S., Schwarzkopf, S.B., Boutros, N., Crilly, J.F., Martin, R., Within-session changes in sensory gating assessed by P50 evoked potentials in normal subjects. Prog. Neuro-Psychopharmacol. Biol. Psychiatry 17 (1993), 781–791, 10.1016/0278-5846(93)90060-6.
57. Lance, C.E., Butts, M.M., Michels, L.C., The sources of four commonly reported cutoff criteria: what did they really say?. Organ. Res. Methods 9 (2006), 202–220, 10.1177/1094428105284919.
58. Larson, M.J., Commitment to cutting-edge research with rigor and replication in psychophysiological science by Michael J. Larson. Int. J. Psychophysiol., 102, 2016, ix.
59. Larson, M.J., Carbine, K.A., Sample size calculations in human electrophysiology (EEG and ERP) studies: a systematic review and recommendations for increased rigor. Int. J. Psychophysiol. 111 (2017), 33–41.
60. Larson, M.J., Clayson, P.E., Clawson, A., Making sense of all the conflict: a theoretical review and critique of conflict-related ERPs. Int. J. Psychophysiol. 93 (2014), 283–297, 10.1016/j.ijpsycho.2014.06.007.
61. Laszlo, S., Ruiz-Blondet, M., Khalifian, N., Chu, F., Jin, Z., A direct comparison of active and passive amplification electrodes in the same amplifier system. J. Neurosci. Methods 235 (2014), 298–307, 10.1016/j.jneumeth.2014.05.012.
62. Lilienfeld, S.O., Cognitive neuroscience and depression: legitimate versus illegitimate reductionism and five challenges. Cogn. Ther. Res. 31 (2007), 263–272, 10.1007/s10608-007-9127-0.
63. Lu, B.Y., Edgar, J.C., Jones, A.P., Smith, A.K., Huang, M.-X., Miller, G.A., Cañive, J.M., Improved test-retest reliability of 50-ms paired-click auditory gating using magnetoencephalography source modeling. Psychophysiology 44 (2007), 86–90, 10.1111/j.1469-8986.2006.00478.x.
64. Luck, S.J., An Introduction to the Event-Related Potential Technique. 2nd ed., 2014, The MIT Press, Cambridge, MA.
65. Lykken, D.T., McGue, M., Tellegen, A., Bouchard, T.J., Emergenesis. Genetic traits that may not run in families. Am. Psychol. 47 (1992), 1565–1577, 10.1177/1066480793014009.
66. Lykken, D.T., Tellegen, A., Iacono, W.G., EEG spectra in twins: evidence for a neglected mechanism of genetic determination. Physiol. Psychol. 10 (2013), 60–65, 10.3758/BF03327008.
67. Mathewson, K.E., Harrison, T.J.L., Kizuk, S.A.D., High and dry? Comparing active dry EEG electrodes to active and passive wet electrodes. Psychophysiology, 2016 (in press).
68. Meehl, P.E., Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. J. Consult. Clin. Psychol. 46 (1978), 806–834.
69. Melinder, M.R.D., Barch, D.M., Heydebrand, G., Csernansky, J.G., Easier tasks can have better discriminating power: the case of verbal fluency. J. Abnorm. Psychol. 114 (2005), 385–391, 10.1037/0021-843X.114.3.383.
70. Messick, S., Meaning and values in test validation: the science and ethics of assessment. Educ. Res. 18 (1989), 5–11.
71. Messick, S., Validity of psychological assessment: validation of inferences from persons' responses and performances as scientific inquiry into score meaning. Am. Psychoanal. 50 (1995), 741–749.
72. Meyer, A., Riesel, A., Hajcak, G., Reliability of the ERN across multiple tasks as a function of increasing errors. Psychophysiology 50 (2013), 1220–1225, 10.1111/psyp.12132.
73. Meyer, A., Bress, J.N., Proudfit, G.H., Psychometric properties of the error-related negativity in children and adolescents. Psychophysiology 51 (2014), 602–610, 10.1111/psyp.12208.
74. Michel, C.M., Murray, M.M., Lantz, G., Gonzalez, S., Spinelli, L., Grave de Peralta, R., EEG source imaging. Clin. Neurophysiol. 115 (2004), 2195–2222, 10.1016/j.clinph.2004.06.001.
75. Miller, G.A., Presidential address: how we think about cognition, emotion, and biology in psychopathology. Psychophysiology 33 (1996), 615–628.
76. Miller, G.A., Another quasi-30 years of slow progress. Appl. Prev. Psychol. 11 (2004), 61–64, 10.1016/j.appsy.2004.02.010.
77. Miller, G.A., Mistreating psychology in the decades of the brain. Perspect. Psychol. Sci. 5 (2010), 716–743, 10.1177/1745691610388774.
78. Miller, G.A., Chapman, J.P., Misunderstanding analysis of covariance. J. Abnorm. Psychol. 110 (2001), 40–48, 10.1037/0021-843X.110.1.40.
79. Miller, G.A., Rockstroh, B., Endophenotypes in psychopathology research: where do we stand?. Annu. Rev. Clin. Psychol. 9 (2013), 177–213, 10.1146/annurev-clinpsy-050212-185540.
80. Miller, G.A., Yee, C.M., Moving psychopathology forward. Psychol. Inq. 26 (2015), 263–267, 10.1080/1047840X.2015.1037818.
81. Miller, M.B., Chapman, J.P., Chapman, L.J., Collins, J., Task difficulty and cognitive deficits in schizophrenia. J. Abnorm. Psychol. 104 (1995), 251–258, 10.1037/0021-843X.104.2.251.
82. Miller, G.A., Elbert, T., Sutton, B.P., Heller, W., Innovative clinical assessment technologies: challenges and opportunities in neuroimaging. Psychol. Assess. 19 (2007), 58–73, 10.1037/1040-3590.19.1.58.
83. Miller, G.A., Crocker, L.D., Spielberg, J.M., Infantolino, Z.P., Heller, W., Issues in localization of brain function: the case of lateralized frontal cortex in cognition, emotion, and psychopathology. Front. Integr. Neurosci., 2013, 7, 10.3389/fnint.2013.00002.
84. Miller, G.A., Clayson, P.E., Yee, C.M., Hunting genes, hunting endophenotypes. Psychophysiology 51 (2014), 1329–1330, 10.1111/psyp.12354.
85. Miller, G.A., Rockstroh, B.S., Hamilton, H.K., Yee, C.M., Psychophysiology as a core strategy in RDoC. Psychophysiology 53 (2016), 410–414, 10.1111/psyp.12581.
86. Nitschke, J.B., Miller, G.A., Cook, E.W. III, Digital filtering in EEG/ERP analysis: Some technical and methodological comparisons. Eimer, M., (eds.) Behavior Research Methods, Instruments, and Computers, vol. 30, 1998, 54–67.
87. Nunnally, J.C., Bernstein, I.H., Psychometric Theory. 3rd ed., 1994, McGraw-Hill, New York, NY.
88. Patterson, J.V., Hetrick, W.P., Boutros, N.N., Jin, Y., Sandman, C., Stern, H.,.. Bunney, W.E., P50 sensory gating ratios in schizophrenics and controls: a review and data analysis. Psychiatry Res. 158 (2008), 226–247, 10.1016/j.psychres.2007.02.009.
89. Pedhazur, E.J., Schmelkin, L.P., Measurement, Design, and Analysis: An Integrate Approach. 1991, Erlbaum, Hillsdale, NJ.
90. Perry, N.W., Signal versus noise in the evoked potential. Science, 153, 1966, 1022.
91. Picton, T.W., Bentin, S., Berg, P., Donchin, E., Hillyard, S.A., Johnson, R.,.. Taylor, M.J., Guidelines for using human event-related potentials to study cognition: recording standards and publication criteria. Psychophysiology 37 (2000), 127–152.
92. Pontifex, M.B., Scudder, M.R., Brown, M.L., O'Leary, K.C., Wu, C.-T., Themanson, J.R., Hillman, C.H., On the number of trials necessary for stabilization of error-related brain activity across the life span. Psychophysiology 47 (2010), 767–773, 10.1111/j.1469-8986.2010.00974.x.
93. Popov, T., Rockstroh, B., Weisz, N., Elbert, T., Miller, G.A., Adjusting brain dynamics in schizophrenia by means of perceptual and cognitive training. PLoS One, 7, 2012, e39051, 10.1371/journal.pone.0039051.
94. Rentzsch, J., Jockers-Scherübl, M.C., Boutros, N.N., Gallinat, J., Test-retest reliability of P50, N100 and P200 auditory sensory gating in healthy subjects. Int. J. Psychophysiol. 67 (2008), 81–90, 10.1016/j.ijpsycho.2007.10.006.
95. Riesel, A., Weinberg, A., Endrass, T., Meyer, A., Hajcak, G., The ERN is the ERN is the ERN? Convergent validity of error-related brain activity across different tasks. Biol. Psychol. 93 (2013), 377–385, 10.1016/j.biopsycho.2013.04.007.
96. Sattler, J.M., Assessment of Children: Cognitive Applications. 4th ed., 2001, Jerome M. Sattler Publisher, Inc., San Diego, CA.
97. Schimmel, H., The (±) reference: accuracy of estimated mean components in average response studies. Science 157 (1967), 92–94.
98. Shavelson, R.J., Webb, N.M., Rowley, G.L., Generalizability theory. Am. Psychoanal. 44 (1989), 922–932, 10.1037/0003-066X.44.6.922.
99. Sherman, E.M.S., Brooks, B.L., Iverson, G.L., Slick, D.J., Strauss, E., Reliability and Validity in Neuropsychology. 2011, Springer US, Boston, MA.
100. Smith, G.T., McCarthy, D.M., Methodological considerations in the refinement of clinical assessment instruments. Psychol. Assess. 7 (1995), 300–308, 10.1037/1040-3590.7.3.300.
101. Smith, D.A., Boutros, N.N., Schwarzkopf, S.B., Reliability of P50 auditory event-related potential indices of sensory gating. Psychophysiology 31 (1994), 495–502, 10.1111/j.1469-8986.1994.tb01053.x.
102. Strauss, M.E., Demonstrating specific cognitive deficits: a psychometric perspective. J. Abnorm. Psychol. 110 (2001), 6–14, 10.1037//0021-843X.110.1.6.
103. Strauss, M.E., Smith, G.T., Construct validity: advances in theory and methodology. Annu. Rev. Clin. Psychol. 5 (2009), 1–25, 10.1146/annurev.clinpsy.032408.153639.
104. Strauss, E., Sherman, E.M.S., Spreen, O., A Compendium of Neuropsychological Tests. 3rd ed., 2006, Oxford University Press, New York.
105. Thompson, B., Guidelines for authors reporting score reliability estimates. Thompson, B., (eds.) Score Reliability: Contemporary Thinking on Reliability Issues, 2003, Sage Publications, Inc., Thousand Oaks, CA, 91–101.
106. Thompson, B., Snyder, P.A., Statistical significance and reliability analyses in recent journal of counseling & development research articles. J. Couns. Dev. 76 (1998), 436–441, 10.1002/j.1556-6676.1998.tb02702.x.
107. Tversky, A., Kahneman, D., Belief in the law of small numbers. Psychol. Bull. 76 (1971), 105–110.
108. Vacha-Haase, T., Reliability generalization: exploring variance in measurement error affecting score reliability across studies. Educ. Psychol. Meas. 58 (1998), 6–20, 10.1177/0013164498058001002.
109. Vacha-Haase, T., Ness, C., Nilsson, J., Reetz, D., Practices regarding reporting of reliability coefficients: a review of three journals. J. Exp. Educ. 67 (1999), 335–341, 10.1080/00220979909598487.
110. Vacha-Haase, T., Kogan, L.R., Thompson, B., Sample compositions and variabilities in published studies versus those in test manuals: validity of score reliability inductions. Educ. Psychol. Meas. 60 (2000), 509–522, 10.1177/00131640021970682.
111. Watson, D., Stability versus change, dependability versus error: issues in the assessment of personality over time. J. Res. Pers. 38 (2004), 319–350, 10.1016/j.jrp.2004.03.001.
112. Wendel, K., Väisänen, O., Malmivuo, J., Gencer, N.G., Vanrumste, B., Durka, P.,.. Grave de Peralta Menendez, R., EEG/MEG source imaging: methods, challenges, and open issues. Comput. Intell. Neurosci., 2009, 2009, 10.1155/2009/656092 (656092-656012).
113. Whittington, D., How well do researchers report their measures? An evaluation of measurement in published educational research. Educ. Psychol. Meas. 58 (1998), 21–37, 10.1177/0013164498058001003.
114. Widmann, A., Schröger, E., Filter effects and filter artifacts in the analysis of electrophysiological data. Front. Psychol., 3, 2012, 233, 10.3389/fpsyg.2012.00233.
115. Widmann, A., Schröger, E., Maess, B., Digital filter design for electrophysiological data–a practical approach. J. Neurosci. Methods 250 (2015), 34–46, 10.1016/j.jneumeth.2014.08.002.
116. Wilkerson, L., The APA Task force on statistical inference, Statistical methods in Psychology Journals: Guidelines and Expectations. 54, 1999, 594–604, 10.1037/0003-066X.54.8.594.
117. Williams, R.H., Zimmerman, D.W., Statistical power analysis and reliability of measurement. J. Gen. Psychol. 116 (1989), 359–369, 10.1080/00221309.1989.9921123.
118. Williams, R.H., Zimmerman, D.W., Zumbo, B.D., Impact of measurement error on statistical power: review of an old paradox. J. Exp. Educ. 63 (1995), 363–370.
119. Williams, T.J., Nuechterlein, K.H., Subotnik, K.L., Yee, C.M., Distinct neural generators of sensory gating in schizophrenia. Psychophysiology 48 (2011), 470–478, 10.1111/j.1469-8986.2010.01119.x.
120. Yee, C.M., Javitt, D.C., Miller, G.A., Replacing DSM categorical analyses with dimensional analyses in psychiatry research: the research domain criteria initiative. JAMA Psychiatry 72 (2015), 1159–1160, 10.1001/jamapsychiatry.2015.1900.
121. Yeung, N., Bogacz, R., Holroyd, C.B., Cohen, J.D., Detection of synchronized oscillations in the electroencephalogram: an evaluation of methods. Psychophysiology 41 (2004), 822–832, 10.1111/j.0048-5772.2004.00239.x.
122. Yeung, N., Bogacz, R., Holroyd, C.B., Nieuwenhuis, S., Cohen, J.D., Theta phase resetting and the error-related negativity. Psychophysiology 44 (2007), 39–49, 10.1111/j.1469-8986.2006.00482.x.
123. Zinbarg, R.E., Suzuki, S., Uliaszek, A.A., Lewis, A.R., Biased parameter estimates and inflated type I error rates in analysis of covariance (and analysis of partial variance) arising from unreliability: alternatives and remedial strategies. J. Abnorm. Psychol. 119 (2010), 307–319, 10.1037/a0017552.