Standards for talking and thinking about validity

Psychological Methods

Volumn 18, Issue 3, 2013, Pages 301-319

  Newton, Paul E ^a   Shaw, Stuart D ^b  

^a UNIVERSITY OF LONDON   (United Kingdom)

^b Cambridge International Examinations   (United Kingdom)

Author keywords

Assessment; Evaluation; Quality; Test; Validation; Validity

Indexed keywords

ARTICLE; HUMAN; METHODOLOGY; NOMENCLATURE; PSYCHOMETRY; REPRODUCIBILITY; STATISTICS;

HUMANS; PSYCHOMETRICS; REPRODUCIBILITY OF RESULTS; STATISTICS AS TOPIC; TERMINOLOGY AS TOPIC;

EID: 84883775614     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0032969     Document Type: Article

References (150)

1. Allen, M. J. (2004). Assessing academic programs in higher education. Bolton, MA: Anker.
2. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1985). Standards for educational and psychological testing. Washington, DC: Author.
3. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: Author.
4. American Psychological Association. (1952). Technical recommendations for psychological tests and diagnostic techniques: Preliminary proposal. American Psychologist, 7, 461-475. doi:10.1037/h0056631
5. American Psychological Association, American Educational Research Association, & National Council on Measurement in Education. (1966). Standards for educational and psychological tests and manuals. Washington, DC: Author.
6. American Psychological Association, American Educational Research Association, & National Council on Measurement in Education. (1974). Standards for educational and psychological tests. Washington, DC: Author.
7. American Psychological Association, American Educational Research Association, & National Council on Measurements Used in Education. (1954). Technical recommendations for psychological tests and diagnostic techniques. Psychological Bulletin, 51(2, pt. 2), 1-38. doi: 10.1037/h0053479
8. Angoff, W. H. (1988). Validity: An evolving concept. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 19-32). Hillsdale, NJ: Erlbaum.
9. Austin, J. (1995). The province of jurisprudence determined (W. E. Rumble, Ed.). Cambridge, England: Cambridge University Press. doi: 10.1017/CBO9780511521546 (Original work published 1832)
10. Bechtoldt, H. P. (1959). Construct validity: A critique. American Psychologist, 14, 619-629. doi:10.1037/h0040359
11. Bemis, S. E. (1968). Occupational validity of the General Aptitude Test Battery. Journal of Applied Psychology, 52, 240-244. doi:10.1037/h0025733
12. Bennett, R. E. (2012). Consequences that cannot be avoided: A response to Paul Newton. Measurement: Interdisciplinary Research and Perspectives, 10, 30-32. doi:10.1080/15366367.2012.686865
13. Boehm, V. R. (1972). Negro-White differences in validity of employment and training selection procedures. Journal of Applied Psychology, 56, 33-39. doi:10.1037/h0032130
14. Borsboom, D. (2012). Whose consensus is it anyway? Scientific versus legalistic conceptions of validity. Measurement: Interdisciplinary Research and Perspectives, 10, 38-41. doi:10.1080/15366367.2012.681971
15. Borsboom, D., Cramer, A. O. J., Keivit, R. A., Scholten, A. Z., & Franic, S. (2009). The end of construct validity. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 135-170). Charlotte, NC: Information Age.
16. Borsboom, D., & Mellenbergh, G. J. (2007). Test validity in cognitive assessment. In J. P. Leighton & M. J. Gierl (Eds.), Cognitive diagnostic assessment for education: Theory and applications (pp. 85-115). New York, NY: Cambridge University Press. doi:10.1017/CBO9780511611186.004
17. Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept of validity. Psychological Review, 111, 1061-1071. doi:10.1037/0033-295X.111.4.1061
18. Bracht, G. H., & Glass, G. V. (1968). The external validity of experiments. American Educational Research Journal, 5, 437-474. doi:10.3102/00028312005004437
19. Brennan, R. L. (1998). Misconceptions at the intersection of measurement theory and practice. Educational Measurement: Issues and Practice, 17, 5-9. doi:10.1111/j.1745-3992.1998.tb00615.x
20. Briggs, D. C. (2004). Comment: Making an argument for design validity before interpretive validity. Measurement: Interdisciplinary Research and Perspectives, 2, 171-174. doi:10.1207/s15366359mea0203_2
21. Brookhart, S. M. (2009). The many meanings of multiple measures. Educational Leadership, 67, 6-12.
22. Buckingham, B. R., McCall, W. A., Otis, A. S., Rugg, H. O., Trabue, M. R., & Courtis, S. A. (1921). Report of the Standardization Committee. Journal of Educational Research, 4, 78-80.
23. Burns, W. C. (1995). Content validity, face validity, and quantitative face validity. Retrieved from http://www.burns.com/wcbcontval.htm
24. Camara, W. J., & Lane, S. (2006). A historical perspective and current views on the Standards for Educational and Psychological Testing. Educational Measurement: Issues and Practice, 25, 35-41. doi: 10.1111/j.1745-3992.2006.00066.x
25. Campbell, D. T. (1957). Factors relevant to the validity of experiments in social settings. Psychological Bulletin, 54, 297-312. doi:10.1037/h0040950
26. Campbell, D. T. (1960). Recommendations for APA test standards regarding construct, trait, or discriminant validity. American Psychologist, 15, 546-553. doi:10.1037/h0048255
27. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105. doi:10.1037/h0046016
28. Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasiexperimental designs for research. Chicago, IL: Rand McNally.
29. Carver, R. P. (1974). Two dimensions of tests: Psychometric and edumetric. American Psychologist, 29, 512-518. doi:10.1037/h0036782
30. Cattell, R. B. (1964). Validity and reliability: A proposed more basic set of concepts. Journal of Educational Psychology, 55, 1-22. doi:10.1037/h0046462
31. Cizek, G. J. (2012). Defining and distinguishing validity: Interpretations of score meaning and justification of test use. Psychological Methods, 17, 31-43. doi:10.1037/a0026975
32. Cizek, G. J., Rosenberg, S. L., & Koons, H. H. (2008). Sources of validity evidence for educational and psychological tests. Educational and Psychological Measurement, 68, 397-412. doi:10.1177/0013164407310130
33. Cone, J. D. (1995). Assessment practice standards. In S. C. Hayes, V. M. Follette, R. M. Dawe, & K. Grady (Eds.), Scientific standards for psychological practice: Issues and recommendations (pp. 201-224). Reno, NV: Context Press.
34. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin.
35. Cronbach, L. J. (1949). Essentials of psychological testing. New York, NY: Harper.
36. Cronbach, L. J. (1988). Five perspectives on validity argument. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 3-17). Hillsdale, NJ: Erlbaum.
37. Cronbach, L. J. (1989). Construct validation after thirty years. In R. L. Linn (Ed.), Intelligence: Measurement, theory and public policy (pp. 147-171). Urbana: University of Illinois Press.
38. Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302. doi:10.1037/h0040957
39. Cureton, E. E. (1951). Validity. In E. F. Lindquist (Ed.), Educational measurement (pp. 621-694). Washington, DC: American Council on Education.
40. Cureton, E. E. (1965). Reliability and validity: Basic assumptions and experimental designs. Educational and Psychological Measurement, 25, 327-346. doi:10.1177/001316446502500204
41. Dick, W., & Hagerty, N. (1971). Topics in measurement: Reliability and validity. New York, NY: McGraw-Hill.
42. Downing, S. M. (2003). Validity: On the meaningful interpretation of assessment data. Medical Education, 37, 830-837. doi:10.1046/j.1365-2923.2003.01594.x
43. Dunnette, M. D. (1992). It was nice to be there: Construct validity then and now. Human Performance, 5, 157-169. doi:10.1207/s15327043hup0501&2_9
44. Dunnette, M. D., & Borman, W. C. (1979). Personnel selection and classification systems. Annual Review of Psychology, 30, 477-525. doi:10.1146/annurev.ps.30.020179.002401
45. Ebel, R. L. (1983). The practical validation of tests of ability. Educational Measurement: Issues and Practice, 2, 7-10. doi:10.1111/j.1745-3992.1983.tb00688.x
46. English, H., & English, A. A. (1958). Comprehensive dictionary of psychological and psychoanalytical terms. New York, NY: Longmans, Green.
47. Evers, A., Sijtsma, K., Lucassen, W., & Meijer, R. R. (2010). The Dutch review process for evaluating the quality of psychological tests: History, procedure, and results. International Journal of Testing, 10, 295-317. doi:10.1080/15305058.2010.518325
48. Fernberger, S. W. (1932). The American Psychological Association: A historical summary, 1892-1930. Psychological Bulletin, 29, 1-89. doi: 10.1037/h0075733
49. Fitzpatrick, A. R. (1983). The meaning of content validity. Applied Psychological Measurement, 7, 3-13. doi:10.1177/014662168300700102
50. Foster, S. L., & Cone, J. D. (1995). Validity issues in clinical assessment. Psychological Assessment, 7, 248-260. doi:10.1037/1040-3590.7.3.248
51. Freebody, P., & Wyatt-Smith, C. (2004). The assessment of literacy: Working the zone between system and site validity. Journal of Educational Enquiry, 5, 30-49.
52. Frisbie, D. A. (2005). Measurement 101: Some fundamentals revisited. Educational Measurement: Issues and Practice, 24, 21-28. doi: 10.1111/j.1745-3992.2005.00016.x
53. Gaylord, R. H., & Stunkel, E. R. (1954). Validity and the criterion. Educational and Psychological Measurement, 14, 294-300. doi: 10.1177/001316445401400209
54. Greene, H. A., Jorgensen, A. N., & Gerberich, J. R. (1943). Measurement and evaluation in the secondary school. New York, NY: Longmans, Green.
55. Guilford, J. P. (1946). New standards for test evaluation. Educational and Psychological Measurement, 6, 427-438.
56. Guilford, J. P. (1954). Psychometric methods (2nd ed.). New York, NY: McGraw-Hill.
57. Guion, R. M. (1977a). Content validity-The source of my discontent. Applied Psychological Measurement, 1, 1-10. doi:10.1177/014662167700100103
58. Guion, R. M. (1977b). Content validity: Three years of talk-What's the action Public Personnel Management, 6, 407-414.
59. Guion, R. M. (1980). On Trinitarian doctrines of validity. Professional Psychology, 11, 385-398. doi:10.1037/0735-7028.11.3.385
60. Guion, R. M. (2009). Was this trip really necessary? Industrial and Organizational Psychology, 2, 465-468. doi:10.1111/j.1754-9434.2009.01174.x
61. Guion, R. M. (2011). Assessment, measurement, and prediction for personnel decisions (2nd ed.). Hove, England: Routledge.
62. Gulliksen, H. (1950). Intrinsic validity. American Psychologist, 5, 511-517. doi:10.1037/h0054604
63. Guttman, L. (1950). The problem of attitude and opinion measurement. In S. A. Stouffer et al. (Eds.), Studies in social psychology in World War II: Vol. 4. Measurement and prediction (pp. 46-59). Princeton, NJ: Princeton University Press.
64. Halkidi, M., Batistakis, Y., & Vazirgiannis, M. (2002). Cluster validity methods: Part 1. SIGMOD Record, 31, 40-45. doi:10.1145/565117.565124
65. Hambleton, R. K. (1980). Test score validity and standard-setting methods. In R. A. Berk (Ed.), Criterion-referenced measurement: The state of the art (pp. 80-123). Baltimore, MD: Johns Hopkins University Press.
66. Hanlon, C., Medhin, G., Alem, A., Araya, M., Abdulahi, A., Hughes, M.,... Prince, M. (2008). Detecting perinatal common mental disorders in Ethiopia: Validation of the Self-Reporting Questionnaire and Edinburgh Postnatal Depression Scale. Journal of Affective Disorders, 108, 251-262. doi:10.1016/j.jad.2007.10.023
67. Hill, H. C., Dean, C., & Gaffney, I. M. (2007). Assessing elemental and structural validity: Data from teachers, non-teachers, and mathematicians. Measurement: Interdisciplinary Research and Perspectives, 5, 81-92. doi:10.1080/15366360701486999
68. Hoffman, R. G., & Davis, G. L. (1995). Prospective validity study: CPI Work Orientation and Managerial Potential Scales. Educational and Psychological Measurement, 55, 881-890. doi:10.1177/0013164495055005024
69. Hogan, T. P., & Agnello, J. (2004). An empirical study of reporting practices concerning measurement validity. Educational and Psychological Measurement, 64, 802-812. doi:10.1177/0013164404264120
70. Holtzman, N. A., & Watson, M. S. (Eds.). (1997). Promoting safe and effective genetic testing in the United States: Final report of the Task Force on Genetic Testing. Retrieved from http://www.genome.gov/10001733
71. Hopwood, C. J., Baker, K. L., & Morey, L. C. (2008). Extratest validity of selected personality assessment inventory scales and indicators in an inpatient substance abuse setting. Journal of Personality Assessment, 90, 574-577. doi:10.1080/00223890802388533
72. Hubley, A. M., & Zumbo, B. D. (1996). A dialectic on validity: Where we have been and where we are going. Journal of General Psychology, 123, 207-215. doi:10.1080/00221309.1996.9921273
73. Jolliffe, D., Farrington, D. P., Hawkins, J. D., Catalano, R. F., Hill, K. G., & Kosterman, R. (2003). Predictive, concurrent, prospective and retrospective validity of self-reported delinquency. Criminal Behaviour and Mental Health, 13, 179-197. doi:10.1002/cbm.541
74. Jonson, J. L., & Plake, B. S. (1998). A historical comparison of validity standards and validity practices. Educational and Psychological Measurement, 58, 736-753. doi:10.1177/0013164498058005002
75. Julnes, G. (2011). Reframing validity in research and evaluation: A multidimensional, systematic model of valid inference. In H. T. Chen, S. I. Donaldson, & M. M. Mark (Eds.), Advancing validity in outcome evaluation: Theory and practice (pp. 55-67). Hoboken, NJ: Wiley.
76. Kane, M. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38, 319-342. doi:10.1111/j.1745-3984.2001.tb01130.x
77. Kane, M. (2004). The analysis of interpretive arguments: Some observations inspired by the comments. Measurement: Interdisciplinary Research and Perspectives, 2, 192-200. doi:10.1207/s15366359mea0203_3
78. Kane, M. (2008). Terminology, emphasis, and utility in validation. Educational Researcher, 37, 76-82. doi:10.3102/0013189X08315390
79. Kane, M. (2009). Validating the interpretations and uses of test scores. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 39-64). Charlotte, NC: Information Age.
80. Kane, M. (2012). All validity is construct validity. Or is it? Measurement: Interdisciplinary Research and Perspectives, 10, 66-70. doi: 10.1080/15366367.2012.681977
81. Kane, M. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50, 1-73. doi:10.1111/jedm.12000
82. Karelitz, T. M., Parrish, D. M., Yamada, H., & Wilson, M. (2010). Articulating assessments across childhood: The cross-age validity of the Desired Results Developmental Profile-Revised. Educational Assessment, 15, 1-26. doi:10.1080/10627191003673208
83. Kvale, S. (1995). The social construction of validity. Qualitative Inquiry, 1, 19-40. doi:10.1177/107780049500100103
84. Landy, F. L. (1986). Stamp collecting versus science: Validation as hypothesis testing. American Psychologist, 41, 1183-1192. doi: 10.1037/0003-066X.41.11.1183
85. Larsen, K. R., Nevo, D., & Rich, E. (2008). Exploring the semantic validity of questionnaire scales. In R. H. Sprague, Jr. (Ed.), Proceedings of the 41st Hawaii International Conference on System Sciences [CD]. Washington, DC: IEEE Computer Society. doi:10.1109/HICSS.2008.165
86. Lather, P. (1986). Issues of validity in openly ideological research: Between a rock and a hard place. Interchange, 17, 63-84. doi:10.1007/BF01807017
87. Lather, P. (1993). Fertile obsession: validity after poststructuralism. The Sociological Quarterly, 34, 673-693.
88. Lawshe, C. H. (1952). Employee selection. Personnel Psychology, 5, 31-34. doi:10.1111/j.1744-6570.1952.tb00990.x
89. Lawshe, C. H. (1985). Inferences from personnel tests and their validity. Journal of Applied Psychology, 70, 237-238.
90. Lennon, R. T. (1956). Assumptions underlying the use of content validity. Educational and Psychological Measurement, 16, 294-304. doi: 10.1177/001316445601600303
91. Lievens, F., Buyse, T., & Sackett, P. R. (2005). The operational validity of a video-based situational judgment test for medical college admission: Illustrating the importance of matching predictor and criterion construct domains. Journal of Applied Psychology, 90, 442-452. doi:10.1037/0021-9010.90.3.442
92. Lindquist, E. F. (1936). The theory of test construction. In H. E. Hawkes, E. F. Lindquist, & C. R. Mann (Eds.), The construction and use of achievement examinations: A manual for secondary school teachers (pp. 17-106). Cambridge, MA: Riverside Press.
93. Linn, R. L. (1978). Single-group validity, differential validity, and differential prediction. Journal of Applied Psychology, 63, 507-512. doi: 10.1037/0021-9010.63.4.507
94. Linn, R. L. (1997). Evaluating the validity of assessments: The consequences of use. Educational Measurement: Issues and Practice, 16, 14-16. doi:10.1111/j.1745-3992.1997.tb00587.x
95. Lissitz, R. W., & Samuelsen, K. (2007). A suggested change in terminology and emphasis regarding validity and education. Educational Researcher, 36, 437-448. doi:10.3102/0013189X07311286
96. Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychological Reports, 3(Suppl. 9), 635-694. doi:10.2466/pr0.1957.3.3.635
97. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
98. MacPhail, F. (1998). Moving beyond statistical validity in economics. Social Indicators Research, 45, 119-149. doi:10.1023/A: 1006989612799
99. Maguire, T., Hattie, J., & Haig, B. (1994). Construct validity and achievement assessment. Alberta Journal of Educational Research, 40(2), 109-126.
100. Maraun, M. D., Slaney, K. L., & Gabriel, S. M. (2009). The Augustinian methodological family of psychology. New Ideas in Psychology, 27, 148-162. doi:10.1016/j.newideapsych.2008.04.011
101. Marcoulides, G. A. (2004). Conceptual debates in evaluating measurement procedures. Measurement: Interdisciplinary Research and Perspectives, 2, 182-184. doi:10.1207/s15366359mea0203_2
102. Markus, K. A. (2012). Constructs and attributes in test validity: Reflections on Newton's account. Measurement: Interdisciplinary Research and Perspectives, 10, 84-87. doi:10.1080/15366367.2012.677348
103. Markus, M. L., & Robey, D. (1980). The organizational validity of management information systems. Cambridge, MA: Massachusetts Institute of Technology, Center for Information Systems Research.
104. Maxwell, J. A. (1992). Understanding and validity in qualitative research. Harvard Educational Review, 62, 279-300.
105. McCrae, R. R. (1982). Consensual validation of personality traits: Evidence from self-reports and ratings. Journal of Personality and Social Psychology, 43, 293-303. doi:10.1037/0022-3514.43.2.293
106. McIntire, S. A., & Miller, L. A. (2007). Foundations of psychological testing: A practical approach (2nd ed.). Thousand Oaks, CA: Sage.
107. Mehrens, W. A. (1997). The consequences of consequential validity. Educational Measurement: Issues and Practice, 16, 16-18. doi: 10.1111/j.1745-3992.1997.tb00588.x
108. Messick, S. (1975). The standard problem: Meaning and values in measurement and evaluation. American Psychologist, 30, 955-966. doi: 10.1037/0003-066X.30.10.955
109. Messick, S. (1980). Test validity and the ethics of assessment. American Psychologist, 35, 1012-1027. doi:10.1037/0003-066X.35.11.1012
110. Messick, S. (1981). Evidence and ethics in the evaluation of tests. Educational Researcher, 10, 9-20. doi:10.3102/0013189X010009009
111. Messick, S. (1988). The once and future issues of validity: Assessing the meaning and consequences of measurement. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 33-48). Hillsdale, NJ: Erlbaum.
112. Messick, S. (1989). Validity. In R. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). Washington, DC: American Council on Education.
113. Messick, S. (1992). Validity of test interpretation and use. In M. C. Alkin (Ed.), Encyclopedia of educational research (6th ed., Vol. 4, pp. 1487-1495). New York, NY: Macmillan.
114. Messick, S. (1998). Test validity: A matter of consequences. Social Indicators Research, 45, 35-44. doi:10.1023/A:1006964925094
115. Miller, M. D., Linn, R. L., & Gronlund, N. E. (2009). Measurement and assessment in teaching (10th ed.). Upper Saddle River, NJ: Pearson Education.
116. Mosier, C. I. (1947). A critical examination of the concepts of face validity. Educational and Psychological Measurement, 7, 191-205. doi:10.1177/001316444700700201
117. Moss, P. A. (1995). Themes and variations in validity theory. Educational Measurement: Issues and Practice, 14, 5-13. doi:10.1111/j.1745-3992.1995.tb00854.x
118. Murphy, K. R. (2009). Content validation is useful for many things, but validity isn't one of them. Industrial and Organizational Psychology, 2, 453-464. doi:10.1111/j.1754-9434.2009.01173.x
119. Nevo, B. (1985). Face validity revisited. Journal of Educational Measurement, 22, 287-293. doi:10.1111/j.1745-3984.1985.tb01065.x
120. Newton, P. E. (2012a). Clarifying the consensus definition of validity. Measurement: Interdisciplinary Research and Perspectives, 10, 1-29. doi:10.1080/15366367.2012.669666
121. Newton, P. E. (2012b). Questioning the consensus definition of validity. Measurement: Interdisciplinary Research and Perspectives, 10, 110-122. doi:10.1080/15366367.2012.688456
122. Pollitt, A. (2012). Validity cannot be created, it can only be lost. Measurement: Interdisciplinary Research and Perspectives, 10, 100-103. doi: 10.1080/15366367.2012.686868
123. Popham, W. J. (1978). Criterion-referenced measurement. Englewood Cliffs, NJ: Prentice-Hall.
124. Popham, W. J. (1997). Consequential validity: Right concern-wrong concept. Educational Measurement: Issues and Practice, 16, 9-13. doi:10.1111/j.1745-3992.1997.tb00586.x
125. Reynolds, C. R., Livingston, R. B., & Willson, V. (2010). Measurement and assessment in education (2nd ed.). Upper Saddle River, NJ: Pearson.
126. Richardson, M. W. (1936). The relation between the difficulty and the differential validity of a test. Psychometrika, 1, 33-49. doi:10.1007/BF02288003
127. Rosenberg, M. (1979). Conceiving the self. New York, NY: Basic Books.
128. Rulon, P. J. (1946). On the validity of educational tests. Harvard Educational Review, 16, 290-296.
129. Scriven, M. (2002). Assessing six assumptions in assessment. In H. I. Braun, D. N. Jackson, & D. E. Wiley (Eds.), The role of constructs in psychological and educational measurement (pp. 255-275). Mahwah, NJ: Erlbaum.
130. Sechrest, L. (1963). Incremental validity: A recommendation. Educational and Psychological Measurement, 23, 153-158. doi:10.1177/001316446302300113
131. Shaw, D. J., & Linden, J. D. (1964). A critique of the Hand Test. Educational and Psychological Measurement, 24, 283-284. doi: 10.1177/001316446402400209
132. Shaw, S., & Weir, C. J. (2007). Examining writing: Research and practice in assessing second language writing. Cambridge, England: Cambridge University Press.
133. Shepard, L. A. (1993). Evaluating test validity. Review of Research in Education, 19, 405-450.
134. Shepard, L. A. (1997). The centrality of test use and consequences for test validity. Educational Measurement: Issues and Practice, 16, 5-24. doi:10.1111/j.1745-3992.1997.tb00585.x
135. Sireci, S. G. (1998). The construct of content validity. Social Indicators Research, 45, 83-117. doi:10.1023/A:1006985528729
136. Sireci, S. G. (2007). On validity theory and test validation. Educational Researcher, 36, 477-481. doi:10.3102/0013189X07311609
137. Sireci, S. G. (2009). Packing and unpacking sources of validity evidence: History repeats itself again. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 19-37). Charlotte, NC: Information Age.
138. Smith, G. T. (2005). On construct validity: Issues of method and measurement. Psychological Assessment, 17, 396-408. doi:10.1037/1040-3590.17.4.396
139. Tenopyr, M. L. (1986). Needed directions for measurement in work settings. In J. V. Mitchell, Jr. (Series Ed.) & B. S. Plake & J. C. Witt (Vol. Eds.), Buros-Nebraska Symposium on Measurement and Testing: Vol. 2. The future of testing (pp. 269-288). Hillsdale, NJ: Erlbaum.
140. Thurstone, L. L. (1931). The reliability and validity of tests: Derivation and interpretation of fundamental formulae concerned with reliability and validity of tests and illustrative problems. Ann Arbor, MI: Edwards. doi:10.1037/11418-000
141. Trochim, W. M. (2006). The research methods knowledge base (2nd ed.). Retrieved from http://www.socialresearchmethods.net/kb/
142. Tryon, R. C. (1957a). Communality of a variable: Formulation by cluster analysis. Psychometrika, 22, 241-260. doi:10.1007/BF02289125
143. Tryon, R. C. (1957b). Reliability and behavior domain validity: Reformulation and historical critique. Psychological Bulletin, 54, 229-249. doi: 10.1037/h0047980
144. Waluchow, W. J. (2009). Four concepts of validity: Reflections on inclusive and exclusive positivism. In M. D. Adler & K. E. Himma (Eds.), The rule of recognition and the United States Constitution (pp. 123-143). Oxford, England: Oxford University Press. doi:10.1093/acprof: oso/9780195343298.003.0005
145. Watson, G., & Forlano, G. (1935). Prima facie validity in character tests. Journal of Educational Psychology, 26, 1-16. doi:10.1037/h0057103
146. Willcutt, E. G., & Carlson, C. L. (2005). The diagnostic validity of attention-deficit/hyperactivity disorder. Clinical Neuroscience Research, 5, 219-232. doi:10.1016/j.cnr.2005.09.003
147. Wolming, S., & Wikstrom, C. (2010). The concept of validity in theory and practice. Assessment in Education: Principles, Policy and Practice, 17, 117-132.
148. Woody, C. (1935). A symposium on the effects of measurement on instruction. Journal of Educational Research, 28, 481-483.
149. Yalow, E., & Popham, W. J. (1983). Content validity at the crossroads. Educational Researcher, 12, 10-21. doi:10.3102/0013189X012008010
150. Zumbo, B. D. (2009). Validity as contextualized and pragmatic explanation, and its implications for validation practice. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 65-82). Charlotte, NC: Information Age.