Validity and its threats

Assessment in Health Professions Education

Volumn , Issue , 2009, Pages 21-55

  Downing, Steven M ^a   Haladyna, Thomas M ^b  

^a UNIVERSITY OF ILLINOIS AT CHICAGO   (United States)

^b ARIZONA STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84917177995     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4324/9780203880135     Document Type: Chapter

References (60)

1. Abedi, J. (2006). Language issues in item development. In S.M. Downing & T.M. Haladyna (Eds.), Handbook of test development (pp. 377-398). Mahwah, NJ: Lawrence Erlbaum Associates.
2. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing . Washington, DC:American Educational Research Association.
3. Angoff, W.H. (1971). Scales, norms, and equivalent scores. In R.L. Thorndike (Ed.), Educational measurement (2d ed., pp. 508-600). Washington:American Council on Education.
4. Auewarakul, C., Downing, S.M., Jaturatumrong, U., & Praditsuwan, R. (2005). Sources of validity evidence for an internal medicine student evaluation system: An evaluative study of assessment methods. Medical Education, 39, 276-283.
5. Baranowski, R. (2006). Item editing and editorial review. In S.M. Downing & T.M. Haladyna (Eds.), Handbook of test development (pp. 349-357). Mahwah, NJ: Lawrence Erlbaum Associates.
6. Beckman, T.J., & Cook, D.A. (2005). What is the validity evidence for assessments of clinical teaching? Journal of General Internal Medicine, 20, 1159-1164.
7. Boulet, J.R., McKinley, D.W., Whelan, G.P., & Hambleton, R.K. (2003). Quality assurance methods for performance-based assessments. Advances in Health Sciences Education, 8, 27-47.
8. Brennan, R.L. (2001). Generalizability theory . New York: Springer-Verlag.
9. Campbell, D.T., & Fiske D.W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.
10. Case, S.M., & Swanson, D.B. (1998). Constructing written test questions for the basic and clinical sciences (2nd ed.). Philadelphia, PA: National Board of Medical Examiners.
11. Cizek, G.J. (2006). Standard setting. In S.M. Downing & T.M. Haladyna (Eds.), Handbook of test development (pp. 225-258). Mahwah, NJ:Lawrence Erlbaum Associates.
12. Cook, D.A., & Beckman, T.J. (2006). Current concepts in validity and reliability for psychometric instruments: Theory and application. American Journal of Medicine, 119(2), e7-16.
13. Cronbach, L.J. (1980). Validity on parole: How can we go straight? New directions for testing and measurement: Measuring achievement over a decade. Proceedings of the 1979 ETS Invitational Conference (pp. 99-108). San Francisco: Jossey-Bass.
14. Cronbach, L.J. (1988). Five perspectives on validity argument. In H. Wainer & H. Braun (Eds.), Test validity (pp. 3-17). Hillsdale, NJ: Lawrence Erlbaum.
15. Cronbach, L.J. (1989). Construct validation after thirty years. In R.E. Linn (Ed.), Intelligence: Measurement, theory, and public policy (pp. 147-171). Urbana, IL: University of Illinois Press.
16. Cronbach, L.J., & Meehl, P.E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302.
17. Crossley, J., Davies, H., Humphris, G., & Jolly, B. (2002). Generalisability; a key to unlock professional assessment. Medical Education, 36, 972-978.
18. Cureton, E.E. (1951). Validity. In E.F. Lingquist (Ed.), Educational measurement (pp. 621-694). Washington: American Council on Education.
19. De Champlain, A.F., & Floreck, L.M. (2002, April). Assessing potential bias in a large-scale standardized patient examination: An application of common DIF methods for polytomous items. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA.
20. Downing S.M. (1996). Test validity evidence: What about face validity? CLEAR Exam Review, Sum, 31-33.
21. Downing S.M. (2002a). Construct-irrelevant variance and flawed test questions:Do multiple-choice item writing principles make any difference? Academic Medicine, 77, S103-S104.
22. Downing, S.M. (2002b). Threats to the validity of locally developed multiplechoice tests in medical education: Construct-irrelevant variance and construct underrepresentation. Advances in Health Sciences Education, 7, 235-241.
23. Downing, S.M. (2003a). Guessing on selected-response examinations. Medical Education, 37, 670-671.
24. Downing, S.M. (2003b). Item response theory: Applications in modern test theory in medical education. Medical Education, 37, 1-7.
25. Downing, S.M. (2005). The effects of violating standard item writing principles on tests and students: The consequences of using flawed test items on achievement examinations in medical education. Advances in Health Sciences Education, 10, 133-143.
26. Downing, S.M. (2006a). Face validity of assessments: Faith-based interpretations or evidence-based science? Medical Education, 40, 7-8.
27. Downing, S.M. (2006b). Twelve steps for effective test development. In S.M. Downing & T.M. Haladyna (Eds.), Handbook of test development (pp. 3-25). Mahwah, NJ: Lawrence Erlbaum Associates.
28. Downing, S.M., Tekian, A., & Yudkowsky, R. (2006). Procedures for establishing defensible absolute passing scores on performance examinations in health professions education. Teaching & Learning in Medicine, 18(1), 50-57.
29. Ebel, R.L. (1972). Essentials of educational measurement (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall.
30. Elstein, A.S, Shulman, L.S, & Sprafka, S.A. (1978). Medical problem solving:An analysis of clinical reasoning . Cambridge, MA: Harvard University Press.
31. Embretson, S.E., & Reise, S.P. (2000). Item response theory for psychologists . Mahwah, NJ: Lawrence Erlbaum Associates.
32. Engelhard, G. Jr. (2002). Monitoring raters in performance assessments. In G. Tindal & T.M. Haladyna (Eds.), Large-scale assessment programs for all students: Validity, technical adequacy, and implementation (pp. 261-288). Mahwah, NJ: Lawrence Erlbaum Associates.
33. Haladyna, T.M. (2006). Roles and importance of validity studies in test development. In S.M. Downing & T.M. Haladyna (Eds.), Handbook of test development (pp. 739-755). Mahwah, NJ: Lawrence Erlbaum Associates.
34. Haladyna, T.M., & Downing, S.M. (1993). How many options is enough for a multiple-choice test item? Education and Psychological Measurement, 53, 999-1010.
35. Haladyna, T.M, & Downing, S.M. (2004). Construct-irrelevant variance in high-stakes testing. Educational Measurement: Issues and Practice, 23(1), 17-27.
36. Haladyna, T.M., Downing, S.M., & Rodriguez, M.C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309-334.
37. Holland, P.W., & Wainer, H. (Eds.). (1993). Differential item functioning . Mahwah, NJ: Lawrence Erlbaum.
38. Kane, M.T. (1992). An argument-based approach to validation. Psychological Bulletin, 112, 527-535.
39. Kane, M.T. (1994). Validating interpretive arguments for licensure and certification examinations. Evaluation and the Health Professions, 17, 133-159.
40. Kane, M. (2006a). Content-related validity evidence in test development. In S.M. Downing & T.M. Haladyna (Eds.), Handbook of test development (pp. 131-153). Mahwah, NJ: Lawrence Erlbaum Associates.
41. Kane, M. (2006b) Validation. In R.L. Brennan (Ed.), Educational measurement (4th ed., pp. 17-64). New York: American Council on Education and Greenwood.
42. Linn, R.L. (2002). Validation of the uses and interpretations of results of state assessment and accountability systems. In G. Tindal & T. Haladyna (Eds.), Large-scale assessment programs for all students: Development, implementation, and analysis (pp. 27-48). Mahwah, NJ: Lawrence Erlbaum Associates.
43. Linn, R.L. (2006). The standards for educational and psychological testing:Guidance in test development. In S.M. Downing & T.M. Haladyna (Eds.), Handbook of test development (pp. 27-38). Mahwah, NJ: Lawrence Erlbaum Associates.
44. Messick, S. (1989). Validity. In R.L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-104). New York: American Council on Education and Macmillan.
45. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749.
46. Miller, G.E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65, s63-67.
47. Mosier, C.I. (1947). A critical examination of the concepts of face validity. Educational and Psychological Measurement, 7, 191-205.
48. Newble, D.I., & Jaeger, K. (1983). The effects of assessment and examinations on the learning of medical students. Medical Education, 17, 165-171.
49. Norcini, J.J. (2003). Setting standards on educational tests. Medical Education, 37, 464-469.
50. Norcini, J., & Guille, R. (2002). Combining tests and setting standards. In G.R. Norman, C.P.M. van der Vleuten, & D.I. Newble (Eds.), International handbook of research in medical education (pp. 811-834). Dordrecht, The Netherlands: Kluwer Academic Publishers.
51. Norcini, J.J., & Shea, J.A. (1997). The credibility and comparability of standards. Applied Measurement in Education, 10, 39-59.
52. Norman, G.R., Tugwell, P., Feightner, J.W., Muzzin, L.J., & Jacoby, L.L. (1985). Knowledge and problem-solving. Medical Education, 19, 344-356.
53. Penfield, R.D., & Lam, R.C.M. (2000). Assessing differential item functioning in performance assessment: Review and recommendations. Educational Measurement: Issues and Practice, 19, 5-15.
54. Raymond, M., & Neustel, S. (2006). Determining the content of credentialing examinations. In S.M. Downing & T.M. Haladyna (Eds.), Handbook of test development (pp. 181-223). Mahwah, NJ: Lawrence Erlbaum Associates.
55. Rodriguez, M.C. (2005). Three options are optimal for multiple-choice items:A meta-analysis of 80 years of research. Educational Measurement: Issues and Practice, Summer, 3-13.
56. Subkoviak, M.J. (1988). A practitioner's guide to computation and interpretation of reliability indices for mastery tests. Journal of Educational Measurement, 25, 47-55.
57. Van der Linden, W.J., & Hambleton, R.K. (1997). Item response theory:Brief history, common models, and extensions. In W.J. van der Linden & R.K. Hambleton (Eds.), Handbook of modern item response theory (pp. 1-28). New York: Springer-Verlag.
58. van der Vleuten, C.P.M., & Swanson, D.B. (1990). Assessment of clinical skills with standardized patients: State of the art. Teaching and Learning in Medicine, 2, 58-76.
59. Williams, R.G., Klamen, D.A., & McGaghie, W.C. (2003). Cognitive, social and environmental sources of bias in clinical competence ratings. Teaching and Learning in Medicine, 15, 270-292.
60. Zieky, M. (2006). Fairness review in assessment. In S.M. Downing & T.M. Haladyna (Eds.), Handbook of test development (pp. 359-376). Mahwah, NJ: Lawrence Erlbaum Associates.