Validity in work-based assessment: Expanding our horizons

Medical Education

Volumn 47, Issue 12, 2013, Pages 1164-1174

  Govaerts, Marjan ^a   van der Vleuten, Cees Pm ^a  

^a MAASTRICHT UNIVERSITY   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

CLINICAL COMPETENCE; EDUCATION; HUMAN; LEARNING; MEDICAL EDUCATION; PROCEDURES; PROFESSIONAL COMPETENCE; PSYCHOMETRY; REPRODUCIBILITY; STANDARDS; WORKPLACE;

CLINICAL COMPETENCE; EDUCATION, MEDICAL; EDUCATIONAL MEASUREMENT; HUMANS; LEARNING; PROFESSIONAL COMPETENCE; PSYCHOMETRICS; REPRODUCIBILITY OF RESULTS; WORKPLACE;

EID: 84887356685     PISSN: 03080110     EISSN: 13652923     Source Type: Journal    
DOI: 10.1111/medu.12289     Document Type: Article

References (62)

1. Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA 2002;287 (2):226-35.
2. Albanese MA. Challenges in using rater judgments in medical education. J Eval Clin Pract 2000;6 (3):305-19.
3. Downing SM. Threats to the validity of clinical teaching assessments: what about rater error? Med Educ 2005;39:353-5.
4. Kreiter CD, Ferguson KJ. Examining the generalizability of ratings across clerkships using a clinical evaluation form. Eval Health Prof 2001;24:36-46.
5. van Barneveld C. The dependability of medical students' performance ratings as documented on in-training evaluations. Acad Med 2005;80 (3):309-12.
6. Cook DA, Beckman TJ, Mandrekar JN, Pankratz VS. Internal structure of mini-CEX scores for internal medicine residents: factor analysis and generalizability. Adv Health Sci Educ Theory Pract 2010;15 (5):633-45.
7. Crossley J, Jolly B. Making sense of work-based assessment: ask the right questions, in the right way, about the right things, of the right people. Med Educ 2012;46 (1):28-37.
8. Kane M. Validation. In: Brennan RL, ed. Educational Measurement. Westport, CT: American Council on Education/Praeger 2006; 621-94.
9. Moss PA, Girard BJ, Haniford LC. Validity in educational measurement. Rev Res Educ 2006;30:109-62.
10. American Educational Research Association, American Psychological Association & National Council on Measurement in Education. Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association 1999.
11. Koch MJ, DeLuca C. Rethinking validation in complex high-stakes assessment contexts. Assess Educ Princ Pol Pract 2012;19 (1):99-116.
12. Delandshere G. Assessment as inquiry. Teach Coll Rec 2002;104 (7):1461-84.
13. Fisher CD. What if we took within-person performance variability seriously? Ind Organ Psychol 2008;1:185-9.
14. Sturman MC, Cashen LH, Cheramie RA. The impact of job complexity and performance measurement on the temporal consistency, stability, and test-retest reliability of employee job performance ratings. J Appl Psychol 2005;90 (2):269-83.
15. Durning SJ, Artino AR, Boulet JR, Dorrance K, Van der Vleuten CPM, Schuwirth LWT. The impact of selected contextual factors on experts' clinical reasoning performance (does context impact clinical reasoning performance in experts?). Adv Health Sci Educ Theory Pract 2012;17 (1):65-79.
16. Lance CE, Hoffman BJ, Gentry WA, Baranik LE. Rater source factors represent important subcomponents of the criterion construct space, not rater bias. Hum Resource Manag Rev 2008;18:223-32.
17. Hager P. Theories of workplace learning. In: Malloch M, Cairns L, Evans K, O'Connor BN, The SAGE Handbook of Workplace Learning. Los Angeles, CA: Sage Publications 2011; 17-32.
18. Bleakley A. Broadening conceptions of learning in medical education: the message from teamworking. Med Educ 2006;40:150-7.
19. Mann KV. Theoretical perspectives in medical education: past experience and future possibilities. Med Educ 2011;45:60-8.
20. Lingard L. Rethinking competence in the context of teamwork. In: Hodges BD, Lingard L, The Question of Competence: Reconsidering Medical Education in the Twenty-first Century. London: Cornell University Press 2012; 42-70.
21. Fraser SW, Greenhalgh T. Coping with complexity: educating for capability. BMJ 2001;323:799-803.
22. Engeström Y, Sannino A. Studies of expansive learning: foundations, findings and future challenges. Educ Res Rev 2010;5 (1):1-24.
23. Mennin S. Self-organisation, integration and curriculum in the complex world of medical education. Med Educ 2010;44:20-30.
24. Bleakley A. Blunting Occam's razor: aligning medical education with studies of complexity. J Eval Clin Pract 2010;16:849-55.
25. Ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med 2007;82:542-7.
26. Hager P, Hodkinson P. Moving beyond the metaphor of transfer of learning. Br Educ Res J 2009;35 (4):619-38.
27. Eva KW. On the generality of specificity. Med Educ 2003;37 (7):587-8.
28. Fisher CD, Noble CS. A within-person examination of correlates of performance and emotions while working. Hum Perf 2004;17:145-68.
29. Deadrick D, Bennett N, Russell C. Using hierarchical linear modeling to examine dynamic performance criteria over time. J Manage 1997;23:745-57.
30. Stewart GL, Nandkeolyar AK. Exploring how constraints created by other people influence intraindividual variation in objective performance measures. J Appl Psychol 2007;92 (4):1149-58.
31. Norcini J, Burch V. Workplace-based assessment as an educational tool. AMEE Guide no. 31. Med Teach 2007;29 (9):855-71.
32. Beal DJ, Weiss HM, Barros E, MacDermid SM. An episodic process model of affective influences on performance. J Appl Psychol 2005;90 (6):1054-68.
33. Groysberg B, McLean AN, Nohria N. Are leaders portable? Harv Bus Rev 2006;84 (5):92-101.
34. Groysberg B, Lee L-E. Hiring stars and their colleagues: exploration and exploitation in professional service firms. Organ Sci 2009;20 (4):740-58.
35. Wenghofer E, Williams AP, Klass DJ. Factors affecting physician performance: implications for performance improvement and governance. Health Policy 2009;5 (2):e141-60.
36. Ginsburg S, Bernabeo E, Ross KM, Holmboe ES. "It depends": results of a qualitative study investigating how practicing internists approach professional dilemmas. Acad Med 2012;87 (12):1-9.
37. Schuwirth LWT, van der Vleuten CPM. A plea for new psychometric models in educational assessment. Med Educ 2006;40:296-300.
38. Gipps C. Socio-cultural aspects of assessment. Rev Res Educ 1999;24:355-92.
39. Murphy KR, Cleveland JN. Understanding Performance Appraisal. Social, Organizational and Goal-based Perspectives. Thousand Oaks, CA: Sage Publications 1995.
40. Levy PE, Williams JR. The social context of performance appraisal: a review and framework for the future. J Manag 2004;30:881-905.
41. Tziner A, Murphy KR, Cleveland JN. Contextual and rater factors affecting rating behaviour. Group Organ Manag 2005;30 (1):89-98.
42. Ferris GR, Munyon TP, Basik K, Buckley MR. The performance evaluation context: social, emotional, cognitive, political and relationship components. Hum Resource Manag Rev 2008;18:146-63.
43. Delandshere G, Petrosky A. Capturing teachers' knowledge: performance assessment and post-structuralism. Educ Res 1994;23 (5):11-8.
44. Landy FJ, Farr JL. Performance rating. Psychol Bull 1980;87 (1):72-107.
45. Kogan JR, Conforti L, Bernabeo E, Lobst W, Holmboe E. Opening the black box of clinical skills assessment via observation: a conceptual model. Med Educ 2011;45:1048-60.
46. Govaerts MJB, Van de Wiel MWJ, Schuwirth LWT, Van der Vleuten CPM, Muijtjens AMM. Workplace-based assessment: raters' performance theories and constructs. Adv Health Sci Educ Theory Pract 2013;18 (3):375-96.
47. Shay SB. The assessment of complex performance: a socially situated interpretive act. Harv Educ Rev 2004;74 (3):307-29.
48. Moss PA. Enlarging the dialogue in educational measurement: voices from interpretive research traditions. Educ Res 1996;25 (1):20-8. 43.
49. Johnston B. Summative assessment of portfolios: an examination of different approaches to agreement over outcomes. Stud High Educ 2004;29 (3):395-412.
50. Kuper A, Reeves S, Albert M, Hodges BD. Assessment: do we need to broaden our methodological horizons? Med Educ 2007;41:1121-3.
51. Delandshere G, Petrosky AR. Assessment of complex performances: limitations of key measurement assumptions. Educ Res 1998;27 (2):14-24.
52. Kane MT. Terminology, emphasis, and utility in validation. Educ Res 2008;37 (2):76-82.
53. Guba E, Lincoln Y. Fourth Generation Evaluation. London: Sage Publications 1989.
54. Lincoln YS, Guba EG. Naturalistic Inquiry. Newbury Park, CA: Sage Publications 1985.
55. Krefting L. Rigor in qualitative research: the assessment of trustworthiness. Am J Occup Ther 1991;45 (3):214-22.
56. Driessen E, van der Vleuten CPM, Schuwirth L, van Tartwijk J, Vermunt J. The use of qualitative research criteria for portfolio assessment as an alternative to reliability evaluation: a case study. Med Educ 2005;39:214-20.
57. Tigelaar DEH, Dolmans DHJM, Wolfhagen IHAP, van der Vleuten CPM. Quality issues in judging portfolios: implications for organizing teaching portfolio assessment procedures. Stud High Educ 2005;30 (5):595-610.
58. van der Vleuten CPM, Schuwirth LWT, Scheele F, Driessen EW, Hodges B. The assessment of professional competence: building blocks for theory development. Best Pract Res Cl Ob 2010;24 (6):703-19.
59. Wigton RS. What do the theories of Egon Brunswik have to say to medical education? Adv Health Sci Educ Theory Pract 2008;13:109-21.
60. Hodges B. Assessment in the post-psychometric era: learning to love the subjective and collective. Med Teach 2013;35(7):564-8.
61. Schuwirth L, Ash J. Assessing tomorrow's learners: in competency-based education only a radically different holistic method of assessment will work. Six things we could forget. Med Teach 2013;35(7):555-9.
62. Schuwirth LWT, Van der Vleuten CPM. Programmatic assessment: from assessment of learning to assessment for learning. Med Teach 2011;33:478-85.