Validity of portfolio assessment: Which qualities determine ratings?

Medical Education

Volumn 40, Issue 9, 2006, Pages 862-866

  Driessen, Erik W ^a   Overeem, Karlijn ^b   Van Tartwijk, Jan ^c   Van Der Vleuten, Cees P M ^a   Muijtjens, Arno M M ^a  

^a MAASTRICHT UNIVERSITY   (Netherlands)

^b RADBOUD UNIVERSITY MEDICAL CENTER   (Netherlands)

^c LEIDEN UNIVERSITY   (Netherlands)

Author keywords

Documentation *standards methods; Education, medical, undergraduate *methods; Educational measurement *standards methods; Netherlands; Observer variation; Teaching *methods

Indexed keywords

ARTICLE; CLINICAL COMPETENCE; CORRELATION ANALYSIS; CORRELATION COEFFICIENT; DOCUMENTATION; MEDICAL EDUCATION; MEDICAL STUDENT; REGRESSION ANALYSIS; SCORING SYSTEM; VALIDITY;

DOCUMENTATION; EDUCATION, MEDICAL, UNDERGRADUATE; EDUCATIONAL MEASUREMENT; NETHERLANDS; OBSERVER VARIATION; PILOT PROJECTS; REGRESSION ANALYSIS; TEACHING;

EID: 33747705479     PISSN: 03080110     EISSN: 13652923     Source Type: Journal    
DOI: 10.1111/j.1365-2929.2006.02550.x     Document Type: Article

References (17)

1. McMullan M, Endacott R, Gray MA, Jasper M, Miller CML, Scholes J, Webb C. Portfolios and assessment of competence: a review of the literature. J Adv Nurs 2003;41(3):283-94.
2. Pitts J, Coles C, Thomas P. Educational portfolios in the assessment of general practice trainers: reliability of assessors. Med Educ 1999;33:515-20.
3. Cadbury-Amyot C, Kim J, Palm R, Mills E, Noble E, Overman P. Validity and reliability of portfolio assessment of competency in a baccalaureate dental hygiene programme. J Dent Educ 2003;67(9):991-1002.
4. Tillema H. Design and validity of a portfolio instrument for professional training. Studies Educational Eval 1998;24(3):265-74.
5. Valencia S, Au K. Portfolios across educational contexts: issues of evaluation, teacher development and system validity. Educational Assessment 1997;4(1):1-35.
6. Norman GR, van der Vleuten CPM, De Graaff E. Pitfalls in the pursuit of objectivity: issues of validity, efficiency and acceptability. Med Educ 1991;25:119-26.
7. Ryan JM, Kuhs TM. Assessment of pre-service teachers and the use of portfolios. Theory Pract 1993;32:75-81.
8. Driessen EW, van der Vleuten CPM, Schuwirth L, van Tartwijk J, Vermunt JD. The use of qualitative research criteria for portfolio assessment as an alternative to reliability evaluation: a case study. Med Educ 2005;39:214-20.
9. Quinlan KM. Inside the peer review process: how academics review a colleague's teaching portfolio. Teach Teacher Educ 2002;18:1035-49.
10. Heller J, Sheingold K, Myford C. Reasoning about evidence in portfolios: cognitive foundations for valid and reliable assessment. Educational Assessment 1998;5(1):5-40.
11. Driessen EW, van Tartwijk J, Vermunt JD, van der Vleuten CPM. Use of portfolios in early undergraduate medical training. Med Teacher 2003;25(1):18-23.
12. Driessen EW, van Tartwijk J, Overeem K, Vermunt JD, van der Vleuten CPM. Conditions for successful use of portfolios for reflection. Med Educ 2005;39:1230-5.
13. Cohen JA. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 1968;70:213-20.
14. Graham P, Jackson R. The analysis of ordinal agreement data: beyond weighted kappa. J Clin Epidemiol 1993;46:1055-62.
15. Kianifard F. Evaluation of clinimetric scales: basic principles and methods. Statistician 1994;43:475-82.
16. Rees C, Sheard C. The reliability of assessment criteria for undergraduate medical students' communication skills portfolios: the Nottingham experience. Med Educ 2004;38:138-44.
17. Pitts J, Coles C, Thomas P, Smith F. Enhancing reliability in portfolio assessment: discussions between assessors. Med Teacher 2002;24(2):197-01.