Exploring the role of assessment criteria during teachers' collaborative judgement processes of students' portfolios

Assessment and Evaluation in Higher Education

Volumn 37, Issue 7, 2012, Pages 847-860

  van der Schaaf, Marieke ^a   Baartman, Liesbeth ^b   Prins, Frans ^a  

^a UTRECHT UNIVERSITY   (Netherlands)

^b EINDHOVEN UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

assessment; moderation; portfolio; teacher; validity

Indexed keywords

EID: 84868321483     PISSN: 02602938     EISSN: 1469297X     Source Type: Journal    
DOI: 10.1080/02602938.2011.576312     Document Type: Article

References (37)

1. Andresen, L., Boud, D. and Cohen, R. 2000. "Experience-based learning". In Understanding adult education and training, Edited by: Foley, G. 225-239. Sydney: Allen & Unwin.
2. Baartman, L.K.J., Prins, F.J., Kirschner, P.A. and Van der Vleuten, C.P.M. 2007. Determining the quality of Competence Assessment Programs: A self-evaluation procedure. Studies in Educational Evaluation, 33: 258-281.
3. Borman, W.C. 1977. Consistency of rating accuracy and rating error in the judgement of human performance. Organizational Behaviour and Human Performance, 20: 238-252.
4. Borsboom, D., Mellenbergh, G.J. and Van Heerden, J. 2004. The concept of validity. Psychological Review, 111(4): 1061-1071.
5. Clark, D.B. and Sampson, V.D. 2007. Personally-seeded discussions to scaffold online argumentation. International Journal of Science Education, 29(3): 253-277.
6. Clark, D.B., Sampson, V., Weinberger, A. and Erkens, G. 2007. Analytic frameworks for assessing dialogue argumentation in online environments. Educational Psychological Review, 19: 343-374.
7. Cook, T.D. and Campbell, D.T. 1979. Quasi-experimentation: Design and analysis for field settings, Chicago, IL: Rand McNally.
8. Cooksey, R.W., Freebody, P. and Wyatt-Smith, C. 2007. Assessment as judgement-in-context: Analyzing how teachers evaluate students' writing. Educational Research and Evaluation, 13(5): 401-434.
9. Cronbach, L.J. 1988. "Five perspectives on validity argument". In Test validity, Edited by: Wainer, H. and Brain, H.I. 3-17. Hillsdale, NJ: Lawrence Erlbaum.
10. Crooks, T.J. and Kane, M.T. 1996. Threats to the valid use of assessment. Assessment in Education: Principles, Policy and Practice, 3(3): 265-285.
11. DeNisi, A.S., Cafferty, T.P. and Meglino, B.M. 1984. A cognitive view of the performance appraisal process: A model and research propositions. Organizational Behavior and Human Performance, 33: 360-396.
12. Elander, J. 2004. Student assessment from a psychological perspective. Psychology Learning and Teaching, 3(2): 34-41.
13. Elander, J. and Hardman, D. 2002. An application of judgement analysis to examination marking in psychology. British Journal of Psychology, 93(3): 303-328.
14. Erduran, S., Simon, S. and Osborne, J. 2004. Tapping into argumentation: Developments in the application of Toulmin's argument pattern for studying science discourse. Science Education, 88: 915-933.
15. Erkens, G. 2002. MEPA Multiple Episode Protocol Analysis (version 4.8), Utrecht: Utrecht University.
16. Erkens, G., Clark, D.B., Sampson, V. and Weinberger, A. 2007. Analytic frameworks for assessing dialogue argumentation in online learning environments. Educational Psychology Review, 19(3): 343-374.
17. Erkens, G. and Janssen, J.J.H.M. 2008. Automatic coding of dialogue acts in collaboration protocols. International Journal of Computer-supported Collaborative Learning, 3(4): 447-470.
18. Heller, J.I., Scheingold, K. and Myford, C.M. 1998. Reasoning about evidence in portfolios: Cognitive foundations for valid and reliable assessment. Educational Assessment, 5(1): 5-40.
19. Jones, E.E. and Davis, K.E. 1965. "From acts to dispositions: The attribution process in person perception". In Advances in experimental social psychology, Edited by: Berkowitz, L. 219-266. New York, NY: Academic Press.
20. Kane, M.T. 1992. An argument-based approach to validity. Psychological Bulletin, 112(3): 527-535.
21. Landy, F.J. and Farr, J.L. 1980. Performance rating. Psychological Bulletin, 87: 72-107.
22. Linn, R.L. 1994. Performance assessment. Policy promises and technical measurement standards. Educational Researcher, 23(9): 4-14.
23. Little, J.W., Gearhart, M., Curry, M. and Kafka, J. 2003. Looking at student work for teacher learning, teacher community and school reform. Phi Delta Kappan, 85(3): 184-192.
24. Meeus, W., van Looy, L. and van Petegem, P. 2006. Portfolio in higher education: Time for a clarificatory framework. Journal of Teaching and Learning in Higher Education, 17(3): 127-135.
25. Meeus, W., Van Petegem, P. and Engels, N. 2009. Validity and reliability of portfolio-assessment in pre-service teacher education. Assessment and Evaluation in Higher Education, 34(4): 401-413.
26. Messick, S. 1989. "Validity". In Educational measurement, 3rd ed., Edited by: Linn, R.L. 13-103. New York, NY: Macmillan.
27. Messick, S. 1995. Validity of psychological assessment. Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9): 741-749.
28. Moon, J.A. 1999. Reflection in learning and professional development: Theory and practice, London: RoutledgeFalmer.
29. Moss, P.A., Schutz, A.M. and Collins, K.M. 1998. An integrated approach to portfolio evaluation for teacher licensure. Journal of Personnel Evaluation in Education, 12(2): 139-161.
30. Nijveldt, M.J., Beijaard, D., Brekelmans, J.M.G., Verloop, N. and Wubbels, T. 2006. Assessing the interpersonal competence of beginning teachers: The quality of assessors' judgement processes. International Journal of Educational Research, 43: 89-102.
31. Reckase, M.D. 1995. Portfolio assessment: A theoretical estimate of score reliability. Educational Measurement: Issues and Practice, 14(1): 12-14.
32. Rust, C., Price, M. and O'Donovan, B. 2003. Improving students' learning by developing their understanding of assessment criteria and processes. Assessment and Evaluation in Higher Education, 28(2): 147-164.
33. Shapley, K.S. and Bush, M.J. 1999. Developing a valid and reliable portfolio assessment in the primary grades: Building on practical experience. Applied Measurement in Education, 12: 111-132.
34. Toulmin, S. 1958. The uses of argument, Cambridge: Cambridge University Press.
35. Van der Schaaf, M.F. and Stokking, K.M. 2008. Developing and validating a design for teacher portfolio assessment. Assessment and Evaluation in Higher Education, 33(3): 245-258.
36. Van der Schaaf, M.F., Stokking, K. and Verloop, N. 2005. Cognitive representations in raters' assessment of teacher portfolios. Studies in Educational Evaluation, 31(1): 27-55.
37. Wolf, K. and Dietz, M. 1998. Teaching portfolios: Purposes and possibilities. Teacher Education Quarterly, 25: 9-22.