Cognitive representations in raters' assessment of teacher portfolios

Studies in Educational Evaluation

Volumn 31, Issue 1, 2005, Pages 27-55

  van der Schaaf, Marieke F ^a   Stokking, Karel M ^a   Verloop, Nico ^b  

^a UTRECHT UNIVERSITY   (Netherlands)

^b LEIDEN UNIVERSITY   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 19744362669     PISSN: 0191491X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.stueduc.2005.02.005     Document Type: Article

References (58)

1. Andrews T.E. & Barnes S. Assessment of teaching In: W.R. Houston (Ed.) Handbook of research on teacher education 1990 Macmillan New York 569-598
2. Bird T. The schoolteacher's portfolio In: J. Millman & L. Darling-Hammond (Eds.) The new handbook of teacher evaluation: Assessing elementary and secondary school teachers 1990 Sage Newbury Park, CA 241-256
3. Borman W.C. Consistency of rating accuracy and rating error in the judgment of human performance Organizational Behavior and Human Performance 20 1977 238-252
4. Borsboom D. Mellenbergh G.J. Van & Heerden J. The concept of validity Psychological Review 111 4 2004 1061-1071
5. Boud D. Assessment and the promotion of academic values Studies in Higher Education 15 1990 101-111
6. Burns C.W. Teaching portfolios and the evaluation of teaching in higher education: Confident claims, questionable research support Studies in Educational Evaluation 25 1999 131-142
7. Carlston D. Impression formation and the modular mind: The associated systems theory In: L.L. Martin & A. Tesser (Eds.) The construction of social judgments 1992 Erlbaum Hillsdale, NJ 301-341
8. Carlston D. Associated systems theory: A systematic approach to cognitive representations of persons Advances in Social Cognition 7 1994 1-78
9. Cochran-Smith M. & Fries M.K. Sticks, stones, and ideology: The discourse of reform in teacher education Educational Research 30 2001 3-15
10. Conway M.A. Associations between autobiographical memories and concepts Journal of Experimental Psychology: Learning, Memory and Cognition 16 1990 799-812
11. Day D.V. & Sulsky L.M. Effects of frame-of-reference training and information configuration on memory organization and rating accuracy Journal of Applied Psychology 80 1995 158-167
12. De Corte E. Instructional psychology: Overview In: E. De Corte F.E. & Weinert (Eds.) International encyclopedia of developmental and instructional psychology 1996 Elsevier Science Oxford 33-43
13. Delandshere G. & Arens S.A. Representations of teaching and standards-based reform: Are we closing the debate about teacher education? Teaching and Teacher Education 17 2001 57-566
14. Delandshere G. & Petrosky A. Capturing teachers' knowledge: Performance assessment (a) and post-structuralist epistemology, (b) from a post-structuralist perspective, (c) and post-structuralism, (d) none of the above Educational Researcher 23 1994 11-18
15. DeNisi A.S. Cafferty T.P. & Meglino B.M. A cognitive view of the performance appraisal process: A model and research propositions Organizational Behavior and Human Performance 33 1984 360-396
16. Dierick S. & Dochy F. New lines in edumetrics: New forms of assessment lead to new assessment criteria Studies in Educational Evaluation 27 2001 307-329
17. Dunbar S.B. Koretz D. & Hoover H.D. Quality control in the development and use of performance assessments Applied Measurement in Education 4 1991 289-304
18. Ericsson K.A. & Simon H.A. Verbal reports as data Psychological review 87 1980 215-251
19. Ericsson K.A. & Simon H.A. Protocol analysis: Verbal reports as data 1984 MIT Press Cambridge, MA
20. Fazio R.H. Attitudes in associated systems theory In: R.S. Wyer (Ed.), Associated systems theory: A systematic approach to cognitive representations of persons. Advances in social cognition 1994 Erlbaum Hillsdale, NJ 157-167
21. Feldman J.M. Beyond attribution theory: Cognitive processes in performance appraisal Journal of Applied Psychology 66 1981 127-148
22. Fiske S.T. Thinking is for doing: Portraits of social cognition from daguerreotype to laserphoto Journal of Personality and Social Psychology 63 1992 877-889
23. Gentile C. Exploring new methods for collecting students' school-based writing: NAEP's 1990 Portfolio Study 1992 Office of Educational Research and Improvement Washington, DC
24. Gilbert D.T. Thinking lightly about others: Automatic components of the social inference process In: J.S. Uleman & J.A. Bargh (Eds.) Unintended thought 1989 Guilford New York 189-211
25. Huot B.A. The influence of holistic scoring procedures on reading and rating student essays In: M.M. Williamson & B.A. Huot (Eds.) Validating holistic scoring for writing assessment. Theoretical and empirical foundations 1993 Hampton Press Cresskill, NJ 206-232
26. Jones E.E. & Davis K.E. From acts to dispositions: The attribution process in person perception In: L. Berkowitz (Ed.), Advances in Experimental Social Psychology 2 1965 Academic Press New York 219-266
27. Johnson R.L. McDaniel F. & Willeke M.J. Using portfolios in program evaluation: An investigation of interrater reliability The American Journal of Evaluation 21 2000 65-80
28. Kane M.T. An argument-based approach to validity Psychological Bulletin 112 1992 527-535
29. Kelly G.A. The psychology of personal constructs 1955 Norton New York
30. Koretz D. Klein S. McCaffrey D. & Stecher B. The reliability of scores from the 1992 Vermont portfolio assessment program 1992 RAND Institute on Education & Teaching Washington, DC
31. Landy F.J. & Farr J.L. Performance rating Psychological Bulletin 87 1980 72-107
32. LeMahieu P. Gitomer D. & Eresh J. Portfolios in large-scale assessment: Difficult but not impossible Educational Measurement: Issues and Practice 14 1995 11-16 25-28
33. Lievens F. Assessor training strategies and their effects on accuracy, interrater reliability, and discriminant validity Journal of Applied Psychology 86 2001 255-264
34. Linn R.L. Performance assessment. Policy promises and technical measurement standards Educational Researcher 23 1994 4-14
35. Linn R.L. Baker E.L. & Dunbar S.B. Complex, performance-based assessment: Expectations and validation criteria Educational Researcher 20 1991 15-21
36. Long D.L. & Bourgh T. Thinking aloud: Telling a story about a story. Commentary Discourse Processes 21 1996 329-339
37. Lyons N. With portfolio in hand. Validating the new teacher professionalism 1998 Teachers College Press New York
38. Martindale C. Cognitive psychology: A neural-network approach 1991 Brooks/Cole Pacific Grove, CA
39. Messick S. Validity in: R.L. Linn (Ed.) Educational measurement 1989 MacMillan New York 13-103
40. Moss P.A. Shifting conceptions of validity in educational measurement: Implications for performance assessment Review of Educational Research 62 3 1992 229-258
41. Moss P.A. Can there be validity without reliability? Educational Researcher 23 1994 5-12
42. Murphy K.R. & De Shon R. Interrater correlations do not estimate the reliability of job performance ratings Personnel Psychology 53 2000 873-900
43. Norman D.A. Human information processing: The conventional view In: A.M. Aitkenhead & J.M. Slack (Eds.) Issues in cognitive modelling 1985 Erlbaum Hillsdale, NJ 309-336
44. Nunally J. Psychometric theory 2nd ed. 1978 McGraw-Hill New York
45. Pula J.J. & Huot B.A. A model of background influences on holistic raters In: M.M. Williamson & B.A. Huot (Eds.) Validating holistic scoring for writing assessment. Theoretical and empirical foundations 1993 Hampton Press Cresshill, NJ 237-265
46. (Unpublished doctoral dissertation) Popping R. Overeenstemmingsmaten voor nominale data [Computing agreement on nominal data] 1983 Rijksuniversiteit Groningen Groningen, the Netherlands
47. Reckase M.D. Portfolio assessment: A theoretical estimate of score reliability Educational Measurement: Issues and Practice 14 1995 12-14 31
48. Russo J.E. Johnson E.J. Stephens & D.L. The validity of verbal protocols Memory & Cognition 17 1989 759-769
49. Schleicher D.J. & Day D.V. A cognitive evaluation of frame-of-reference rater training: Content and process issues Organizational Behavior and Human Decision Processes 73 1998 76-101
50. Shapley K.S. & Bush M.J. Developing a valid and reliable portfolio assessment in the primary grades: Building on practical experience Applied Measurement in Education 12 1999 111-132
51. Shavelson R.J. Baxter G.P. & Gao X. Sampling variability of performance assessments Journal of Educational Measurement 30 1993 215-232
52. Stokking K. Van der Schaaf M. Jaspers J. & Erkens G. Teachers' assessment of students' research skills British Journal of Educational Research 30 2004 93-115
53. Tulving E. Episodic and semantic memory In: E. Tulving & W. Donaldson (Eds.) Organization and memory 1972 Academic Press New York 381-403
54. Tulving E. Elements of episodic memory 1983 Clarendon Press Oxford
55. Van der Schaaf M.F. Stokking K.M. & Verloop N. Developing performance standards for teacher assessment by policy capturing Assessment & Evaluation in Higher Education 28 2003 395-410
56. Van der Schaaf M.F. Stokking K.M. & Verloop N. (submitted). Developing teaching content standards using a delphi method.
57. Woehr D.J. & Huffcutt A.I. Rater training for performance appraisal: A quantitative review Journal of Occupational and Organizational Psychology 67 1994 189-205
58. Zwaan R.A. & Brown C.M. The influence of language proficiency and comprehension skill on situation-model construction Discourse Processes 21 1996 289-327