Assessment as judgment-in-context: Analysing how teachers evaluate students' writing

Educational Research and Evaluation

Volumn 13, Issue 5, 2007, Pages 401-434

  Cooksey, Ray W ^a   Freebody, Peter ^b   Wyatt Smith, Claire ^c  

^a UNIVERSITY OF NEW ENGLAND   (Australia)

^b UNIVERSITY OF SYDNEY   (Australia)

^c GRIFFITH UNIVERSITY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 36249031547     PISSN: 13803611     EISSN: 17444187     Source Type: Journal    
DOI: 10.1080/13803610701728311     Document Type: Article

References (52)

1. Bransford, J. D., & Schwartz, D. L. (1999). Rethinking transfer: A simple proposal with multiple implications. In A. Iran-Nejad & P. D. Pearson (Eds.), Review of research in education (Vol. 24, pp. 61-100). Washington, DC: American Educational Research Association.
2. Brehmer, A. (1989). Grading as a quasi-rational judgment process. In J. Lowyck, C. Clark, & R. Halkes (Eds.), Teacher thinking and professional action. Leuven, Belgium: Leuven University Press.
3. Brehmer, B., & Joyce, C. R. B. (Eds.). (1988). Human judgment: The SJT view. Amsterdam: North-Holland Elsevier.
4. Brunswik, E. (1952). The conceptual framework of psychology. Chicago: University of Chicago Press.
5. Brunswik, E. (1955). Representative design and probabilistic theory in a functional psychology. Psychological Review, 62, 193-217.
6. Brunswik, E. (1956). Perception and the representative design of psychological experiments (2nd ed.). Berkeley, CA: University of California Press.
7. Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
8. Cooksey, R. W. (1988). Social judgment dieory in education: Current and potential applications. In B. Brehmer & C. R. B. Joyce (Eds.), Human judgment: The SJT view (pp. 273-315). Amsterdam: North-Holland Elsevier.
9. Cooksey, R. W. (1996). Judgment analysis: Theory, methods and applications. San Diego, CA: Academic Press.
10. Cooksey, R. W. (2001). What is Complexity Science? A contextually- grounded tapestry of systemic dynamism, paradigm diversity, theoretical eclecticism, and organizational learning. Emergence, 3, 77-103.
11. Cooksey, R. W., Freebody, P., & Davidson, G. R. (1986). Teacher's predictions of children's early reading achievement: An application of social judgment theory. American Educational Research Journal, 23, 41-64.
12. Curriculum Corporation. (1998). Literacy standards: Years 3 & 5 writing, spelling and reading. Carlton South, Victoria, Australia: The Craftsman Press.
13. Doherty, M., & Kurz, E. (1996). Social judgement theory. Thinking & Reasoning, 2, 109-140.
14. Einhorn, H. J., Kleinmuntz, D. N., & Kleinmuntz, B. (1979). Linear regression and process-tracing models of judgment. Psychological Review, 86, 465-485.
15. Einhorn, H. J., & Koelb, C. T. (1982). A psychometric study of literary-critical judgment. Modern Language Studies, 12, 59-82.
16. Elander, J., & Hardman, D. (2002). An application of judgment analysis to examination marking in psychology. British Journal of Psychology, 95(3), 303-326.
17. Ericsson, K. A. (2003). Valid and non-reactive verbalization of thoughts during performance of tasks: Towards a solution to the central problems of introspection as a source of scientific data. In A. Jack & A. Roepstorff (Eds.), Trusting the subject: The use of introspective evidence in cognitive science (Vol. 1, pp. 1 - 18). Exeter, UK: Imprint Academic.
18. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (Rev. ed.). Cambridge, MA: The MIT Press.
19. Forster, M., & Masters, G. (2004). Bridging the conceptual gap between classroom assessment and system accountability. In M. Wilson (Ed.), Towards coherence between classroom assessment and accountability (Part II, pp. 51-73). Chicago: National Society for the Study of Education.
20. Freebody, P., Cumming, J., & Falk, I. (1993). A mailing and evaluation of the Adult Literacy and Numeracy (ALAN) Scales (Vol. 1). Canberra, ACT, Australia: Commonwealth Department of Employment, Education and Training.
21. Freebody, P., & Wyatt-Smith, C. (2004). The assessment of literacy: Working die zone between "system" and "site" validity. Journal of Educational Enquiry, 5(2), 30-49.
22. Gamire, E., & Pearson, P. (Eds.). (2006). Tech tally: Approaches to assessing technological literacy. Washington, DC: National Research Council of the National Academies.
23. Goldstein, W. M. (2004). Social judgment theory: Applying and extending Brunswik's probabilistic functionalism. In D. Koehler & N. Harvey (Eds.), Blackwell handbook of judgment and decision making (pp. 10-11). Oxford: Blackwell.
24. Hamm, R. (1988). Moment-by-moment variation in experts' analytic and intuitive cognitive activity. IEEE Transactions on Systems, Man, and Cybernetics, 18, 757-776.
25. Hammond, K. R. (1980). Introduction to Brunswikian theory and methods. In K. R. Hammond & N. E. Wascoe (Eds.), Realizations of Brunswik's representative design (pp. 1 - 12). San Francisco: Jossey-Bass.
26. Hammond, K. R. (1996). Human judgment and social policy: irreducible uncertainty, inevitable error, unavoidable injustice. New York: Oxford University Press.
27. Hammond, K. R., & Stewart, T. R. (2001). The essential Brunswik: Beginnings, explications, applications. New York: Oxford University Press.
28. Hammond, K., Stewart, T., Brehmer, B., & Steinmann, D. (1975). Social judgment theory. In M. Kaplan & S. Schwartz (Eds.), Human judgment and decision processes (pp. 271-312). New York: Academic Press.
29. Harlen, W. (2005). Teachers' summative practices and assessment for learning-tensions and synergies. The Curriculum Journal, 16(2), 207-223.
30. Hursch, C. J., Hammond, K. R., & Hursch, J. L. (1964). Some methodological considerations in multiple-cue probability studies. Psychological Review, 71, 42-60.
31. Linn, R. L. (1999). Validity standards and principles on equity in educational testing and assessment. In A. L. Nettles & M. T. Nettles (Eds.), Measuring up: Challenges minorities face in educational assessment (pp. 13-31). Boston: Kluwer Academic Publishers.
32. Luke, A., Freebody, P., Lau, S., & Gopinathan, S. (2005). Towards research-based innovation and reform: Singapore schools in transition. Asia Pacific Journal of Education, 25(1), 5-28.
33. Masters, G. N., & Forster, M. (1997). Mapping literacy achievement: Results of the 1996 National School English Literacy Survey. Canberra, ACT, Australia: Commonwealth Department of Employment, Education and Training.
34. Organisation for Economic Co-operation and Development. (2002a). Manual for the PISA 2000 Database. Paris: Author.
35. Organisation for Economic Co-operation and Development. (2002b). PISA 2000 Technical Report. Paris: Author.
36. Richardson, K. A., Cilliers, P., & Lissack, M. (2001). Complexity science: A "gray" science for the "stuff in between". Emergence, 3(2), 6-18.
37. Scanlon, P. A., & Ford, M. P. (1998). Grading student performance in real-world settings. New Directions for Teaching and Learning, 74(Summer), 97-105.
38. Shulman, L. S., & Elstein, A. (1975). Studies of problem solving, judgment, and decision making: Implications for educational research. In F. N. Kerlinger (Ed.), Review of research in education (Vol. 3, pp. 3-42). Itasca, NY: Peacock.
39. Slomp, D. H., & Fuite, J. (2004). Following Phaedrus: Alternate choices in surmounting the reliability/validity dilemma. Assessing Writing, 3, 190-207.
40. Snow, R. E. (1968). Brunswikian approaches to research on teaching. American Educational Research Journal, 5, 475-489.
41. Snow, R. E. (1974). Representative and quasi-representative designs for research on teaching. Review of Educational Research, 44(3), 265-291.
42. Speck, B. W. (1998). Unveiling some of the mystery of professional judgment in classroom assessment. New Directions for Teaching and Learning, 74(Summer), 17-31.
43. Stewart, T. R. (1988). Judgment analysis: Procedures. In B. Brehmer & C. R. B. Joyce (Eds.), Human judgment: The SJT view (pp. 41-74). Amsterdam: North-Holland.
44. Suter, L. (2004). Tools for two masters: Classroom assessment and school system assessment. In M. Wilson (Ed.), Towards coherence between classroom assessment and accountability (Part II, pp. 169-182). Chicago: National Society for the Study of Education.
45. Thompson, B. (2000). Q-technique factor analysis: One variation on the two-mode factor analysis of variables. In L. G. Grimm & P. R. Yarnold (Eds.), Reading and understanding more multivariate statistics (pp. 207-226). Washington, DC: American Psychological Association.
46. Tucker, L. R. (1964). A suggested alternative formulation in the developments by Hursch, Hammond, & Hursch and by Hammond, Hursch, & Todd. Psychological Review, 71, 528-530.
47. Willett, L. (2004, November). Marking writing tasks for tests of literacy in years 3, 5 and 7: Questions and challenges. Paper presented at the 9th Annual Roundtable Assessing Assessment, Sydney, NSW, Australia.
48. Wilson, M. (2004). Assessment, accountability and the classroom: A community of judgment. In M. Wilson (Ed.), Towards coherence between classroom assessment and accountability (Part II, pp. 1-19). Chicago: National Society for the Study of Education.
49. Wyatt-Smith, C., Castleton, G., Freebody, P., & Cooksey, R. W. (2003). The nature of teacher's qualitative judgements: A matter of context and salience. Part One: "In-context" judgements. Australian Journal of Language and Literacy, 26(2), 11 - 32.
50. Wyatt-Smith, C., Castleton, G., Cooksey, R. W., & Freebody, P. (2003). The nature of teacher's qualitative judgements: A matter of context and salience. Part Two: Out-of-context judgements. Australian Journal of Language and Literacy, 26(2), 33-42.
51. Van Someren, M., Barnard, Y., & Sandberg, J. (1994). The think-aloud method: A practical guide to modelling cognitive processes. London: Academic Press.
52. Yen, N. S. (2005). The effects of task complexity, task uncertainty and time pressure on judgment in multiple cue probability learning task. Journal of Education and Psychology, 28(1), 101-126.