Teachers' and students' perceptions of assessments: A review and a study into the ability and accuracy of estimating the difficulty levels of assessment items

Educational Research Review

Volumn 1, Issue 2, 2006, Pages 133-147

  van de Watering, Gerard ^a   van der Rijt, Janine ^a  

^a MAASTRICHT UNIVERSITY   (Netherlands)

Author keywords

Assessment; Exams; Higher education; Item difficulty; Student perceptions

Indexed keywords

EID: 33749603973     PISSN: 1747938X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.edurev.2006.05.001     Document Type: Article

References (39)

1. Abbot M. Standard setting for complex performance assessments: A critical examination of the analytic judgment method. Paper presented at the annual congress of the Canadian society for the study of education (2003, May)
2. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. Standards for educational and psychological testing (1985), Washington, DC, American Psychological Association
3. Allen M.J., and Yen W.M. Introduction to measurement theory (1979), Brooks/Cole, Monterey
4. Angoff W.H. Scales, norms and equivalent scores. In: Thorndike R.L. (Ed). Educational measurement (1971), American Council on Education, Washington 508-600
5. Barefoot B. Foundations of excellence: A new model for first-year assessment. Assessment update 16 2 (2004) 5-7
6. Bateman A., and Griffin P. The appropriateness of professional judgement to determine performance rubrics in a graded competency based assessment framework. Paper presented at the AARE/NZARE conference (2003, November-December). http://www.aare.edu.au/03pap/ Retrieved February 2, 2005 from http://www.aare.edu.au/03pap/
7. Bereby-Meijer Y., Meijer J., and Flascher O.M. Prospect theory analysis of guessing in multiple choice tests. Journal of Behavioral Decision Making 15 (2002) 313-327
8. Biggs J. Teaching for quality learning at university. 2nd ed. (2003), SRHE and Open University Press, Buckingham
9. In: Bloom B.S. (Ed). Taxonomy of educational objectives. Handbook I: The cognitive domain (1956), McKay, New York
10. Brandon P.R. Conclusions about frequently studied modified Angoff standard-setting topics. Applied Measurement in Education 17 1 (2004) 59-88
11. Brown S., and Glasner A. Assessment matters in higher education (1999), SRHE and Open University Press, Buckingham
12. Chang L. Judgmental item analysis of the Nedelsky and Angoff standard-setting methods. Applied Measurement in Education 12 2 (1999) 151-165
13. Chinn R.N., and Hertz N.R. Alternative approaches to standard setting for licensing and certification examinations. Applied Measurement in Education 15 1 (2002) 1-14
14. Dochy F.J.R.C. Assessment of prior knowledge as a determinant for future learning (1992), Jessica Kingsley Publisher/Lemma, London/Utrecht
15. Dochy F., Segers M., and Buehl M. The relation between assessment practices and outcomes of studies: The case of research on prior knowledge. Review of Educational Research 69 2 (1999) 147-188
16. Ebel R.L. Essentials of educational measurement (1972), Prentice-Hall, Englewood Cliffs
17. Fehrmann M.L., Woehr D.J., and Arthur W. The Angoff cut-off score method: The impact of frame-of-reference rater training. Educational and Psychological Measurement 51 (1991) 857-872
18. Feldt L.S., and Brennan R.L. Reliability. In: Linn R.L. (Ed). Educational measurement (1989), American Council on Education, Washington, DC 105-146
19. Gibbs G. Using assessment strategically to change the way students learn. In: Brown S., and Glasner A. (Eds). Assessment matters in higher education (1999), SRHE & Open University Press, Buckingham 41-53
20. Goodwin L.D. Focus on quantitative methods: Determining cut-off scores. Research in Nursing & Health 19 (1996) 249-256
21. Goodwin L.D. Relations between observed item difficulty levels and Angoff minimum passing levels for a group of borderline examinees. Applied Measurement in Education 12 1 (1999) 13-28
22. Hurtz G.M., and Auerbach M.A. A meta-analysis of the effects of modifications to the Angoff method on cutoff scores and judgment consensus. Educational and Psychological Measurement 63 4 (2003) 584-601
23. Impara J.C., and Plake B.S. Teachers' ability to estimate item difficulty: A test of the assumptions in the Angoff standard setting method. Journal of Educational Measurement 35 (1998) 69-81
24. Irwin P.M., Plake B.S., and Impara J.C. Validity of item performance estimates from an Angoff standard setting study. Paper presented at the annual meeting of the American educational research association (2000). http://www.unl.edu/BIACO/AERA/ Retrieved February 11, 2005, from http://www.unl.edu/BIACO/AERA/
25. Leathwood C. Assessment policy and practice in higher education: Purpose, standards and equity. Assessment & Evaluation in Higher Education 30 3 (2005) 307-324
26. Lee F.L., and Heyworth R.M. Problem complexity: A measure of problem difficulty in algebra for use in cai systems. Education Journal 28 1 (2000). http://www.fed.cuhk.edu.hk/fllee/Papers/JournPa/ Retrieved December 15, 2004 from http://www.fed.cuhk.edu.hk/fllee/Papers/JournPa/
27. Mattar, J. D. (2000). Investigation of the validity of the Angoff standard setting procedure for multiple-choice items. Unpublished doctoral dissertation, University of Massachusetts, Amherst.
28. Mehrens W.A., and Lehmann I.J. Measurement and evaluation in education and psychology (1991), Holt, Rinehart & Winston, Fort Worth
29. Messick S. Validity. In: Linn R.L. (Ed). Educational measurement (1989), Macmillan, New York 13-103
30. Mills C.N., Melican G.J., and Ahluwalia N.T. Defining minimal competence. Educational Measurement: Issues and Practice 10 2 (1991) 7-10
31. Nedelsky L. Absolute grading standards for objective tests. Educational and Psychological Measurement 14 (1954) 3-19
32. Plake B.S., and Impara J.C. Ability of panellists to estimate item performance for a target group of candidates: An issue in judgemental standard setting. Educational Assessment 7 2 (2001) 87-97
33. Scouller K. The influence of assessment method on students' learning approaches: Multiple choice question examination versus assignment essay. Higher Education 35 (1998) 453-472
34. Shepard L.A. Implications for standard setting of the national academy of education evaluation of national assessment of educational progress achievement levels proceedings from the joint conference on standard setting for large-scale assessments (1995), National Assessment Governing Board and National Center for Education Statistics, Washington
35. Stanley J.C. Reliability. In: Thorndike R.L. (Ed). Educational measurement (1971), American Council on Education, Washington, DC 356-442
36. Stowell M. Equity, justice and standards: Assessment decision making in higher education. Assessment & Evaluation in Higher Education 29 4 (2004) 495-510
37. Van de Watering G., and Claessens S. The discrepancy between tutors' perceptions of student performance and achievement results. Paper presented at the 10th European Association for Research on learning and instruction conference (2003, August)
38. Verhoeven B.H., Verwijnen G.M., Muijtjens A.M.M., Scherpbier A.J.J.A., and Van der Vleuten C.P.M. Panel expertise for an Angoff standard setting procedure in progress testing: Item writers compared to recently graduated students. Medical Education 36 (2002) 860-867
39. Webb N.M., and Farivar S. Developing productive group interaction in middle school Mathematics. American Educational Research Journal 31 2 (1999) 369-395