Grading scheme, test difficulty, and the immediate feedback assessment technique

Journal of Experimental Education

Volumn 77, Issue 4, 2009, Pages 311-338

  DiBattista, David ^a   Gosse, Leanne ^a   Sinnige Egger, Jo Anne ^a   Candale, Bela ^a   Sargeson, Kim ^a  

^a BROCK UNIVERSITY   (Canada)

Author keywords

Grading scheme; Immediate feedback assessment technique; Multiple choice testing; Test difficulty

Indexed keywords

EID: 67649510570     PISSN: 00220973     EISSN: None     Source Type: Journal    
DOI: 10.3200/JEXE.77.4.311-338     Document Type: Article

References (65)

1. Aamodt, M. G., & McShane, T. D. (1992). A meta-analytic investigation of the effect of various test item characteristics on test scores and test completion times. Public Personnel Management, 21, 151-160.
2. Ace, M. C., & Dawis, R. V. (1973). Item structure as a determinant of item difficulty in verbal analogies. Educational and Psychological Measurement, 33, 143-149.
3. Aiken, L. R. (1987). Testing with multiple-choice items. Journal of Research and Development in Education, 20, 44-58.
4. Attali, Y., & Bar-Hillel, M. (2003). Guess where: The position of correct answers in multiple-choice test items as a psychometric variable. Journal of Educational Measurement, 40, 109-128.
5. Bangert-Drowns, R. L., Kulik, C. C., Kulik, J. A., & Morgan, M. T. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61, 213-238.
6. Ben-Simon, A., Budescu, D. V., & Nevo, B. (1997). A comparative study of measures of partial knowledge in multiple-choice tests. Applied Psychological Measurement, 21, 65-88.
7. Bereby-Meyer, Y., Meyer, J., & Flascher, O. M. (2002). Prospect theory analysis of guessing in multiple choice tests. Journal of Behavioral Decision Making, 15, 313-327.
8. Biggs, J. (1999). Teaching for quality learning at university. Buckingham, England: Society for Research into Higher Education.
9. Brosvic, G. M., Epstein, M. L., Cook, M. J., & Dihoff, R. E. (2005). Efficacy of error for the correction of initially incorrect assumptions and of feedback for the affirmation of correct responding: Learning in the classroom. Psychological Record, 55, 401-418.
10. Brosvic, G. M., Walker, M. A., Perry, N., Degnan, S., & Dihoff, R. E. (1997). Illusion decrement as a function of duration of inspection and figure type. Perceptual and Motor Skills, 84, 779-783.
11. Budescu, D., & Bar-Hillel, M. (1993). To guess or not to guess: A decision-theoretic view of formula scoring. Journal of Educational Measurement, 30, 277-292.
12. Bush, M. (2001). A multiple choice test that rewards partial knowledge. Journal of Further and Higher Education, 25, 157-163.
13. Byrnes, J. P., Miller, D. C., & Schafer, W. D. (1999). Gender differences in risk taking: A meta- analysis. Psychological Bulletin, 125, 367-383.
14. Carver, C. S., & Scheier, M, F. (1981). Attention and self regulation: A control theory approach to human behavior. New York: Springer- Verlag.
15. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
16. Coombs, C. H., Milholland, J. E., & Womer, F. B. (1956). The assessment of partial knowledge. Educational & Psychological Measurement, 16, 13-37.
17. Cross, L. H., & Frary, R. B. (1977). An empirical test of Lord's theoretical results regarding formula- scoring of multiple-choice tests. Journal of Educational Measurement, 14, 313-321.
18. De Finetti, B. (1965). Methods for discriminating levels of partial knowledge concerning a test item. British Journal of Mathematical & Statistical Psychology, 18, 87-123.
19. DiBattista, D. (2003). [Awarding of partial credit with the IFAT]. Unpublished raw data.
20. DiBattista, D. (2005). The Immediate Feedback Assessment Technique: A learner-centered multiple- choice response form. Canadian Journal of Higher Education, 35, 111-131.
21. DiBattista, D., & Gosse, L. (2006). Test anxiety and the Immediate Feedback Assessment Technique. Journal of Experimental Education, 74, 311-327.
22. DiBattista, D., Mitterer, J. O., & Gosse, L. (2004). Acceptance by undergraduates of the Immediate Feedback Assessment Technique for multiple-choice testing. Teaching in Higher Education, 9, 17-28.
23. Dihoff, R. E., Brosvic, G. M., & Epstein, M. L. (2003). The role of feedback during academic testing: The delay retention effect revisited. Psychological Record, 53, 533-548.
24. Dressel, P. L., & Schmid, J. (1953). Some modifications of the multiple-choice item. Educational & Psychological Measurement, 13, 574-595.
25. Ebel, R. L. (1968). Blind guessing on objective assessment tests. Journal of Educational Measurement, 5, 321-325.
26. Elliott, D. (1988). The influence of visual target and limb information on manual aiming. Canadian Journal of Psychology, 42, 57-68.
27. Epstein Educational Enterprises. (2006). IF-AT frequently asked questions. Retrieved July, 2006, from http://www.epsteineducation.com/faq. php
28. Epstein, M. L., Brosvic, G. M., Costner, K. L., Dihoff, R. E., & Lazarus, A. D. (2003). Effectiveness of feedback during the testing of preschool children, elementary school children, and adolescents with developmental delays. Psychological Record, 53, 177-195.
29. Epstein, M. L., Epstein, B. B., & Brosvic, G. M. (2001). Immediate feedback during academic testing. Psychological Reports, 88, 889-894.
30. Epstein, M. L., Lazarus, A. D., Calvano, T. B., Matthews, K. A., Hendel, R. A., Epstein, B. B., et al. (2002). Immediate feedback assessment technique promotes learning and corrects inaccurate first responses. Psychological Record, 52, 187-201.
31. Frary, R. B. (1980). The effect of misinformation, partial information, and guessing on expected multiple-choice test item scores. Applied Psychological Measurement, 4, 79-90.
32. Frary, R. B. (1988). Formula scoring of multiple-choice tests (correction for guessing). Educational Measurement: Issues and Practices, 7(2), 33-38.
33. Green, K. E. (1981). Text anxiety level and format preference. Psychological Reports, 48, 537-538.
34. Gustav, A. (1963). Response set in objective achievement tests. The Journal of Psychology: Interdisciplinary and Applied, 56, 421-427.
35. Jaradat, D., & Tollefson, N. (1988). The impact of alternative scoring procedures for multiple-choice items on test reliability, validity, and grading. Educational and Psychological Measurement, 48, 627-635.
36. Kehoe, J. (1995). Basic item analysis for multiple-choice tests. Practical Assessment, Research & Evaluation, 4(10). Retrieved July, 2006, from http://www.PAREonline.net/getvn.asp?v=4&n=10/
37. Keppel, G. (1991). Design and analysis: A researcher's handbook. Englewood Cliffs, NJ: Prentice Hall.
38. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback intervention on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254-284.
39. Kluger, A. N., & DeNisi, A. (1998). Feedback interventions: Toward the understanding of a double- edged sword. Current Directions in Psychological Science, 7, 67-72.
40. Kulhavy, R. W., & Anderson, R. C. (1972). Delay-retention effect with multiple-choice tests. Journal of Educational Psychology, 68, 505-512.
41. Kulik, J. A., & Kulik, C. C. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58, 79-97.
42. Levine, T. R., & Hullett, C. R. (2002). Eta squared, partial eta squared, and misreporting of effect size in communication research. Human Communication Research, 28, 612-625.
43. Marcus, A. (1963). The effect of correct response location on the difficulty level of multiple-choice questions. Journal of Applied Psychology, 47, 48-51.
44. Mayer, R. E. (2003). Learning and Instruction. Upper Saddle River, NJ: Merrill Prentice Hall.
45. McDougall, D. (1997). College faculty's use of objective tests: State -of -the -practice versus state -of- the- art. Journal of Research and Development in Education, 30, 183-193.
46. McMorris, R. F., Boothroyd, R. A., & Pietrangelo, D. J. (1997). Humor in educational testing: A review and discussion. Applied Measurement in Education, 10, 269-297.
47. McNamara, W. J., & Weitzman, E. (1945). The effect of choice placement on the difficulty of multiple-choice questions. Journal of Educational Psychology, 36, 103-113.
48. Mikulincer, M. (1988). Reactance and helplessness following exposure to unsolvable problems: The effects of attributional style. Journal of Personality and Social Psychology, 54, 679-686.
49. Mikulincer, M. (1989). Cognitive interference and learned helplessness: The effects of off-task cognitions on performance following unsolvable problems. Journal of Personality and Social Psychology, 57, 129-135.
50. Mikulincer, M., Yinon, A., & Kabili, D. (1991). Epistemic needs and learned helplessness. European Journal of Personality, 5, 249-258.
51. Olejnik, S., & Algina, J. (2000). Measures of effect size for comparative studies: Applications, interpretations, and limitations. Contemporary Educational Psychology, 25, 241-286.
52. Pashler, H., Cepeda, N. J., Wixted, J. T., & Rohrer, D. (2005). When does feedback facilitate learning of words? Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 3-8.
53. Phye, G. D., & Andre, T. (1989). Delayed retention effect: Attention, perseveration, or both? Contemporary Educational Psychology, 14, 173-185.
54. Phye, G. D., & Bender, T. (1989). Feedback complexity and practice: Response pattern analysis in retention and transfer. Contemporary Educational Psychology, 14, 97-110.
55. Prieto, G., & Delgado, A. R. (1999). The effect of instructions on multiple-choice test scores. European Journal of Psychological Assessment, 15, 143-150.
56. Roda, C., & Thomas, J. (2006). Attention aware systems: Theories, applications, and research agenda. Computers in Human Behavior, 22, 557-587.
57. Roediger, H. L., & Marsh, E. J. (2005). The positive and negative consequences of multiple-choice testing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 1155-1159.
58. Sassenrath, J. M., Yonge, G. D., & Schrable, K. (1968). Immediate and delayed feedback on examinations and immediate and delayed retention. California Journal of Educational Research, 19, 226-231.
59. Skinner, B. F. (1954). The science of learning and the art of teaching. Harvard Educational Review, 24, 86-97.
60. Taras, M. (2003). To feedback or not to feedback in student self-assessment. Assessment & Evaluation in Higher Education, 28, 549-565.
61. Thorndike, E. L. (1913). Educational Psychology. New York: Columbia University Press.
62. Thorndike, E. L. (1920). A constant error in psychological ratings. Journal of Applied Psychology, 4, 469-477.
63. Thorndike, R. M. (2005). Measurement and evaluation in psychology and education (7th ed.). Upper Saddle River, NJ: Pearson Education.
64. Toppino, T. C., & Luipersbeck, S. M. (1993). Generality of the negative suggestion effect in objective tests. Journal of Educational Research, 86, 357-362.
65. Tozoglu, D., Tozoglu, M. D., Gurses, A., & Dogar, C. (2004). The students' perceptions: Essay versus multiple-choice type exams. Journal of Baltic Science Education, 6, 52-59.