A comparative study of measures of partial knowledge in multiple-choice tests

Applied Psychological Measurement

Volumn 21, Issue 1, 1997, Pages 65-88

  Ben Simon, Anat ^b   Budescu, David V ^a   Nevo, Baruch ^b  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b University of Illinois ^*

Author keywords

Guessing; Misinformation; Partial knowledge; Response methods; Scoring methods; Self assessment

Indexed keywords

EID: 0031533459     PISSN: 01466216     EISSN: None     Source Type: Journal    
DOI: 10.1177/0146621697211006     Document Type: Article

References (65)

1. Angoff, W. H. (1989). Does guessing really help? Journal of Educational Measurement, 26, 323-336.
2. Armstrong, R. J., & Mooney, R. F. (1969, February). Confidence testing: Is it reliable? Paper presented at the annual meeting of the National Council on Measurement in Education, Los Angeles.
3. Arnold, J. C., & Arnold, P. L. (1970). On scoring multiple choice exams allowing for partial knowledge. The Journal of Experimental Education, 39, 8-13.
4. Bernhardson, C. S. (1967). Comparison of the three-decision and conventional multiple-choice tests. Psychological Reports, 20, 695-698.
5. Boldt, R. F. (1971). A simple confidence testing format (Research Bulletin 71-42). Princeton NJ: Educational Testing Service.
6. Budescu, D. V. (1988). On the feasibility of multiple matching tests: Variations on a theme by Gulliksen. Applied Psychological Measurement, 12, 5-14.
7. Budescu, D. V. (1993). Self-evaluation of success in psychological testing. In B. Nevo & R. S. Jager (Eds.), Educational and psychological testing: The test-taker's outlook (pp. 153-176). Gottingen, Germany: Hogrefe & Huber Publishers.
8. Budescu, D. V., & Bar-Hillel, M. (1993). To guess or not to guess: A decision theoretic analysis of formula scoring. Journal of Educational Measurement, 30, 277-291.
9. Collet, L. S. (1971). Elimination scoring: An empirical evaluation. Journal of Educational Measurement, 8, 209-214.
10. Coombs, C. H., Milholland, J. E., & Womer, F. B. (1956). The assessment of partial knowledge. Educational and Psychological Measurement, 16, 13-37.
11. Cross, L. H., & Frary, R. B. (1977). An empirical test of Lord's theoretical results regarding formula scoring of multiple choice tests. Journal of Educational Measurement, 14, 313-321.
12. Cross, L. H., & Thayer, N. F. (1979). A new method for administering and scoring multiple choice tests: Theoretical and empirical considerations. Unpublished manuscript, Virginia Polytechnic Institute and State University, Blacksburg.
13. De Finetti, B. (1965). Methods for discriminating levels of partial knowledge concerning a test item. British Journal of Mathematical and Statistical Psychology, 18, 87-123.
14. Diamond, J. J. (1975). A preliminary study of the reliability and validity of a scoring procedure, based upon confidence and partial information. Journal of Educational Measurement, 12, 129-133.
15. Diamond, J. J., & Evans, W. (1973). The correction for guessing. Review of Educational Research, 43, 181-191.
16. Dressel, P. L., & Schmidt, J. (1953). Some modifications of the multiple choice item. Educational and Psychological Measurement, 13, 574-595.
17. Echternacht, G. J. (1976). Reliability and validity of item option weighting schemes. Educational and Psychological Measurement, 36, 301-310.
18. Evans, R. M., & Misfeldt, K. (1974). Effects of self-scoring procedures on test reliability. Perceptual and Motor Skills, 38, 1248.
19. Feldt, L. S. (1969). A test of the hypothesis that Cronbach's alpha or Kuder Richardson coefficient twenty is the same for two tests. Psychometrika, 34, 363-373.
20. Frary, R. B. (1980). The effect of misinformation, partial information, and guessing on expected multiple- choice test item scores. Applied Psychological Measurement, 4, 79-90.
21. Frary, R. B. (1989). Partial-credit scoring methods for multiple-choice tests. Applied Measurement in Education, 2, 79-96.
22. Gibbons, J. D., Olkin, I., & Sobel, M. (1979). A subset selection technique for scoring items on a multiple choice test. Psychometrika, 44, 259-270.
23. Gilman, D. A., & Ferry, P. (1972). Increasing test reliability through self scoring procedures. Journal of Educational Measurement, 9, 205-207.
24. Gulliksen, H. (1950). Theory of mental tests. New York: Wiley.
25. Gulliksen, H. (1986). Perspective on educational measurement. Applied Psychological Measurement, 10, 109-132.
26. Hakstian, A. R., & Kansup, W. (1975). A comparison of several methods of assessing partial knowledge in multiple choice tests: II. Testing procedures. Journal of Educational Measurement, 12, 219-230.
27. Hakstian, A. R., & Whalen, T. E. (1976). A k-sample significance test for independent alpha coefficients. Psychometrika, 41, 219-231.
28. Haladyna, T. M. (1988, April). Empirically based polychotomous scoring of multiple choice test items: A review. Paper presented at the annual meeting of the American Educational Research Association, New Orleans.
29. Hambleton, R. K., Roberts, D. M., & Traub, R. E. (1970). A comparison of the reliability and validity of two methods for assessing partial knowledge on a multiple choice test. Journal of Educational Measurement, 7, 75-82.
30. Hanna, G. S. (1975). Incremental reliability and validity of multiple choice tests with an answer-until-correct procedure. Journal of Educational Measurement, 12, 175-178.
31. Hanna, G. S. (1977). Study of reliability and validity effects of total and partial immediate feedback in multiple choice testing. Journal of Educational Measurement, 14, 1-7.
32. Hopkins, K. D., Hakstian, A. R., & Hopkins, B. R. (1973). Validity and reliability consequences of confidence weighting. Educational and Psychological Measurement, 33, 135-141.
33. Horst, P. (1966). Psychological measurement and prediction. Belmont CA: Wadsworth.
34. Hsu, T. C., Moss, P. A., & Khampalikit, C. (1984). The merits of multiple answer items as evaluated by using six scoring formulas. Journal of Experimental Education, 52, 152-158.
35. Hutchinson, T. P. (1982). Some theories of performance in multiple choice tests, and their implications for variants of the task. British Journal of Mathematical and Statistical Psychology, 35, 71-89.
36. Jaradat, O., & Swagad, S. (1986). The subset selection technique for multiple choice tests: An empirical inquiry. Journal of Educational Measurement, 23, 369-376.
37. Jaradat, D., & Tollefson, N. (1988). The impact of alternative scoring procedures for multiple-choice items on test reliability, validity and grading. Educational and Psychological Measurement, 48, 627-635.
38. Kansup, W., & Hakstian, A. R. (1975). A comparison of several methods of assessing partial knowledge in multiple-choice tests: I. Scoring procedures. Journal of Educational Measurement, 12, 212-230.
39. Keren, G. (1992). Calibration and probability judgments: Conceptual and methodological issues. Acta Psychologica, 77, 217-273.
40. Koehler, R. A. (1971). A comparison of the validities of conventional choice testing and various confidence marking procedures. Journal of Educational Measurement, 8, 297-303.
41. Leclercq, D. (1983). Confidence marking: Its use in testing. Evaluation in Education, 6, 161-287.
42. Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982). Calibration of probabilities: The state of the art to 1980. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 306-334). Cambridge, England: Cambridge University Press.
43. Lord, F. M. (1975). Formula scoring and number-right scoring. Journal of Educational Measurement, 12, 7-11.
44. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading MA: Addison-Wesley.
45. Mabe, P. A., & West, S. G. (1982). Validity of self-evaluation of ability: A review and meta-analysis. Journal of Applied Psychology, 67, 280-296.
46. Michael, J. C. (1968). The reliability of a multiple choice examination under various test-taking instructions. Journal of Educational Measurement, 5, 307-314.
47. Poizner, S. B., Nicewander, W. A., & Gettys, C. F. (1978). Alternative response and scoring methods for multiple-choice items: An empirical study of probabilistic and ordinal response modes. Applied Psychological Measurement, 2, 83-96.
48. Pugh, R. C., & Brunza, J. J. (1975). Effects of a confidence weighted scoring system on measures of test reliability and validity. Educational and Psychological Measurement, 35, 73-78.
49. Rippey, R. M. (1970). A comparison of five different scoring functions for confidence tests. Journal of Educational Measurement, 7, 165-170.
50. Shuford, E. H., Albert, A., & Massengill, H. E. (1966). Admissible probability measurement procedures. Psychometrika, 31, 125-145.
51. Smith, R. M. (1987). Assessing partial knowledge in vocabulary. Journal of Educational Measurement, 24, 217-231.
52. Stanley, J. C., & Wang, M. D. (1970). Weighting test items and test item options, an overview of the analytical and empirical literature. Educational and Psychological Measurement, 30, 21-35.
53. Suhadolnik, D., & Weiss, D. J. (1983). Effect of examinee certainty on probabilistic test scores and a comparison of scoring methods for probabilistic responses (Research Report 83-3). University of Minnesota, Department of Psychology, Psychometric Methods Program, Computerized Adaptive Testing Laboratory, Minneapolis.
54. Traub, R. D., & Hambleton, R. K. (1972). The effect of scoring instructions and the degree of speededness on the validity and reliability of multiple choice tests. Educational and Psychological Measurement, 32, 737-758.
55. Traub, R. D., Hambleton, R. K., & Singh, B. (1969). Effects of promised reward and threatened penalty on performance of a multiple choice vocabulary test. Educational and Psychological Measurement, 29, 847-861.
56. Wallsten, T. S., & Budescu, D. V. (1983). Encoding subjective probabilities: A psychological and psychometric review. Management Science, 29, 151-171.
57. Wallsten, T. S., Budescu, D. V., & Zwick, R. (1993). Comparing the calibration and coherence of numerical and verbal probabilistic judgments. Management Science, 39, 176-190.
58. Wang, M. W., & Stanley, J. C. (1970). Differential weighting: A review of methods and empirical studies. Review of Educational Research, 40, 663-705.
59. Wesman, A. G. (1971). Writing the test item. In R. L. Thorndike (Ed.), Educational measurement (pp. 81-129). Washington DC: American Council on Education.
60. Wilcox, R. R. (1981). Solving measurement problems with an answer until correct scoring procedure. Applied Psychological Measurement, 5, 399 -413.
61. Wilcox, R. R. (1982a). Some new results on an answer until correct scoring procedure. Journal of Educational Measurement, 19, 67-74.
62. Wilcox, R. R. (1982b). Some empirical and theoretical results on an answer until correct scoring procedure. British Journal of Mathematical and Statistical Psychology, 35, 57-70.
63. Wilcox, R. R. (1983). A simple model for diagnostic testing when there are several types of misinformation. Journal of Experimental Education, 52, 57-62.
64. Willey, C. E. (1960). The three-decision multiple choice test: A method of increasing the sensitivity of the multiple-choice item. Psychological Reports, 7, 475-477.
65. Wood, R. (1977). Multiple choice: A state of the art report. In B. Choppin & T. N. Postlewaite, Evaluation in education: International progress (pp. 191-280). Oxford: Pergamon.