The assessment of quantitative problem-solving skills with "none of the above"-items (NOTA items)

European Journal of Psychology of Education

Volumn 16, Issue 2, 2001, Pages 163-177

  Dochy, Filip ^a,b   Moerkerke, George ^c   De Corte, Erik ^a   Segers, Mien ^b  

^a UNIVERSITY OF LEUVEN   (Belgium)

^b MAASTRICHT UNIVERSITY   (Netherlands)

^c OPEN UNIVERSITY OF THE NETHERLANDS   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0035354515     PISSN: 02562928     EISSN: None     Source Type: Journal    
DOI: 10.1007/bf03173023     Document Type: Article

References (37)

1. Birenbaum, M., & Tatsuoka, K.K. (1987). Open-ended versus multiple-choice response formats: It does make a difference for diagnostic purposes. Applied Psychological Measurement, 11, 385-395.
2. Birenbaum, M., Tatsuoka, K.K., & Gutvirtz, Y. (1992). Effects of response format on diagnostic assessment of scholastic achievement. Applied Psychological Measurement, 16, 353-363.
3. Crehan, K.D., & Haladyna, T.M. (1991). The validity of two item-writing rules. Journal of Experimental Education, 59, 183-192.
4. Crehan, K.D., Haladyna, T.M., & Brewer, B.W. (1993). Use of an inclusive option and the optimal number of options for multiple-choice items. Educational and Psychological Measurement, 53, 241-247.
5. Dochy, F., & Mc Dowell, L. (1997). Assessment as a tool for learning. Studies in Educational Evaluation, 23, 279-298.
6. Dochy, F., & Moerkerke, G. (1997). The present, the past and the future of achievement testing and performance assessment. International Journal of Educational Research, 27, 415-432.
7. Dudycha, A.L., & Carpenter, J.B. (1973). Effects of item format on item discrimination and difficulty. Journal of Applied Psychology, 58, 116-121.
8. Forsyth, R.A., & Spratt, K.F. (1980). Measuring problem solving ability in mathematics with multiple-choice items: The effect of item format on selected item and test characteristics. Journal of Educational Measurement, 17, 31-43.
9. Frary, R.B. (1995). The none-of-the-above option: An empirical study. Applied Measurement in Education, 4, 115-124.
10. Garća-Pérez, M.A. (1993). In defense of "none of the above". British Journal of Mathematical and Statistical Psychology, 46, 213-229.
11. Gronlund, N.E. (1988). How to construct achievement tests (4th ed.). Englewood Cliffs, NJ: Prentice-Hall.
12. Gross, L.J. (1994). Logical versus empirical guidelines for writing test items: The case of "None of the Above". Evaluation and the Health Professions, 17,1, 123-126.
13. Haladyna, T.M., & Downing, S.M. (1989a). A taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 2, 37-50.
14. Haladyna, T.M., & Downing, S.M. (1989b). Validity of a taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 2, 51-78.
15. Horgan, J.D. (1978). Engineering reality in single-answer problems. IEEE Transactions on Education, Vol. E-27, No. 2, 65-68.
16. Kane, M.T. (1992). An argument-based approach to validity. Psychological Bulletin. 112, 527-535.
17. Knowles, S.L., & Welch, C.A. (1992). A meta-analytic review of item discrimination and difficulty in multiple-choice items using "none of the above". Educational and Psychological Measurement, 52, 571-577.
18. Kramers-Pals, H., & Pilot, A. (1988). Solving quantitative problems: Guidelines for teaching derived from research. International Journal of Science Education, 10, 511 -521.
19. Larkin, J., McDermott, J., Simon, D.P., & Simon, H.A. (1980). Expert and novice performance in solving physics problems. Science, 208, 1335-1342.
20. Leclercq, D.A. (1990). Confidence marking. In H.J. Walberg & G.D. Haertel (Eds.), The International encyclopedia of educational evaluation (pp. 343-345). Oxford: Pergamon Press.
21. Lord, F.M., & Novick, M.R. (1986). Statistical theories of mental test scores. Reading: Addison-Welsey.
22. Messick, S. (1989). Validity. In R.L. Linn (Ed.), Educational Measurement (3rd ed., pp. 13-103). New York: Macmillan.
23. Mettes, C.T.C.W., Pilot, A., & Roossink, H.J. (1981). Linking factual and procedural knowledge in solving science problems: A case study in a thermodynamics course. Instructional Science, 10, 333-361.
24. Mettes, C.T.C.W., Pilot, A., Roossink, H.J., & Kramers-Pals, H (1980). Teaching and learning problem solving in science, part I. Journal of Chemical Education, 57, 882-885.
25. Mettes, C.T.C.W., Pilot, A., Roossink, H.J., & Kramers-Pals, H (1981). Teaching and learning problem solving in science, part II. Journal of Chemical Education, 58, 51-55.
26. Millman, J., & Greene, J. (1989). The specification and development of tests of achievement and ability. In R.L. Linn (Ed.), Educational Measurement (3rd ed., pp. 335-366). New York: Macmillan Publishing Company.
27. Norris, S.P. (1991). Informal reasoning assessment: Using verbal reports of thinking to improve multiple-choice test validity. In J.F. Voss, D.N. Perkins & J.W. Segal (Eds.), Informal reasoning and education (pp. 451-472). Hillsdale, NJ: Erlbaum.
28. Oosterhof, A.C., & Coats, P.K. (1984). Comparison of difficulties and reliability of quantitative word problems in completion and multiple-choice formats. Applied Psychological Measurement, 8, 287-294.
29. Rich, C.E., & Johanson, G.A. (1990). An item-level analysis of "None-of-the-above". Paper presented at the annual meeting of the American Educational Research Association (Boston, MA, April 16-20).
30. Shepard, L.A. (1993). Evaluating test validity. In L. Darling-Hammond (Ed.), Review of Research in Education (vol. 19, pp. 405-450). Washington, DC: American Educational Research Association.
31. Simon, H.A. (1978). Information-processing theory of human problem solving. In W.K. Estes (Ed.), Handbook of learning and cognitive processes: Human information processing: Vol. 5. (pp. 271-295). Hillsdale, NJ: Lawrence Erlbaum Associates.
32. Tollefson, N. (1987). A comparison of the item difficulty and item discrimination of multiple-choice items using the "none of the above" and one correct response options. Educational and Psychological Measurement, 47, 377-383.
33. Tollefson, N., & Tripp, A. (1983). The effect of item formal on item difficulty and item discrimination. Paper presented at the Annual meeting of the American Educational Research Association, Montreal, Canada.
34. Traub, R.E. (1992). On the equivalence of the traits assessed by multiple-choice and constructed-response tests. In R. Bennet & W. Ward (Eds.), Construction versus choice in cognitive measurement: Issues in constructed response, performance testing, and portfolio assessment (pp. 236-256). Hillsdale, NJ: Lawrence Erlbaum Associates.
35. Traub, R.E., & MacRury, K. (1990). Antwort-Auswahl-vs. Freie-Antwort-Aufgaben bei Lernerfolgstests [Multiple-choice vs. free-response in the testing of scholastic achievement]. In K. Ingenkamp & R.S. Jäger (Eds.), Test und Trends 8 (pp. 128-156). Weinheim und Basel: Beltz Verlag.
36. Wesman, A.G. (1971). Writing the test item. In R.L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 81-129). Washington DC: American Council on Education.
37. Woods, D.R. (1987). How might I teach problem solving. In J.E. Stice (Ed.), Developing critical thinking and problem-solving abilities (pp. 55-71). San Francisco: Jossey-Bass.