Verbal reports as data for cognitive diagnostic assessment

Cognitive Diagnostic Assessment for Education: Theory and Applications

Volumn , Issue , 2007, Pages 146-172

  Leighton, Jacqueline P ^a   Gierl, Mark J ^a  

^a UNIVERSITY OF ALBERTA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 36348929681     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1017/CBO9780511611186.006     Document Type: Chapter

References (49)

1. Anderson, J. R. (1990). Cognitive psychology and its implications. New York:W. H. Freeman.
2. Baddeley, A. D. (1986). Working memory. Oxford, UK: Oxford University Press.
3. Begg, I., & Harris, G. (1982). On the interpretation of syllogisms. Journal of Verbal Learning and Verbal Behavior, 21, 595–620.
4. Byrne, R. M. (1989). Suppressing valid inferences with conditionals. Cognition, 31, 61–83.
5. Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. Journal of the Learning Sciences, 6, 271–315.
6. Chi, M. T. H., de Leeuw, N., Chiu, M. H., & La Vancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439–477.
7. Chi, M. T. H., & VanLehn, K. A. (1991). The content of physics self-explanations. Journal of the Learning Sciences, 1, 69–105.
8. Dawson, M.R.W. (1998). Understanding cognitive science. Malden, MA:Blackwell.
9. Desimone, L. M., & LeFloch, K. C. (2004). Are we asking the right questions? Using cognitive interviews to improve surveys in education research. Educational Evaluation and Policy Analysis, 26, 1–22.
10. Embretson, S., & Gorin, J. (2001). Improving construct validity with cognitive psychology principles. Journal of Educational Measurement, 38, 343–368.
11. Ericsson, K. A., & Simon, H. A. (1980).Verbal reports as data. Psychological Review, 87, 215–251.
12. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis. Cambridge, MA: MIT Press.
13. Evans, J. St. B. T., Handley, S. J., Harper, C. N. J., & Johnson-Laird, P. N. (1999). Reasoning about necessity and possibility: A test of the mental model theory of deduction. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 1495–1513.
14. Frederiksen, N., Glaser, R., Lesgold, A., & Shafto, M. G. (Eds.). (1990). Diagnostic monitoring of skill and knowledge acquisition. New Jersey: Lawrence Erlbaum Associates.
15. Galotti, K. M., Baron, J., & Sabini, J. P. (1986). Individual differences in syllogistic reasoning: Deduction rules or mental models? Journal of Experimental Psychology: General, 115, 16–25.
16. Gierl, M. J., Leighton, J. P., & Hunka, S. (2000). Exploring the logic of Tatsuoka’s rule-space model for test development and analysis. Educational Measurement: Issues and Practice, 19, 34–44.
17. Girotto, V. (2004). Task understanding. In J. P. Leighton & R. J. Sternberg (Eds.), Nature of reasoning(pp. 103–128). New York: Cambridge University Press.
18. Hamilton, L. S., Nussbaum, E. M., & Snow, R. E. (1997). Interview procedures for validating science assessments. Applied Measurement in Education, 10, 181–200.
19. Johnson-Laird, P. N. (1983). Mental models. Towards a cognitive science of language, inference, and consciousness. Cambridge, MA: Harvard University Press.
20. Johnson-Laird, P. N. (2004). Mental models and reasoning. In J. P. Leighton & R. J. Sternberg (Eds.), Nature of reasoning(pp. 169–204). Cambridge, UK: Cambridge University Press.
21. Johnson-Laird, P. N., & Bara, B. G. (1984). Syllogistic inference. Cognition,16, 1–61.
22. Katz, I. R., Bennett, E., & Berger, A. E. (2000). Effects of response format on difficulty of SAT-Mathematics items: It’s not the strategy. Journal of Educational Measurement, 37, 39–57.
23. Leighton, J. P. (2004). Avoiding misconceptions, misuse, and missed opportunities: The collection of verbal reports in educational achievement testing. Educational Measurement: Issues and Practice, Winter, 1–10.
24. Leighton, J. P. (2005). Teaching and assessing deductive reasoning skills. Journal of Experimental Education, 74, 109–136.
25. Leighton, J. P., & Gierl, M. J. (in press). Defining and evaluating models of cognition used in educational measurement to make inferences about examinees’ thinking processes. Educational Measurement: Issues and Practice.
26. Leighton, J. P., Gierl, M. J., & Hunka, S. (2004). The attribute hierarchy model: An approach for integrating cognitive theory with assessment practice. Journal of Educational Measurement, 41, 205–236.
27. Leighton, J. P., & Gokiert, R. (2005a, April). The cognitive effects of test item features: Identifying construct irrelevant variance and informing item generation. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal.
28. Leighton, J. P., & Gokiert, R. (2005b, April). Investigating test items designed to measure higher-order reasoning using think-aloud methods. Paper presented at the annual meeting of the American Educational Research Association (AERA), Montreal.
29. Lohman, D. F. (2000). Complex information processing and intelligence. In R. J. Sternberg (Ed.), Handbook of intelligence(pp. 285–340). New York: Cambridge University Press.
30. Lukin, L. E., Bandalos, D. L., Eckhout, T. J., & Mickelson, K. (2004). Facilitating the development of assessment literacy. Educational Measurement: Issues and Practice, 23, 26–32.
31. Millman, J., & Greene, J. (1989). The specification and development of tests of achievement and ability. In R. L. Linn (Ed.), Educational measurement(3rd ed., pp. 335–366). New York: American Council of Education/Macmillan.
32. Mislevy, R. J. (1996). Test theory reconceived. Journal of Educational Measurement, 33, 379–416.
33. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, New Jersey: Prentice Hall.
34. Nichols, P. (1994). A framework of developing cognitively diagnostic assessments. Review of Educational Research, 64, 575–603.
35. Nichols, P. D., Chipman, S. F., & Brennan, R. L. (Eds.). (1995). Cognitively diagnostic assessment. Hillsdale, NJ: Erlbaum.
36. Nisbett, R., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231–259.
37. Norris, S. P. (1990). Effect of eliciting verbal reports of thinking on critical thinking test performance. Journal of Educational Measurement, 27, 41–58.
38. Payne, J. W., Braunstein, M. L., & Carroll, J. S. (1978). Exploring predecisional behavior: An alternative approach to decision research. Organizational Behavior and Human Performance, 22, 17–44.
39. Pellegrino, J. W., Baxter, G. P., & Glaser, R. (1999). Addressing the “Two Disciplines” problem: Linking theories of cognition and learning with assessment and instructional practice. Review of Research in Education, 24, 307–353.
40. Pressley, M., & Afflerbach, P. (1995). Verbal protocols of reading: The nature of constructively responsive reading. Hillsdale, NJ: Erlbaum.
41. Roberts, M. J. (2004). Heuristics and reasoning I: Making deduction simple. In J. P. Leighton & R. J. Sternberg (Eds.), Nature of reasoning(pp. 234–272). New York: Cambridge University Press.
42. Russo, J. E., Johnson, E. J., & Stephens, D. L. (1989). The validity of verbal protocols. Memory & Cognition,17, 759–769.
43. Snow, R. E., & Lohman, D. F. (1989). Implications of cognitive psychology for educational measurement. In R. L. Linn (Ed.), Educational measurement(3rd ed., pp. 263–331). New York: American Council on Education/Macmillan.
44. Sternberg, R. J. (1990). Metaphors of mind: Conceptions of the nature of intelligence. Cambridge, UK: Cambridge University Press.
45. Tatsuoka, K. K. (1983). Rule space: Anapproach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345–354.
46. Tatsuoka, K.K. (1990). Toward an integration of item-response theory and cognitive error diagnosis. In N. Frederiksen, R. Glaser, A. Lesgold, & M. Shafto (Eds.), Diagnostic monitoring of skill and knowledge acquisition(pp. 453–488). Hillsdale, NJ: Erlbaum.
47. Taylor, K. L., & Dionne, J-P. (2000). Accessing problem-solving strategy knowledge: The complementary use of concurrent verbal protocols and retrospective debriefing. Journal of Educational Psychology, 92, 413–425.
48. Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, CA: Sage.
49. Wilson, T. D. (1994). The proper protocol: Validity and completeness of verbal reports. Psychological Science, 5, 249–252.