Cognitive foundations of structured item response models

Cognitive Diagnostic Assessment for Education: Theory and Applications

Volumn , Issue , 2007, Pages 205-241

  Rupp, André A ^a   Mislevy, Robert J ^b  

^a HUMBOLDT UNIVERSITY   (Germany)

^b UNIVERSITY OF MARYLAND   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 64649101613     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1017/CBO9780511611186.008     Document Type: Chapter

References (95)

1. Adams, R., Wilson, M. R., & Wang, W.-C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1–23.
2. Almond, R. G., & Mislevy, R. J. (1999). Graphical models and computerized adaptive testing. Applied Psychological Measurement, 23, 223–237.
3. Anderson, J. R. (1995). Learning and memory: An integrated approach. New York: Wiley.
4. Anderson, J. R. (2000). Cognitive psychology and its implications(5th ed.). New York: Worth.
5. Anderson, J. R., & Gluck, K. A. (2001). What role do cognitive architectures play in intelligent tutoring systems? In S. M. Carver & D. Klahr (Eds.), Cognition and instruction: Twenty-five years of progress(pp. 227–261). Mahwah, NJ: Erlbaum.
6. Baumert, J., Artelt, C., Klieme, E., Neubrand, M., Prenzel, M., Schiefele, U., Schneider, W., Stanat, P., Tillmann, K.-J., & Weiß, M. (Eds.). (2001). [PISA 2000. Achievement of basic competencies for German students in an international comparison]. Opladen: Leske & Budrich.
7. Beaton, A. E., & Allen, N. (1992). Interpreting scales through scale anchoring. Journal of Educational Statistics, 17, 191–204.
8. Bechtel, W., Abrahamsen, A., & Graham, G. (1998). Part I. The life of cognitive science. In W. Bechtel & G. Graham (Eds.), A companion to cognitive science(pp. 2–104). Boston, MA: Blackwell.
9. Bolt, D., & Fu, J. (2004, April). A polytomous extension of the fusion model and its Bayesian parameter estimation. Paper presented at the annual meeting of the National Council on Measurement in Education, San Diego.
10. Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept of validity. Psychological Review, 111, 1061–1071.
11. Bradlow, E. T., Wainer, H., & Wang, X. (1999). A Bayesian random effects model for testlets. Psychometrika, 64, 153–168.
12. Carroll, J. B. (1976). Psychometric tests as cognitive tasks: A new structure of intellect. In L. B. Resnick (Ed.), The nature of intelligence(pp. 27–57). Hillsdale, NJ: Erlbaum.
13. Cizek, G. (2001). (Ed.). Setting performance standards: Concepts, methods and perspectives. Mahwah, NJ: Erlbaum.
14. Clark, A. (1997). Being there: Putting brain, body and world together again. Cambridge, MA: MIT Press.
15. Council of Europe (Europarat). (2001). Common European reference framework for languages. Accessed November 28, 2006, from www.coe.int/t/dg4/linguistic/CADRE˙EN.asp
16. de Boeck, P., & Wilson, M. (Eds.). (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York: Springer.
17. de la Torre, J., & Douglas, J. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333–353.
18. diBello, L.V., Stout, W. F., & Roussos, L. (1995). Unified cognitive psychometric assessment likelihood-based classification techniques. In P. D. Nichols, S. F. Chipman, & R. L. Brennan (Eds.), Cognitively diagnostic assessment(pp. 361–390). Hillsdale, NJ: Erlbaum.
19. Drum, P. A., Calfee, R. C., & Cook, L. K. (1981). The effects of surface structure variables on performance in reading comprehension tests. Reading Research Quarterly, 16, 486–514.
20. Edwards, C., & Willis, J. (Eds.). (2005). Teachers exploring tasks in English language teaching. Hampshire, England: Palgrave Macmillan.
21. Embretson, S. E. (1985). A general latent trait model for response processes. Psychometrika, 49, 175–186.
22. Embretson, S. E. (1994). Applications of cognitive design systems to test development. In C. R. Reynolds (Ed.), Cognitive assessment: A multidisciplinary perspective(pp. 107–135). New York: Plenum Press.
23. Enright, M. K., Grabe, W., Koda, K., Mosenthal, P., Mulcahy-Ernt, P., & Schedl, M. (2000). TOEFL 2000 reading framework: A working paper(TOEFL Monograph Series No. MS-17). Princeton, NJ: Educational Testing Service.
24. Fischer, G. H. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37, 359–374.
25. Fischer, G. H. (1997). Unidimensional linear logistic Rasch models. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory(pp. 221–224). New York: Springer Verlag.
26. Formann, A. K. (1985). Constrained latent class models: Theory and applications. British Journal of Mathematical and Statistical Psychology, 38, 87–111.
27. Freedle, R., & Kostin, I. (1993). The prediction of TOEFL reading item difficulty: Implications for construct validity. Language Testing, 10, 133–170.
28. Freidin, R. (1991). Principles and parameters in comparative grammar. Cambridge, MA: MIT Press.
29. Glaser, R., Lesgold, A., & Lajoie, S. (1987). Toward a cognitive theory for the measurement of achievement. In R. Ronning, J. Glover, J. C. Conoley, & J.Witt (Eds.), The influence of cognitive psychology on testing and measurement: The Buros-Nebraska Symposium on Measurement and Testing(Vol. 3; pp. 41–85). Hillsdale, NJ: Erlbaum.
30. Gorin, J. S., & Embretson, S. E. (2006). Predicting item properties without tryout: Cognitive modeling of paragraph comprehension items. Applied Psychological Measurement, 30(5), 394–411.
31. Greeno, J. G., & Moore, J. L. (1993). Situativity and symbols: Response to Vera and Simon. Cognitive Science, 17, 49–59.
32. Guttman, L. (1959). A structural theory for inter-group beliefs and action. American Sociological Review, 24, 318–328.
33. Guttman, L. (1970). Integration of test design and analysis. Proceedings of the 1969 Invitational Conference on Testing Problems(pp. 53–65). Princeton, NJ: Educational Testing Service.
34. Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 333–352.
35. Hartig, J., & Frey, A. (2005, July). Application of different explanatory item response models for model based proficiency scaling. Paper presented at the 14th international meeting of the Psychometric Society, Tilburg, The Netherlands.
36. Hartz, S. M. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. Unpublished doctoral dissertation, University of Illinois at Urbana–Champaign, Department of Statistics.
37. Hively, W., Patterson, H. L., & Page, S. H. (1968). A “universe-defined” system of arithmetic achievement tests. Journal of Educational Measurement, 5, 275–290.
38. Junker, B.W. (1999). Some statistical models and computational methods that may be useful for cognitively-relevant assessment. Unpublished manuscript. Accessed November 28, 2006, from http://www.stat.cmu.edu/∼brian/nrc/cfa
39. Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.
40. Klein, M. F., Birenbaum, M., Standiford, S. N., & Tatsuoka, K. K. (1981). Logical error analysis and construction of tests to diagnose student “bugs” in addition and subtraction of fractions(Research Report 81–6). Urbana: Computer-Based Education Research Laboratory, University of Illinois.
41. KMK. (2003). [National educational standards for mathematics at the lower secondary level (Mittlerer Schulabschluss)]. Munich, Germany: Wolters Kluwe Deutschland.
42. KMK. (2004). [National educational standards for mathematics at the lower secondary level (Hauptschulabschluss)]. Munich, Germany: Wolters Kluwe Deutschland.
43. Levy, R., & Mislevy, R. J. (2004). Specifying and refining a measurement model for a computer-based interactive assessment. International Journal of Testing,4, 333–369.
44. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
45. Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika, 64, 187–212.
46. Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13–23.
47. Mislevy, R. J. (1996). Test theory reconceived. Journal of Educational Measurement, 33, 379–416.
48. Mislevy, R. J., & Verhelst, N. (1990). Modeling item responses when different subjects employ different solution strategies. Psychometrika, 55, 195–215.
49. Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1, 3–67.
50. Mosenthal, P. B. (1996). Understanding the strategies of document literacy and their conditions of use. Journal of Educational Psychology, 88, 314–332.
51. Mullis, I. V. S., Martin, M. O., & Foy, P. (2005). IEA’s TIMSS 2003 international report on achievement in the mathematics domain: Findings from a developmental project. Chestnut Hill, MA: TIMSS&PIRLS International Study Center, Boston College.
52. Muthén, B. O. (2002). Beyond SEM: General latent variable modeling. Behaviourmetrika, 29, 81–117.
53. National Research Council. (2000). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.
54. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice Hall.
55. Nichols, P. (1994). A framework for developing cognitively diagnostic assessments. Review of Educational Research, 64, 575–603.
56. Nichols, P. D., Chipman, S. F., & Brennan, R. L. (Eds.). (1995). Cognitively diagnostic assessment. Hillsdale, NJ: Erlbaum.
57. North, B., & Schneider, G. (1998). Scaling descriptors for language proficiency scales. Language Testing, 15, 217–262.
58. Nunan, D., & Swan, M. (2004). Task-based language teaching. Cambridge, UK: Cambridge University Press.
59. OECD. (2003). PISA 2003 assessment framework: Mathematics, reading, science and problem solving knowledge and skills. Retrieved November 28, 2006, from www.pisa.oecd.org/dataoecd/46/14/33694881.pdf
60. Patz, R. J., & Junker, B.W. (1999). Applications and extensions of MCMC in IRT: Multiple item types, missing data, and rated responses. Journal of Educational and Behavioural Statistics, 24, 342–366.
61. Rasch, G. (1960/1980). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danish Institute for Educational Research. (reprint: Chicago: University of Chicago Press).
62. Rost, J. (2004). [Psychometric models to assess the attainment of national educational standards with models of competency]. Zeitschrift für Pädagogik, 50(5), 662–679.
63. Rumelhart, D. A. (1980). Schemata: The building blocks of cognition. In R. Spiro, B. Bruce, & W. Brewer (Eds.), Theoretical issues in reading comprehension(pp. 33–58). Hillsdale, NJ: Erlbaum.
64. Rupp, A. A. (2002). Feature selection for choosing and assembling measurement models: a building-block-based organization. International Journal of Testing, 2, 311–360.
65. Rupp, A. A. (in press). The answer is in the question: A guide for investigating and describing the theoretical foundations and statistical properties of cognitive psychometric models. International Journal of Testing.
66. Rupp, A. A., Garcia, P., & Jamieson, J. (2001). Combining multiple regression and CART to understand difficulty in second language reading and listening comprehension test items. International Journal of Testing, 1, 185–216.
67. Scheiblechner, H. (1972). [The learning and solution of complex cognitive tasks.] Zeitschrift für experimentalle und Angewandte Psychologie, 19, 476–506.
68. Schraagen, J. M., Chipman, S. F., & Shalin, V. J. (2000). Cognitive task analysis. Mahwah, NJ: Erlbaum.
69. Schum, D. A. (1994). The evidential foundations of probabilistic reasoning. New York: Wiley.
70. Sheehan, K. M., & Ginther, A. (2001). What do passage-based MC verbal reasoning items really measure? An analysis of the cognitive skills underlying performance on the current TOEFL reading section. Paper presented at the 2000 annual meeting of the National Council of Measurement in Education, New Orleans, LA.
71. Sheehan, K. M., & Mislevy, R. J. (2001). An inquiry into the nature of the sentence completion task: Implications for item generation(GRE Board Report No. 95–17bP). Princeton, NJ: Educational Testing Service.
72. Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal, and structural equation modeling. Boca Raton, FL: Sage.
73. Snow, R. E., & Lohman, D. F. (1989). Implications of cognitive psychology for educational measurement. In R. L. Linn (Ed.), Educational measurement(3rd ed., pp. 263–331). New York: American Council on Education/Macmillan.
74. Steinberg, L. S., & Gitomer, D. G. (1996). Intelligent tutoring and assessment built on an understanding of a technical problem-solving task. Instructional Science, 24, 223–258.
75. Sternberg, R. J. (1977). Intelligence, information processing and analogical reasoning: The componential analysis of human abilities. New York: Wiley.
76. Suppes, P., & Morningstar, M. (1972). Computer-assisted instruction at Stanford, 1966–68: Data, models, and evaluation of the arithmetic programs. New York: Academic Press.
77. Tatsuoka, K. K. (1983). Rule-space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345–354.
78. Tatsuoka, K. K. (1995). Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. In P. D. Nichols, S. F. Chipman, & R. L. Brennan (Eds.), Cognitively diagnostic assessment(pp. 327–360). Hillsdale, NJ: Erlbaum.
79. Templin, J. L., He, X., Roussos, L., & Stout, W. (2003). The pseudo-item method: A simple technique for analysis of polytomous data with the fusion model. Unpublished technical report draft for the ETS External Diagnostic Group. Department of Psychology, University of Illinois at Urbana-Champaign, IL.
80. Templin, J. L., & Henson, R. A. (2004). Creating a proficiency scale with models for cognitive diagnosis.Unpublished technical report for the ETS External Diagnostic Group. Department of Psychology, University of Illinois at Urbana-Champaign, IL.
81. Templin, J. L., & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287–305.
82. Templin, J. L., & Henson, R. A. (2005). The random effects reparametrized unified model: A model for joint estimation of discrete skills and continuous ability. Manuscript submitted for publication.
83. Templin, J. L., Roussos, L., & Stout, W. (2003). An extension of the current fusion model to treat polytomous attributes. Unpublished Technical Report for the ETS External Diagnostic Group. Department of Psychology, University of Illinois at Urbana-Champaign, IL.
84. Toulmin, S. E. (1958). The uses of argument.Cambridge, UK: Cambridge University Press.
85. Underwood, G., & Bait, V. (1996). Reading and understanding. Cambridge, MA: Blackwell.
86. van der Linden, W. J., & Hambleton, R. K. (1997). Handbook of modern item response theory. New York: Springer-Verlag.
87. van Lehn, K. (1989). Problem-solving and cognitive skill acquisition. In M. Posner (Ed.), The foundations of cognitive science(pp. 527–580). Cambridge, MA: MIT Press.
88. van Lehn, K. (1990). Mind bugs: The origins of procedural misconceptions. Cambridge, MA: MIT Press.
89. van Lehn, K. (2004, October). The Andes intelligent physics tutoring system: Lessons learned about assessment. Paper presented at the 4th ETS Spearman conference, Philadelphia.
90. von Davier, M. (2005). A general diagnostic model applied to language testing data(Research Report No. RR–05–16). Princeton, NJ: Educational Testing Service.
91. Wainer, H., & Wang, X. (2001). Using a new statistical model for testlets to score TOEFL(Technical Report No. TR-16). Princeton, NJ: Educational Testing Service.
92. Wang, X., Bradlow, E. T., & Wainer, H. (2002). A general Bayesian model for testlets: Theory and applications(Research Report No. 02–02). Princeton, NJ: Educational Testing Service.
93. Wilson, M. R. (1989). Saltus: A psychometric model of discontinuity in cognitive development. Psychological Bulletin, 105, 276–289.
94. Wilson, M. R., & Adams, R. J. (1995). Rasch models for item bundles. Psychometrika, 60, 181–198.
95. Yan, D., Mislevy, R. J., & Almond, R. G. (2003). Design and analysis in a cognitive assessment(Research Report No. RR-03–32). Princeton, NJ: Educational Testing Service.