The attribute hierarchy method for cognitive assessment: A variation on Tatsuoka's rule-space approach

Journal of Educational Measurement

Volumn 41, Issue 3, 2004, Pages 205-237

  Leighton, Jacqueline P ^a   Gierl, Mark J ^a   Hunka, Stephen M ^a  

^a UNIVERSITY OF ALBERTA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 6344237380     PISSN: 00220655     EISSN: None     Source Type: Journal    
DOI: 10.1111/j.1745-3984.2004.tb01163.x     Document Type: Article

References (72)

1. Baxter, G. P., & Glaser, R. (1998). Investigating the cognitive complexity of science assessments. Educational Measurement: Issues and Practice, 17, 37-45.
2. Birenbaum, M., Kelly, A. E., & Tatsuoka, K. K. (1993). Diagnosing knowledge states in algebra using the rule-space model. Journal for Research in Mathematics Education, 24, 442-459.
3. Birenbaum, M., & Shaw, D. J. (1985). Task specification chart: A key to a better understanding of test results. Journal of Educational Measurement, 22, 219-230.
4. Buck, G., & Tatsuoka, K. K. (1998). Application of the rule-space procedure to language testing: Examining attributes of a free response listening test. Language Testing, 15, 119-157.
5. Chipman, S. F., Nichols, P. D., & Brennan, R. L. (1995). Introduction. P. D. Nichols, S. F. Chipman, & R. L. Brennan (Eds.), Cognitively diagnostic assessment (pp. 1-18). Hillsdale, NJ: Erlbaum.
6. Embretson, S. E. (1983). Construct validity: Construct representation versus nomothetic span. Psychological Bulletin, 93, 179-197.
7. Embretson, S. E. (Ed.). (1985). Test design: Developments in psychology and psychometrics. Orlando, FL: Academic Press.
8. Embretson, S. E. (1994). Application of cognitive design systems to test development. In C. R. Reynolds (Ed.), Cognitive assessment: A multidisciplinary perspective (pp. 107-135). New York: Plenum Press.
9. Embretson, S. E. (1995a). Developments toward a cognitive design system for psychological testing: In D. Lupinsky & R. Dawis (Eds.), Assessing individual differences in human behavior (pp. 17-48). Palo Alto, CA: Davies-Black Publishing.
10. Embretson, S. E. (1995b). A measurement model for linking individual learning to processes and knowledge: Application to mathematical reasoning. Journal of Educational Measurement, 32, 277-294.
11. Embretson, S. E. (1998). A cognitive design system approach to generating valid tests: Application to abstract reasoning. Psychological Methods, 3, 380-396.
12. Embretson, S. E. (1999). Cognitive psychology applied to testing. In F. T. Durso, R. S. Nickerson, R. W., Schvaneveldt, S. T. Dumais, D. S. Lindsay, & M. T. H. Chi (Eds.), Handbook of Applied Cognition (pp. 629-660). New York: Wiley.
13. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Hillsdale, NJ: Erlbaum.
14. Embretson, S. E., Schneider, L. M., & Roth, D. L. (1985). Multiple processing strategies and the construct validity of verbal reasoning tests. Journal of Educational Measurement, 23, 13-32.
15. Embretson, S. E., & Wetzel, D. (1987). Component latent trait models for paragraph comprehension tests. Applied Psychological Measurement, 11, 175-193.
16. Ericsson, K. A., & Simon, H. A. (1993). Protocol analyses: Verbal reports as data (Rev. ed.). Cambridge, MA: MIT Press.
17. Evans, J. St. B. T., Handley, S. J., Harper, C. N. J., & Johnson-Laird, P. N. (1999). Reasoning about necessity and possibility: A test of the mental model theory of deduction. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 1495-1513.
18. Evans, J. St. B. T., Newstead, S. E., & Byrne, R. M. (1993). Human reasoning: The psychology of deduction. Hillsdale, NJ: Erlbaum.
19. Frederiksen, N., Glaser, R. L., Lesgold, A. M., & Shafto, M. G. (1990). Diagnostic monitoring of skills and knowledge acquisition. Hillsdale, NJ: Erlbaum.
20. Frederiksen, N., Mislevy, R. J., Bejar, I. I. (1993). Test theory for a new generation of tests. Hillsdale, NJ: Erlbaum.
21. Gierl, M. J. (1997). Comparing the cognitive representations of test developers and students on a mathematics achievement test using Bloom's taxonomy. Journal of Educational Research, 91, 26-32.
22. Gierl, M. J., Leighton, J. P., & Hunka, S. (2000). Exploring the logic of Tatsuoka's rule-space model for test development and analysis. Educational Measurement: Issues and Practice, 19, 34-44.
23. Grice, H. P. (1975). Logic and conversation. In P. Cole & J. Morgan (Eds.), Syntax and semantics volume 3: Speech acts (pp. 41-58). London: Academic Press.
24. Hamilton, L. S., Nussbaum, M., & Snow, R. E. (1997). Interview procedures for validating science assessments. Applied Measurement in Education, 10, 181-200.
25. Hattie, J., Jaeger, R. M., & Bond, L. (1999). Persistent methodological questions in educational testing. Review of Research in Education, 24, 393-446.
26. Irvine, S. H., & Kyllonen, P. C. (Eds.). (2002). Item generation for test development. Mahwah, NJ: Erlbaum.
27. Johnson-Laird, P. N. (1983). Mental models. Towards a cognitive science of language, inference, and consciousness. Cambridge, MA: Harvard University Press.
28. Johnson-Laird, P. N. (1999). Deductive reasoning. Annual Review of Psychology, 50, 109-135.
29. Johnson-Laird, P. N., & Bara, B. G. (1984). Syllogistic inference. Cognition, 16 1-61.
30. Johnson-Laird, P. N., & Byrne, R. M. J. (1991). Deduction. Hillsdale, NJ: Lawrence Erlbaum.
31. Kim, S. (2001). Towards a statistical foundation in combining structures of decomposable graphical models (Research Report No. 01-2). Yusong gu, Taejon: Korea Advanced Institute of Science and Technology, Division of Applied Mathematics.
32. Kuhn, D. (2001). Why development does (and does not occur) occur: Evidence from the domain of inductive reasoning. In J. L. McClelland & R. Siegler (Eds.), Mechanisms of cognitive development: Behavioral and neural perspectives (pp. 221-249). Hillsdale, NJ: Erlbaum.
33. Leighton, J., Gierl, M. J., & Hunka, S. (1999, April). Attributes in Tatsuoka's rule-space model. Poster presented at the annual meeting of the National Council on Measurement in Education, Montréal, Quebec, Canada.
34. Leighton, J. P., Rogers, W. T., & Maguire, T. O. (1999). Assessment of student problem-solving on ill-defined tasks. Alberta Journal of Educational Research, 45, 409-427.
35. Leighton, J. P., & Sternberg, R. J. (2003). Reasoning and problem solving. In A. F. Healy & R. W. Proctor (Eds.), Experimental Psychology (pp. 623-648). Volume 4 in I. B. Weiner (Editor-in-Chief) Handbook of psychology. New York: Wiley.
36. Leighton, J. P., & Sternberg, R. J. (2003). Reasoning and problem solving. In A. F. Healy & R. W. Proctor (Eds.), Experimental Psychology (pp. 623-648). Volume 4 in I. B. Weiner (Editor-in-Chief) Handbook of psychology. New York: Wiley.
37. Magone, M., Cai, J., Silver, E. A., & Wang, N. (1994). Validating the cognitive complexity and content quality of mathematics performance assessment. International Journal of Educational Research, 21, 317-340.
38. Meijer, R. R. (1996). Person-fit research: An introduction. Applied Measurement in Education, 9, 3-8.
39. Meijer, R. R., & Sijtsma, K. (2001). Methodological review: Evaluating person fit. Applied Psychological Measurement, 25, 107-135.
40. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). New York: American Council on Education, Macmillan.
41. Mislevy, R. J. (1996). Test theory reconceived. Journal of Educational Measurement, 33, 379-416.
42. Mislevy, R. J., & Bock, R. D. (1990). BILOG 3: Item analysis and test scoring with binary logistic test models [Computer Program]. Moorseville, IN: Scientific Software.
43. National Research Council (2001). Knowing what students know: The science and design of educational assessment. Committee on the Foundations of Assessment. J. Pellegrino, N. Chudowsky, and R. Glaser (Eds.). Board on Testing and Assessment, Center for Education. Washington, DC: National Academy Press.
44. Nichols, P. (1994). A framework of developing cognitively diagnostic assessments. Review of Educational Research, 64, 575-603.
45. Nichols, P. D., Chipman, S. F., & Brennan, R. L. (1995). Cognitively diagnostic assessment. Hillsdale, NJ: Erlbaum.
46. Nichols, P., & Sugrue, B. (1999). The lack of fidelity between cognitively complex constructs and conventional test development practice. Educational Measurement: Issues and Practice, 18, 18-29.
47. Norris, S. P. (1990). Effect of eliciting verbal reports of thinking on critical thinking test performance. Journal of Educational Measurement, 27, 41-58.
48. Pellegrino, J. W. (1988). Mental models and mental tests. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 49-60). Hillsdale, NJ: Erlbaum.
49. Pellegrino, J. W., Baxter, G. P., & Glaser, R. (1999). Addressing the "two disciplines" problem: Linking theories of cognition and learning with assessment and instructional practices. In A. Iran-Nejad & P. D. Pearson (Eds.), Review of Research in Education (pp. 307-353). Washington, DC: American Educational Research Association.
50. Roussos, L. (1994). Summary and review of cognitive diagnosis models. Unpublished manuscript, University of Illinois, Urbana-Champaign, The Statistical Laboratory for Educational and Psychological Measurement.
51. Royer, J. M., Cisero, C. A., & Carlo, M. S. (1993). Techniques and procedures for assessing cognitive skills. Review of Educational Research, 63, 201-243.
52. Schaeken, W., De Vooght, G., Vandierendonck, A., & d'Ydewalle, G. (Eds.). (2000). Deductive reasoning and strategies. Mahwah, NJ: Lawrence Erlbaum Associates.
53. Scriven, M. (1991). Evaluation thesaurus (4th ed.). Newbury Park, CA: Sage.
54. Snow, R. E., & Lohman, D. F. (1989). Implications of cognitive psychology for educational measurement. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 263-331). New York: American Council on Education, Macmillan.
55. Sternberg, R. J. (1977). Intelligence, information processing, and analogical reasoning. Hillsdale, NJ: Erlbaum.
56. Tatsuoka, K. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345-354.
57. Tatsuoka, K. K. (1984). Caution indices based on item response theory. Psychometrika, 49, 95-110.
58. Tatsuoka, K. K. (1990). Toward an integration of item-response theory and cognitive error diagnosis. In N. Fredrickson, R. L. Glaser, A. M. Lesgold, & M. G. Shafto (Eds.), Diagnostic monitoring of skills and knowledge acquisition (pp. 453-488). Hillsdale, NJ: Erlbaum.
59. Tatsuoka, K. K. (1991). Boolean algebra applied to determination of universal set of knowledge states (Tech. Rep. No. RR-91-44-ONR). Princeton, NJ: Educational Testing Service.
60. Tatsuoka, K. K. (1993). Item construction and psychometric models appropriate for constructed responses. In R. E. Bennett & W. C. Ward (Eds.), Construction versus choice in cognitive measurement (pp. 107-133). Hillsdale, NJ: Erlbaum.
61. Tatsuoka, K. K. (1995). Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. In P. D. Nichols, S. F. Chipman, & R. L. Brennan (Eds.), Cognitively diagnostic assessment (pp. 327-359). Hillsdale, NJ: Erlbaum.
62. Tatsuoka, K. K. (1996). Use of generalized person-fit indexes, Zetas for statistical pattern classification. Applied Measurement in Education, 9, 65-75.
63. Tatsuoka, K. K., & Boodoo, G. (2000). Subgroup differences on the GRE quantitative test based on the underlying cognitive processes and knowledge. In D. Lesh & W. E. Kelly (Eds.), Research design and methodologies for mathematics and science. Hillsdale, NJ: Erlbaum.
64. Tatsuoka, M. M. (1986). Graph theory and its applications in educational research: A review and integration. Review of Educational Research, 56, 291-329.
65. Tatsuoka, M. M., & Tatsuoka, K. K. (1989). Rule space. In S. Kotz & N. L. Johnson (Eds.), Encyclopedia of statistical sciences (pp. 217-220). New York: Wiley.
66. Thissen, D., Steinberg, L., & Fitzpatrick, A. R. (1989). Multiple choice models: The distractors are also part of the item. Journal of Educational Measurement, 26, 161-176.
67. van der Linden, W., & Hambleton, R. K. (Eds.). (1997). Handbook of modern item response theory. New York: Springer.
68. van Essen, T. (2001, April). Developing and presenting enhanced skill descriptors for the PSAT/NMSQT. Paper presented at the annual meeting of the National Council on Measurement in Education, Seattle, WA.
69. Varadi, F., & Tatsuoka, K. K. (1989). BUGLIB [Computer program]. Trenton, NJ: Tanar Software.
70. Varadi, F., & Tatsuoka, K. K. (1992). BUGLIB Modified version [Computer program]. Trenton, NJ: Tanar Software.
71. Vosniadou, S., & Brewer, W. F. (1992). Mental models of the earth: A study of conceptual change in childhood. Cognitive Psychology, 24, 535-585.
72. Wolfram, S. (1996). The mathematica book (3rd ed.). Cambridge: Cambridge University Press.