Developing Score Reports for Cognitive Diagnostic Assessments

Educational Measurement: Issues and Practice

Volumn 29, Issue 3, 2010, Pages 25-38

  Roberts, Mary Roduta ^a   Gierl, Mark J ^a  

^a UNIVERSITY OF ALBERTA   (Canada)

Author keywords

Cognitive diagnostic assessment; Design principles; Score reporting

Indexed keywords

EID: 77957116847     PISSN: 07311745     EISSN: 17453992     Source Type: Journal    
DOI: 10.1111/j.1745-3992.2010.00181.x     Document Type: Article

References (61)

1. Standards for educational and psychological testing 1999, American Educational Research Association (AERA), American Psychological Association, National Council on Measurement in Education, Washington, DC, AERA
2. Aschbacher PR, Herman J L. Guidelines for effective score reporting. 1991, CSE Technical Report 326, Los Angeles, CA, National Center for Research on Evaluation, Standards and Student Testing
3. Beaton A E, Allen N L. Interpreting scales through scale anchoring. Journal of Educational Statistics 1992, 17:191-204.
4. Bertin J. The semiology of graphics 1983, Madison, WI, University of Wisconsin Press
5. The principles of universal design http://www.design.ncsu.edu/cud/about_ud/udprinciplestext.htm, Center for Universal Design, n.d., Accessed March 3, 2008 at
6. Cleveland W S. The elements of graphing data 1994, Monterey, CA, Wadsworth
7. Cleveland W S, McGill R. Graphical perception: Theory, experimentation, and application to the development of graphical methods. Journal of American Statistical Association 1985, 79:531-554.
8. de la Torre J, Douglas J. Higher-order latent trait models for cognitive diagnosis. Psychometrika 2004, 69:333-353.
9. DiBello L, Stout W, Roussos L. Unified cognitive/psychometric diagnostic assessment likelihood-based classification techniques. Cognitively diagnostic assessment 1995, 361-389. Nichols P, Chipman S F, Brennan R L. Eds., Hillsdale, NJ, Erlbaum
10. Ehrenberg A S C. Rudiments of numeracy. Journal of the Royal Statistical Society 1977, 140:277-297.
11. Forsyth R A. Do NAEP scales yield valid criterion-referenced interpretations?. Educational Measurement: Issues and Practice 1991, 10:16-9.
12. Forte Fast E. A guide to effective accountability reporting 2002, Washington, DC, Council of Chief State School Officers
13. Gierl M J, Cui Y, Hunka S M. Using connectionist models to evaluate examinees' response patterns on tests using the Attribute Hierarchy Method. 2007, April, Paper presented at the meeting of the National Council on Measurement in Education, Chicago, IL
14. Gierl M J, Cui Y, Zhou J. Reliability and attribute-based scoring in cognitive diagnostic assessment. Journal of Educational Measurement 2009, 46:293-313.
15. Gierl M J, Cui Y, Hunka S M. Using connectionist models to evaluate examinees' response patterns on tests. Journal of Modern Applied Statistical Methods 2008, 7:234-245.
16. Gierl M J, Wang C, Zhou J. Using the Attribute Hierarchy Method to make diagnostic inferences about examinees' cognitive skills in algebra on the SAT. Journal of Technology, Learning, and Assessment 2008, 6(6):1-49. http://www.jtla.org, Accessed January 28, 2008 at
17. Goodman D P, Hambleton R K. Student test score reports and interpretative guides: Review of current practices and suggestions for future research. Applied Measurement in Education 2004, 17:145-220.
18. Gribbons W M. Organization by design: Some implications for structuring information. Journal of Technical Writing and Communication 1992, 22:57-75.
19. Haertel E H. Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement 1989, 26:333-352.
20. Hambleton R K, Slater S. Are NAEP executive summary reports understandable to policy makers and educators?. 1997, CSE Technical Report 430, Los Angeles, CA, National Center for Research on Evaluation, Standards and Student Teaching
21. Hartley J, Jonassen D H. The role of headings in printed and electronic text: The technology of text 1985, Englewood Cliffs, NJ, Educational Technology Publications
22. Horton W. Overcoming chromophobia: A guide to the confident and appropriate use of color. IEEE Transactions on Professional Communication 1991, 34:160-173.
23. Huff K, Goodman D P. The demand for cognitive diagnostic assessment. Cognitive diagnostic assessment for education: Theory and applications 2007, 19-60. Leighton J P, Gierl M J. Eds., Cambridge, United Kingdom, Cambridge University Press
24. Impara J C, Divine K P, Bruce F A, Liverman M R, Gay A A. Does interpretive test score information help teachers?. Educational Measurement: Issues and Practice 1991, 10:16-18.
25. Jaeger R M. Reporting the results of the National Assessment of Educational Progress 1998, NVS NAEP Validity Studies, Washington, DC, American Institutes for Research
26. Jang E. Cognitive diagnostic assessment of L2 reading comprehension ability: Validity arguments for Fusion Model application to LanguEdge assessment. Language Testing 2009, 26:31-73.
27. The technology of text: Principles for structuring, designing, and displaying text 1983, Jonassen D H. Ed., Englewood Cliffs, NJ, Educational Technology Publications
28. Junker B W, Sijtsma K. Cognitive assessment models with few assumptions and connections with nonparametric item response theory. Applied Psychological Measurement 2001, 12:55-73.
29. Koretz D M, Diebert E. Interpretations of National Assessment of Educational Progress (NAEP) anchor points and achievements levels by the print media in 1991 1993, Santa Monica, CA, RAND
30. Kosslyn S M. Elements of graph design 1994, New York, NY, Freeman
31. Cognitive diagnostic assessment for education: Theory and applications 2007a, Leighton J PGierl M J. Eds., Cambridge, United Kingdom, Cambridge University Press
32. Leighton J P, Gierl M J. Defining and evaluating models of cognition used in educational measurement to make inferences about examinees' thinking processes. Educational Measurement: Issues and Practice 2007b, 26:3-16.
33. Leighton J P, Gierl M J, Hunka S M. The Attribute Hierarchy Model for cognitive assessment: A variation on Tatsuoka's rule-space approach. Journal of Educational Measurement 2004, 41:205-237.
34. Linn R L, Dunbar S B. Issues in the design and reporting of the National Assessment of Educational Progress. Journal of Educational Measurement 1992, 29:177-194.
35. Lissitz R W, Bourque M L. Reporting NAEP results using standards. Educational Measurement: Issues and Practice 1995, 14:31-23.
36. Mislevy R J. Cognitive psychology and educational assessment. Educational measurement 2006, 257-305. Brennan R L. Ed., 4th ed., Washington, DC, American Council on Education
37. Mislevy R J, Almond R G, Yan D, Steinberg L S. Bayes nets in educational assessment: Where do the numbers come from?. Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence 1999, 437-446. Laskey K B, Prade H, Eds., San Francisco, CA, Morgan Kaufmann
38. Mislevy R J, Steinberg L, Almond R. On the structure of educational assessments. Measurement: Interdisciplinary research and perspectives 2003, Hillsdale, NJ, Erlbaum
39. Nielsen J. Design web usability: The practice of simplicity 2000, Indianapolis, New Riders Publishing
40. O'Callaghan R K, Morley M E, Schwartz A. Developing skill categories for the SAT Math section. 2004, April, Paper presented at the meeting of the National Council on Measurement in Education, San Diego, CA
41. Pettersson R. Information design: An introduction 2002, Philadelphia, PA, John Benjamins Publishing
42. Ryan J M. An analysis of item mapping and test reporting strategies 2003, http://www.serve.org/Assessment/assessment-publicationh1.php#StApub, Accessed August 14, 2007 at
43. Schriver K A. Dynamics in document design 1997, New York, NY, John Wiley
44. Shah P, Mayer R E, Hegarty M. Graphs as aids to knowledge construction: Signaling techniques for guiding the process of graph construction. Journal of Educational Psychology 1999, 91:690-702.
45. Stout W. Psychometrics: From practice to theory and back. Psychometrika 2002, 67:485-518.
46. Swarts H L, Flower J, Hayes J. How headings in documents can mislead readers. 1980, Technical Report No. 9, Washington, DC, AIR
47. Tatsuoka K K. Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement 1980, 20:345-354.
48. Tatsuoka K K. Toward an integration of item-response theory and cognitive error diagnosis. Diagnostic monitoring of skills and knowledge acquisition 1990, 453-488. Fredrickson NGlaser R LLesgold A MShafto M G. Eds., Hillsdale, NJ, Erlbaum
49. Tatsuoka K K. Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. Cognitively diagnostic assessment 1995, 327-359. Nichols PChipman S FBrennan R L. Eds., Hillsdale, NJ, Erlbaum
50. Templin J L, Henson R A. Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods 2006, 11:287-305.
51. Trout D L, Hyde E. Developing score reports for statewide assessments that are valued and used: Feedback from K-12 stakeholders. 2006, April, Paper presented at the meeting of the American Educational Research Association, San Francisco, CA
52. Tufte E R. The visual display of quantitative information 2001, Cheshire, CT, Graphics Press
53. Vaiana M E, McGlynn E A. What cognitive science tells us about the design of reports for consumers. Medical Care Research and Review 2002, 59:3-35.
54. von Davier M. Hierarchical general diagnostic models. 2007, Research report 07-19, Princeton, NJ: Educational Testing Service
55. Wainer H. Improving tabular display with NAEP tables as examples and inspirations. Journal of Educational and Behavioral Statistics 1997, 22:1-30.
56. Wainer H, Hambleton R K, Meara K. Alternative displays for communicating NAEP results: A redesign and validity study. Journal of Educational Measurement 1999, 36:301-335.
57. Waller R. Text as diagram: Using typography to improve access and understanding. The technology of text: Principles for structuring, designing, and displaying text 1983, 137-166. Jonassen D H. Ed., Englewood Cliffs, NJ, Educational Technology Publications
58. Whitely S E. Multicomponent latent trait model for ability tests. Psychometrika 1980, 62:495-542.
59. Winn W. Color in document design. IEEE Transactions on Professional Communication 1991, 34:180-185.
60. Wright P. Presenting technical information: A survey of research findings. Instructional Sciences 1977, 6:93-134.
61. Zwick R, Senturk D, Wang J, Loomis S C. An investigation of alternative methods for item mapping in the National Assessment of Educational Progress. Educational Measurement: Issues and Practice 2001, 20:15-25.