Testing historical knowledge: Standards, multiple-choice questions and student reasoning

Theory and Research in Social Education

Volumn 37, Issue 3, 2009, Pages 325-360

  Reich, Gabriel A ^a  

^a VIRGINIA COMMONWEALTH UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84867516299     PISSN: 00933104     EISSN: 21631654     Source Type: Journal    
DOI: 10.1080/00933104.2009.10473401     Document Type: Article

References (98)

1. American Educational Research Association (AERA). (2000). AERA position statement concerning high-stakes testing in PreK-12 education AERA. Retrieved November 11, 2008 from www.aera.net/about/policy/stakes.htm.
2. Amrein, A. L., & Berliner, D. C. (2002a). An analysis of some unintended and negative consequences of high-stakes testing (No. EPSL-0211-125-EPRU). Tempe, AZ: Education Policy Studies Laboratory, Education Policy Research Unit, Arizona State University.
3. Amrein, A. L., & Berliner, D. C. (2002b). The impact of high-stakes test on student academic performance: An analysis of NAEP results in states with high-stakes tests and ACT, SAT and AP test results in states with high school graduation exams (No. EPSL-0211-126-EPRU). Tempe, AZ: Education Policy Studies Laboratory, Education Policy Research Unit, Arizona State University.
4. Anderson, L. W., Krathwohl, D. R., Airasian, P.W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., et al. (2001). A taxonomy for learning, teaching and assessing: A revision of Bloom's Taxonomy of Educational Objectives. New York: Longman.
5. Appleby, J. O., Hunt, L. A., & Jacob, M. C. (1994). Telling the truth about history. New York: Norton.
6. Barton, K. C., & Levstik, L. S. (1998). "It wasn't a good part of history": National identity and students' explanations of historical significance. Teachers College Record, 99(3), 478-513.
7. Beadie, N. (1999). From student markets to credential markets: The creation of the Regents examination system in New York State, 1864-1890. History of Education Quarterly, 39(1), 1-30.
8. Beck, I. L., & McKeown, M. G. (1988). Toward meaningful accounts in history texts for young learners. Educational Researcher, 17(6), 31-39.
9. Boix-Mansilla, V. (2000). Historical understanding: Beyond the past and into the present. In P. Stearns, P. Seixas & S. Wineburg (Eds.), Knowing teaching and learning history (1st ed., pp. 390-418). New York: New York University Press.
10. Bloom, B. S. (1956). Taxonomy of Educational Objectives: The classification of educational goals. New York: D. McKay.
11. Bradley Commission on History in Schools. (1988). Building a history curriculum: Guidelines for teaching history in schools. Washington, DC: Educational Excellence Network.
12. Braun, H. I., Jackson, D. N., Wiley, D. E., & Messick, S. (2002). The role of constructs in psychological and educational measurement. Mahwah, NJ: Erlbaum.
13. Braun, H. I., & Mislevy, R. (2005). Intuitive test theory. Phi Delta Kappan, 86(7), 489-498.
14. Broadfoot, P. M. (1996). Education, assessment and society. Philadelphia, PA: Open University Press.
15. Bruner, J. (1996). The culture of education. Cambridge, Massachusetts: Harvard University Press.
16. Cuban, L. (2004). Looking through the rearview mirror at school accountability. In K. A. Sirotnik (Ed.), Holding accountability accountable (pp. 18-34). New York: Teachers College Press.
17. Dao, J. (1996, April 24). Passing of Regents' exams to be required for diploma. New York Times, p. 1.
18. Darling-Hammond, L. (2004). Standards, accountability, and school reform. Teachers College Record, 106(6), 1047-1085.
19. DeMauro, G. E. (2000). Summary of technical reports on the Regents examinations in comprehensive English (English language arts), mathematics A, global history and geography (global history), and United States history and government. Albany, NY: New York State Department of Education.
20. DeMauro, G. E. (2002). How the scale scores are calculated for New York State Regents examinations. Albany, NY: New York State Education Department.
21. Echternacht, G. (2000). Global history and geography Regents examination--data and information related to standard setting (Technical Report. Albany, NY: New York State Department of Education).
22. Editorial Projects in Education. (2006). Quality counts at 10: New York. Bethesda, MD: Editorial Projects in Education.
23. Edusolution.com: New York State Regents exams. (2007). Retrieved March 10, 2007, from http://edusolution.com/regentsexams/socialstudiesindex.htm
24. Ercikan, K. (2006). Developments in assessment of student learning. In P. A. Alexander, & P. H. Winne (Eds.), Handbook of educational psychology (2nd ed., pp. 929-952). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.
25. Ericsson, K. A., & Simon, H. A. (1992). Protocol analysis: Verbal reports as data (Rev. ed.). Cambridge, MA: MIT Press.
26. Farr, R., Pritchard, R., & Smitten, B. (1990). A description of what happens when an examinee takes a multiple-choice reading comprehension test. Journal of Educational Measurement, 27(3), 209-226.
27. Fickel, L. H. (2006). Paradox of practice: Expanding and contradicting curriculum in a high-stakes climate. In S. G. Grant (Ed.), Measuring history: Cases of state-level testing across the United States (pp. 75-103). Greenwich, CT: Information Age.
28. Finn, C. Jr., Petrilli, M. J. & Julian, L. (2006). The state of state standards 2006. Retrieved September 18, 2006, from http:www.edexcellence.net/foundation/ publication.cfm?id=358.
29. Glaser, R., & Baxter, G. P. (2002). Cognition and construct validity: Evidence for the nature of cognitive performance in assessment situations. In H. I. Braun, D. N. Jackson, D. E. Wiley & S. Messick (Eds.), The role of constructs in psychological and educational measurement (pp. 179-192). Mahwah, NJ: Erlbaum.
30. Grant, S. G. (2001a). It's just the facts, or is it? The relationship between teachers' practices and students' understandings of history. Theory and Research in Social Education, 29(1), 65-108.
31. Grant, S. G. (2001b). An uncertain lever: Exploring the influence of state-level testing in New York State on teaching social studies. Teachers College Record, 103(3), 398-426.
32. Grant, S. G. (2001c). When an "A" is not enough: Analyzing the New York State global history and geography exam. Education Policy Analysis Archives, 9(39). Retrieved September 26, 2005 from http://epaa.asu.edu/epaa/v9n39.html.
33. Grant, S. G. (2003). History lessons: Teaching, learning, and testing in U.S. high school classrooms. Mahwah, NJ: Erlbaum.
34. Grant, S. G., Derme-Insinna, A., Gradwell, J., Lauricella, A. M., Pullano, L., & Tzetzo, K. (2000, November). Teachers, tests, and tensions: Teachers respond to the New York State global history exam. Annual Conference of the College and University Faculty Association, National Council for the Social Studies, San Antonio, TX.
35. Grant, S. G., Gradwell, J., Lauricella, A. M., Derme-Insinna, A., Pullano, L., & Tzetzo, K. (2002). When increasing stakes need not mean increasing standards: The case of the New York State global history and geography exam. Theory and Research in Social Education, 30(4), 488-515.
36. Haladyna, T. M. (1999). Developing and validating multiple-choice test items (2nd ed.). London: Erlbaum.
37. Haladyna, T. M. (2002). Essentials of standardized achievement. Boston, MA: Allyn & Bacon.
38. Haladyna, T. M. (2004). Developing and validating multiple-choice test items (3rd ed.). Mahwah, NJ: Erlbaum.
39. Hamilton, L. S., Nussbaum, E. M., & Snow, R. E. (1997). Interview procedures for validating science assessments. Applied Measurement in Education, 10(2), 181-200.
40. Hamilton, L. S., Stecher, B. M., & Klein, S. P. (2002). Introduction. In L. S. Hamilton, B. M. Stecher, & S. P. Klein, (Eds.), Making sense of test-based accountability in education (No. MR-1554-EDU). Santa Monica, CA: RAND Corporation.
41. Haney, W., & Scott, L. (1987). Talking with children about tests: An exploratory study of test item ambiguity. In R. O. Freedle, & R. P. Duran (Eds.), Cognitive and linguistic analyses of test performance (pp. 298-368). Norwood, NJ: Ablex.
42. Kahn, R. The Princeton Review: Road map to the Regents: Global history and geography (2003). In R. Kahn (Ed.). New York: Princeton Review Publishing.
43. Lee, P. J. (1983). History teaching and philosophy of history. History and Theory, 22(4), 19-49.
44. Lee, P. J. (2005). Putting principals into practice: Understanding history. In M. S. Donovan, & J. D. Bransford (Eds.), How students learn (1st ed., pp. 31-78). Washington, D.C.: National Academies Press.
45. Lee, P. J., & Ashby, R. (2000). Progression in historical understanding among students ages 7-14. In P. Stearns, P. Seixas & S. Wineburg (Eds.), Knowing, teaching and learning history (pp. 199-222). New York: NYU Press.
46. Lee, P. J., & Ashby, R. (2001). Empathy, perspective taking, and rational understanding. In O. L. Davis Jr., E. A. Yeager,& S. J. Foster (Eds.), Historical empathy and perspective taking in the social studies (pp. 21-50). New York: Rowman & Littlefield.
47. Lee, P. J., Dickenson, A., & Ashby, R. (1997). "Just another emperor": Understanding action in the past. International Journal of Education Research, 27(3), 233-244.
48. Linn, R. L. (2003). Accountability: Responsibility and reasonable expectations. Educational Researcher, 32(7), 3-3-13.
49. Lipman, P. (2004). High stakes education: Inequality, globalization, and urban school reform (1st ed.). New York, London: RoutledgeFalmer.
50. Lobes, L. S. (1998). Surveying state standards: National history education network's 1997 report on state social studies standards. The History Teacher, 31(2), 221-234.
51. Lukhele, R., Thissen, D., & Wainer, H. (1994). On the relative value of multiple-choice, constructed response, and examinee-selected items on two achievement tests. Journal of Educational Measurement, 31(3), 234-250.
52. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. The American Psychologist, 50(September), 741-749.
53. Meyers, J. P. (2008). Making sense of a globalizing world: Adolescents' explanatory frameworks for poverty. Theory and Research in Social Education, 36(2), 95-123.
54. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks: Sage.
55. Millman, J., Bishop, C. H., & Ebel, R. (1965). An analysis of test-wiseness. Educational and Psychological Measurement, 25, 707-726.
56. Mink, L. O., Fay, B., Vann, R. T., & Golob, E. O. (1987). Historical understanding. Ithaca, NY: Cornell University Press.
57. National Center for History in the Schools. (1994). In Crabtree C., Nash G. (Eds.), National standards for world history: Paths to the present (Expanded ed.). Los Angeles: National Center for History in the Schools, University of California.
58. National Research Council, Committee on Appropriate Test Use, Board on Testing and Assessment. High SUdpsoTesting for Tracking, Promotion, and Graduation. Heubert, Jay P., and Robert M. Hauser, eds. Washington, D.C.: National Academy Press, 1999.
59. New York State Education Department. (1999a). Global history and geography Regents examination: Test sampler draft. Albany, NY: The University of the State of New York, State Education Department, Office of Curriculum and Instruction.
60. New York State Education Department. (1999b). Resource guide with core curriculum. Albany, NY: The State University of New York, State Education Department, Office of Curriculum and Instruction.
61. New York State Education Department. (1999c). Social studies resource guide: The standards and performance indicators. Albany, NY: The University of the State of New York, The State Education Department, Office of Curriculum and Instruction.
62. New York State Education Department. (2001). New York: The state of learning: A report to the Governor and the legislature on the educational status of the state's schools. Albany, NY: The University of the State of New York/The State Education Department.
63. New York State Education Department. (2004). New York: The state of learning. Albany, NY: The University of the State of New York/The State Education Department.
64. New York State Education Department. (2007). Regents examination in global history and geography - January 2007: Chart for converting total test raw scores to final examination scores (scaled scores). Retrieved March 1, 2007, from http://emsc32.nysed.gov/osa/concht/jan07/global.pdf
65. Nichols, S. L., & Berliner, D. (2008). Collateral damage: How high-stakes testing corrupts America's schools. Cambridge, MA: Harvard Education Press.
66. Norris, S. P., Leighton, J. P., & Phillips, L. M. (2004). What is at stake in knowing the content and capabilities of children's minds? A case for basing high stakes tests on cognitive models. Theory and Research in Education, 2(3), 283-308.
67. Nuthall, G., & Alton-Lee, A. (1995). Assessing classroom learning: How students use their knowledge and experience to answer classroom achievement test questions in science and social studies. American Educational Research Journal, 32(1), 185-223.
68. Pelligrino, J. W., Baxter, G. P., & Glaser, R. (1999). Addressing the "two-disciplines" problem: Linking theories of cognition and learning with assessment and instructional practice. Review of Research in Education, 24, 307-353.
69. Pellegrino, J. W., Chudowsky, N., & Glaser, R. (Eds.). (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.
70. Raptis, H., & Fleming, T. (2006). Large-scale assessment outcomes in British Columbia, 1876-1999. Canadian Journal of Education, 29(4), 1191-1225.
71. Ravitch, D. (2000). Left back: A century of failed school reforms. New York: Simon & Schuster.
72. Rhoten, D., Carnoy, M., Chabran, M., & Elmore, R. F. (2003). The conditions and characteristics of assessment and accountability: The case of four states. In M. Carnoy, R. F. Elmore, & L. S. Siskin (Eds.), The new accountability: High schools and high stakes testing (pp. 13-54). New York: RoutledgeFalmer.
73. Rogers, W. T., & Bateson, D. J. (1991). The influence of test-wiseness on performance of high school seniors on school leaving examinations. Applied Measurement in Education, 4(2), 159-183.
74. Rothstein, R. (2004). We are not ready to assess history performance. Journal of American History, 90(4), 1381-1391.
75. Rusen, J. (2005). History: Narration, interpretation, orientation (1st ed.). New York: Berghahn Books.
76. Sarnacki, R. E. (1979). An examination of test-wiseness in the cognitive test domain. Review of Educational Research, 49(2), 252-279.
77. Segall, A. (2006). Teaching in the age of accountability. In S. G. Grant (Ed.), Measuring history: Cases of state-level testing across the United States (pp. 105-132). Greenwich, CT: Information Age.
78. Seixas, P (1994). Students' understanding of historical significance. Theory and Research in Social Education, 22(3), 281-304.
79. Seixas, P. (1996). Conceptualizing the growth of historical understanding. In D. R. Olson, & N. Torrance (Eds.), Handbook of education and human development: New models of learning, teaching and schooling (pp. 775-783). Cambridge, MA: Blackwell.
80. Seixas, P. (2006). Benchmarks of historical thinking: A framework for assessment in Canada. Retrieved July 1, 2009, from http://www.histori.ca/benchmarks/concepts/understand.php
81. Shemilt, D. (1980). History 13-16 evaluation study: Schools council history 13-16 project. Glasgow: Holmes McDougall.
82. Shemilt, D. (1983). The devil's locomotive. History and Theory, 22(4), 1-18.
83. Shemilt, D. (1984). Beauty and the philosopher: Empathy in history and classroom. In A. Dickinson, P. J. Lee, & P. Rogers (Eds.), Learning history (pp. 39-84). London: Heinemann.
84. Shemilt, D. (2000). The caliph's coin: The currency of narrative frameworks in history teaching. In P. Stearns, P. Seixas & S. Wineburg (Eds.), Knowing, teaching and learning history (pp. 83-101). New York University Press.
85. Smith, J. K. (1982). Converging on correct answers: A peculiarity of multiple choice tests. Journal of Educational Measurement, 19(3), 211-220.
86. Tamir, P. (1990). Justifying the selection of answers in multiple choice items. International Journal of Science Education, 12(5), 563-573.
87. van Hover, S. (2006). Teaching history in the Old Dominion: The impact of Virginia's accountability reform on seven secondary beginning history teachers. In S. G. Grant (Ed.), Measuring history: Cases of state-level testing across the United States (pp. 195-221). Greenwich, CT: Information Age.
88. VanSledright, B. A. (2001). From empathic regard to self-understanding: Im/Positionality, empathy, and historical contextualization. In O. L., Davis Jr., E. A. Yeager, & S. J. Foster (Eds.), Historical empathy and perspective taking in the social studies (pp. 5168). New York: Rowman & Littlefield.
89. VanSledright, B. A. (2004). What does it mean to think historically… and how do you teach it? Social Education, 68(1), 230-233.
90. VanSledright, B. A. (2008). Narratives of nation-state, historical knowledge and school history education. Review of Research in Education, 32, 109-146.
91. VanSledright, B. A., & Limon, M. (2006). Learning and teaching social studies: A review of cognitive research in history and geography. In P. A. Alexander, & P. H. Winne (Eds.), Handbook of educational psychology (2nd ed., pp. 545-570). Mahwah, NJ: Lawrence Erlbaum.
92. Venezky, R. L., Wagner, D. A., and Ciliberti, B. S. (Eds.). (1990). Toward defining literacy. Newark, DE: International Reading Association.
93. White, H. V. (1973). Metahistory: The historical imagination in nineteenth-century Europe. Baltimore: Johns Hopkins University Press.
94. Wilson, S. M., & Wineburg, S. S. (1988). Peering at history through different lenses: The role of disciplinary perspectives in teaching history. Teachers College Record, 89(4), 525-539.
95. Wineburg, S. S. (1991). Historical problem solving: A study of the cognitive processes used in the evaluation of documentary and pictorial evidence. Journal of Educational Psychology, 83(1), 73-73-87.
96. Wineburg, S. S. (2001). Historical thinking and other unnatural acts. Philadelphia: Temple University Press.
97. Yarroch, W. L. (1991). The implications of content versus item validity on science tests. Journal of Research in Science Testing, 28(7), 619-629.
98. Yeager, E. A., Foster, S. J., Maley, S. D., Anderson, T., & Morris, J. W. III (1998). Why people in the past acted as they did: An exploratory study in historical empathy. International Journal of Social Education, 13(1), 8-24.