A Short History of Performance Assessment: Lessons Learned

Phi Delta Kappan

Volumn 80, Issue 9, 1999, Pages 688-695

  Madaus, George F ^a   O'Dwyer, Laura M ^a  

^a NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0037729248     PISSN: 00317217     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Review

References (78)

1. For a list of these claims and references to them, see George F. Madaus, "A National Testing System: Manna from Above? A Historical/Technological Perspective," Educational Assessment, vol. 1, 1993, pp. 9-26.
2. Geoffrey R. Elton, Return to Essentials: Some Reflections on the Present State of Historical Study (Cambridge: Cambridge University Press, 1991), p. 73.
3. The modern and postmodern dichotomy comes from Arieh Lewy, "Postmodernism in the Field of Achievement Testing," Studies in Educational Evaluation, vol. 22, 1996, pp. 223-44. For Lewy, the modern period began after World War I, hit its peak in the 1970s, and is in many respects still with us. We believe that the modern period began earlier, around the turn of this century, but we agree with Lewy that it currently overlaps with the postmodern period. We concur with Lewy's postmodern period as beginning in the 1980s and embodying the current assessment movement. We added the premodern category to cover the period prior to 1900. The time frame on the categories is not exact, and there is considerable overlap between adjacent categories.
4. William Hamilton, Discussions in Philosophy and Literature, Education and University Reform, 2nd ed. (London: Longman, 1853), p. 769. Cited in George F. Madaus and Thomas Kellaghan, "Curriculum Evaluation and Assessment," in Philip W. Jackson, ed., Handbook of Research on Curriculum (New York: Macmillan, 1992), p. 121.
5. William Hamilton, Discussions in Philosophy and Literature, Education and University Reform, 2nd ed. (London: Longman, 1853), p. 769. Cited in George F. Madaus and Thomas Kellaghan, "Curriculum Evaluation and Assessment," in Philip W. Jackson, ed., Handbook of Research on Curriculum (New York: Macmillan, 1992), p. 121.
6. For an interesting comparison of a high-stakes versus a low-stakes performance assessment program, see George F. Madaus and Thomas Kellaghan, "The British Experience with 'Authentic' Testing," Phi Delta Kappan, February 1993, pp. 458-69. For an excellent description of the interplay between policy making, assessment, and teaching in England and Vermont, see William A. Firestone, "A Tale of Two Tests: Tensions in Assessment Policy," Assessment in Education, vol. 5, 1998, pp. 175-91.
7. For an interesting comparison of a high-stakes versus a low-stakes performance assessment program, see George F. Madaus and Thomas Kellaghan, "The British Experience with 'Authentic' Testing," Phi Delta Kappan, February 1993, pp. 458-69. For an excellent description of the interplay between policy making, assessment, and teaching in England and Vermont, see William A. Firestone, "A Tale of Two Tests: Tensions in Assessment Policy," Assessment in Education, vol. 5, 1998, pp. 175-91.
8. For a treatment of teachers' use of assessment in the classroom, see Filip Dochy and Liz McDowell, "Introduction: Assessment as a Tool for Learning," Studies in Educational Evaluation, vol. 23, 1997, pp. 279-98.
9. Evidence from England on how the standard assessment tasks that were part of a high-stakes national testing system affected teachers' views of the curriculum was mixed. For details, see Madaus and Kellaghan, "The British Experience."
10. Each of these three testing methods has technical and practical constraints built into its design. For a discussion of these constraints, see the webpage for the Consortium for Equity in Standards and Testing at http://www.csteep.bc.edu/ctest. Look under "Documents: What Is a Test?"
11. Jerome Bruner, The Culture of Education (Cambridge, Mass.: Harvard University Press, 1996), p. 50.
12. Ibid. See also Leone Burton, "Assessment of Mathematics - What Is the Agenda?," in Menucha Birenbaum and Filip Dochy, eds., Alternatives in Assessment of Achievements, Learning Processes, and Prior Knowledge (Boston: Kluwer Academic Publishers, 1996), pp. 31-62.
13. Burton, p. 33.
14. For a discussion, see George F. Madaus, Anastasia E. Razcek, and Marguerite M. Clarke, "The Historical and Policy Foundations of the Assessment Movement," in A. Lin Goodwin, ed., Assessment for Equity and Inclusion: Embracing All Our Children (New York: Routledge, 1997), pp. 1-34; and George F. Madaus, Marguerite M. Clarke, and Michael O'Leary, "A Century of Standardized Mathematics Testing," in Jeremy Kilpatrick and George Stanic, eds., National Council of Teachers of Mathematics History Volume (forthcoming).
15. For a discussion, see George F. Madaus, Anastasia E. Razcek, and Marguerite M. Clarke, "The Historical and Policy Foundations of the Assessment Movement," in A. Lin Goodwin, ed., Assessment for Equity and Inclusion: Embracing All Our Children (New York: Routledge, 1997), pp. 1-34; and George F. Madaus, Marguerite M. Clarke, and Michael O'Leary, "A Century of Standardized Mathematics Testing," in Jeremy Kilpatrick and George Stanic, eds., National Council of Teachers of Mathematics History Volume (forthcoming).
16. For a concrete example, see George F. Madaus, "The Influence of Testing on the Curriculum," in Laurel Tanner, ed., Critical Issues in Curriculum (Chicago: University of Chicago Press, 1988), pp. 83-121; and Laura S. Hamilton, E. Michael Nussbaum, and Richard E. Snow, "Interview Procedures for Validating Science Assessments," Applied Measurement in Education, vol. 10, 1997, pp. 181-200.
17. For a concrete example, see George F. Madaus, "The Influence of Testing on the Curriculum," in Laurel Tanner, ed., Critical Issues in Curriculum (Chicago: University of Chicago Press, 1988), pp. 83-121; and Laura S. Hamilton, E. Michael Nussbaum, and Richard E. Snow, "Interview Procedures for Validating Science Assessments," Applied Measurement in Education, vol. 10, 1997, pp. 181-200.
18. Graduates of the military examinations were "neglected and disdained," according to Ichisada Miyazaki, China's Examination Hell (New York: Weatherhill, 1976), p. 102.
19. Candidates who studied for and passed the Letters examination progressed more rapidly within the ranks of the civil service than those choosing to sit for Law, History, Rituals, or Classical Studies. For a more detailed discussion, see Wolfgang Franke, The Reform and Abolition of the Traditional Chinese Examination System (Boston: East Asian Research Center, Harvard University, 1963).
20. Edward A. Kracke, Civil Service in Early Sung China - 960-1067 (Cambridge, Mass.: Harvard University Press, 1953).
21. Miyazaki, op. cit.
22. Ben Wilbrink, "Assessment in Historical Perspective," Studies in Educational Evaluation, vol. 23, 1997, p. 43.
23. Alfred W. Crosby, The Measure of Reality: Quantification and Western Society, 1250-1600 (New York: Cambridge University Press, 1997), p. 41.
24. See Daniel J. Boorstin, The Discoverers: A History of Man's Search to Know His World and Himself (New York: Vintage Books, 1985); and idem, Hidden History (New York: Vintage Books, 1985).
25. See Daniel J. Boorstin, The Discoverers: A History of Man's Search to Know His World and Himself (New York: Vintage Books, 1985); and idem, Hidden History (New York: Vintage Books, 1985).
26. Crosby, p. 67.
27. We are indebted to Henry Heaney, retired head librarian of the University of Glasgow, for his help in locating sources for this section of the article. For a history of guilds in England, see Heather Swanson, Medieval Artisans (New York: Blackwell, 1989). For a history of guilds on the Continent, see Prosper Boissonnade, Life and Work in Medieval Europe (London: Kegan Paul, 1927). For a treatment of guilds in America, see W. J. Rorabaugh, The Craft Apprentice: From Franklin to the Machine Age in America (New York: Oxford University Press, 1986).
28. We are indebted to Henry Heaney, retired head librarian of the University of Glasgow, for his help in locating sources for this section of the article. For a history of guilds in England, see Heather Swanson, Medieval Artisans (New York: Blackwell, 1989). For a history of guilds on the Continent, see Prosper Boissonnade, Life and Work in Medieval Europe (London: Kegan Paul, 1927). For a treatment of guilds in America, see W. J. Rorabaugh, The Craft Apprentice: From Franklin to the Machine Age in America (New York: Oxford University Press, 1986).
29. We are indebted to Henry Heaney, retired head librarian of the University of Glasgow, for his help in locating sources for this section of the article. For a history of guilds in England, see Heather Swanson, Medieval Artisans (New York: Blackwell, 1989). For a history of guilds on the Continent, see Prosper Boissonnade, Life and Work in Medieval Europe (London: Kegan Paul, 1927). For a treatment of guilds in America, see W. J. Rorabaugh, The Craft Apprentice: From Franklin to the Machine Age in America (New York: Oxford University Press, 1986).
30. Swanson, pp. 115-16. Swanson notes that "apprenticeship was often no more than a way of acquiring virtually free labour." Further, one could marry the widow of a craftsman and thereby have the "right to practice that craft transferred to him whether he was competent in the craft or not."
31. According to Boissonade, the candidate could "even . . . furnish a simple attestation of capacity according to public repute. . . . pay moderate entry fees, and . . . bear the cost of a not very expensive dinner or feast" (p. 212).
32. Elliot Eisner, "Educational Connoisseurship and Criticism: Their Form and Functions in Educational Evaluation," in George F. Madaus, Michael S. Scriven, and Daniel L. Stufflebeam, eds., Evaluation Models: Viewpoints on Educational and Human Services Evaluation (Boston: Kluwer-Nijhoff Publishing, 1983), pp. 335-48.
33. For further information, see Stephen A. Epstein, Wage Labor and Guilds in Medieval Europe, 2nd ed. (Chapel Hill: University of North Carolina Press, 1991).
34. We are indebted to Stephen Brown, professor of medieval philosophy, Boston College, for his help in locating and translating documents for this section of the article.
35. Stephen Wiseman, Examinations and English Education (Manchester, England; Manchester University Press, 1961).
36. Frank P. Graves, A History of Education During the Middle Ages, 2nd ed. (Westport, Conn.: Greenwood Press, 1920).
37. Keith Hoskin, "The Examination, Disciplinary Power, and Rational Schooling," History of Education, vol. 8, 1979, p. 138.
38. Unpublished translation by Stephen Brown, Boston College, June 1998.
39. See Rosemary Bowler, "Payment by Results: A Study in Achievement and Accountability" (Doctoral dissertation, Boston College, 1983).
40. Hoskin, p. 138.
41. See Carolyn Webber, "The Mandarin Mentality: Civil Service and University Admissions Testing in Europe and Asia," in Bernard R. Gifford, ed., Test Policy and the Politics of Opportunity Allocation: The Workplace and the Law (Boston: Kluwer Academic Publishers, 1989).
42. Hoskins, pp. 135-46; and Madaus and Kellaghan, "Curriculum Evaluation and Assessment," p. 124.
43. Hoskins, pp. 135-46; and Madaus and Kellaghan, "Curriculum Evaluation and Assessment," p. 124.
44. Wiseman, pp. 28-29.
45. George F. Madaus, "Testing as a Social Technology: The Inaugural Boisi Lecture in Education and Public Policy," Boston College, 6 December 1990.
46. Massachusetts Historical Society Documents, 1845.
47. See Lawrence D. Kritzman, ed., Michel Foucault: Politics, Philosophy, Culture: Interviews and Other Writings, 1977-1984 (New York: Routledge, 1990), p. 107.
48. David B. Tyack, The One Best System: A History of American Urban Education (Cambridge, Mass.: Harvard University Press, 1974), p. 41.
49. By 1888 the superintendent in Cincinnati complained that, when these essay tests were used to determine the promotion and classification of children, they "perverted the best efforts of teachers, and narrowed and grooved their instruction; they have occasioned and made well-nigh imperative the use of mechanical and rote methods of teaching; they have occasioned cramming and the most vicious habits of study; they have caused much of the overpressure charged upon schools, some of which is real; they have tempted both teachers and pupils to dishonesty; and last but not least, they have permitted a mechanical method of schools supervision." For further discussion, see E. E. White, "Examinations and Promotions," Education, vol. 8, 1888, pp. 519-22.
50. Madaus, Scriven, and Stufflebeam, op. cit.
51. Michael M. Sokal, "James McKeen Cattell and Mental Anthropometry: Nineteenth-Century Science and Reform and the Origins of Psychological Testing," in idem, ed., Psychological Testing and American Society, 1890-1930 (New Brunswick, N.J.: Rutgers University Press, 1986), pp. 21-45.
52. David F. Lohman and Robert M. Thorndike, A Century of Ability Testing (Chicago: Riverside, 1990), p. 22.
53. Ibid.
54. Madaus and Kellaghan, "Curriculum Evaluation and Assessment," p. 122.
55. D. Starch and E. C. Elliot, "Reliability of Grading High School Work in English," School Review, vol. 21, 1912, pp. 442-57; and idem, "Reliability of Grading Work in Mathematics," School Review, vol. 21, 1913, pp. 254-59.
56. D. Starch and E. C. Elliot, "Reliability of Grading High School Work in English," School Review, vol. 21, 1912, pp. 442-57; and idem, "Reliability of Grading Work in Mathematics," School Review, vol. 21, 1913, pp. 254-59.
57. Raymond E. Callaghan, Education and the Cult of Efficiency (Chicago: University of Chicago Press, 1962).
58. Lohman and Thorndike credit the Chinese with the development and use of the selection/multiple-choice item type. See Lohman and Thorndike, op. cit. Nonetheless, Franz Samelson names Frederick Kelly the inventor of the multiple-choice item in 1914. Franz Samelson, "Was Early Mental Testing (a) racist inspired (b) objective science (c) a technology for democracy (d) the origin of the multiple-choice examination (e) none of the above?," in Sokal, pp. 113-27.
59. F. Baker, "Automation of Test Scoring, Reporting, and Analysis," in Robert L. Thorndike, ed., Educational Measurement (Washington, D.C.: American Council on Education, 1971), pp. 202-34. See also Madaus and Kellaghan, "Curriculum Evaluation and Assessment."
60. F. Baker, "Automation of Test Scoring, Reporting, and Analysis," in Robert L. Thorndike, ed., Educational Measurement (Washington, D.C.: American Council on Education, 1971), pp. 202-34. See also Madaus and Kellaghan, "Curriculum Evaluation and Assessment."
61. For evidence, see Madaus and Kellaghan, "The British Experience." See also Desmond L. Nuttall, "Performance Assessment: The Message from England," Educational Leadership, May 1992, pp. 54-57.
62. For evidence, see Madaus and Kellaghan, "The British Experience." See also Desmond L. Nuttall, "Performance Assessment: The Message from England," Educational Leadership, May 1992, pp. 54-57.
63. Madaus and Kellaghan, "The British Experience."
64. See, for example, Madaus, "The Influence of Testing"; and George F. Madaus and Vincent Greaney, "The Irish Experience in Competency Testing: Implications for American Education," American Journal of Education, vol. 93, 1985, pp. 268-94.
65. See, for example, Madaus, "The Influence of Testing"; and George F. Madaus and Vincent Greaney, "The Irish Experience in Competency Testing: Implications for American Education," American Journal of Education, vol. 93, 1985, pp. 268-94.
66. See Daniel Koretz et al., "The Vermont Portfolio Assessment Program: Findings and Implications," Educational Measurement: Issues and Practice, Fall 1994, pp. 5-16.
67. See, for example, Daniel M. Koretz et al., "The Effects of High-Stakes Testing on Achievement: Preliminary Findings About Generalization Across Tests," paper presented at the annual meeting of the American Educational Research Association, Chicago, 1991. See also Richard J. Shavelson, Xiaohang Gao, and Gail P. Baxter, Sampling Variability of Performance Assessment (Los Angeles: National Center for Research on Evaluations, Standards, and Student Testing, University of California, 1993), p. 361.
68. See, for example, Daniel M. Koretz et al., "The Effects of High-Stakes Testing on Achievement: Preliminary Findings About Generalization Across Tests," paper presented at the annual meeting of the American Educational Research Association, Chicago, 1991. See also Richard J. Shavelson, Xiaohang Gao, and Gail P. Baxter, Sampling Variability of Performance Assessment (Los Angeles: National Center for Research on Evaluations, Standards, and Student Testing, University of California, 1993), p. 361.
69. For cost estimates, see Daniel M. Koretz, George F. Madaus, Edward Haertel, and Alfred Beaton, statement before the Subcommittee on Elementary, Secondary, and Vocational Education, Committee on Education and Labor, U.S. House of Representatives, 19 February 1992. See also George F. Madaus and Thomas Kellaghan, "Student Examination Systems in the European Community: Lesson for the United States," in Gerald Kulm and Shirley Malcom, eds., Science Assessment in the Service of Reform (Washington, D.C.: American Association for the Advancement of Science, 1992), pp. 189-232. See also idem, "National Testing: Lessons for America from Europe," Educational Leadership, November 1991, pp. 87-93.
70. For cost estimates, see Daniel M. Koretz, George F. Madaus, Edward Haertel, and Alfred Beaton, statement before the Subcommittee on Elementary, Secondary, and Vocational Education, Committee on Education and Labor, U.S. House of Representatives, 19 February 1992. See also George F. Madaus and Thomas Kellaghan, "Student Examination Systems in the European Community: Lesson for the United States," in Gerald Kulm and Shirley Malcom, eds., Science Assessment in the Service of Reform (Washington, D.C.: American Association for the Advancement of Science, 1992), pp. 189-232. See also idem, "National Testing: Lessons for America from Europe," Educational Leadership, November 1991, pp. 87-93.
71. See Sally Thomas, George F. Madaus, and Anastasia E. Razcek, "Comparing Teacher Assessments and Standard Task Results in England: The Relationship Between Pupil Characteristics and Attainment," Assessment in Education: Principles, Policy, and Practice, vol. 5, 1999, in press. See also Eva L. Baker, Harold F. O'Neil, and Robert L. Linn, "Policy and Validity Prospects for Performance-Based Assessment," paper presented at the annual meeting of the American Psychological Association, San Francisco, 1991.
72. See Sally Thomas, George F. Madaus, and Anastasia E. Razcek, "Comparing Teacher Assessments and Standard Task Results in England: The Relationship Between Pupil Characteristics and Attainment," Assessment in Education: Principles, Policy, and Practice, vol. 5, 1999, in press. See also Eva L. Baker, Harold F. O'Neil, and Robert L. Linn, "Policy and Validity Prospects for Performance-Based Assessment," paper presented at the annual meeting of the American Psychological Association, San Francisco, 1991.
73. For a discussion of these issues, see Madaus, Razcek, and Clarke, op. cit.
74. For a discussion of this problem of teaching higher-order thinking skills, see Peter Airasian, "Empiricism and Values: Two Faces of Educational Change," International Journal of Educational Research, vol. 27, 1997, pp. 433-45. See also idem. "Critical Pedagogy and the Realities of Teaching," in Henry J. Perkinson, ed., Teachers Without Goals, Students Without Purposes (New York: McGraw-Hill, 1997), pp. 81-93; Peter W. Airasian and Mary E. Walsh, "Constructivist Cautions," Phi Delta Kappan, February 1997, pp. 444-49; Peter Airasian and George F. Madaus, "Linking Testing and Instruction: Policy Issues," Journal of Educational Measurement, vol. 20, 1983, pp. 103-18; and Hamilton, Nussbaum, and Snow, op. cit.
75. For a discussion of this problem of teaching higher-order thinking skills, see Peter Airasian, "Empiricism and Values: Two Faces of Educational Change," International Journal of Educational Research, vol. 27, 1997, pp. 433-45. See also idem. "Critical Pedagogy and the Realities of Teaching," in Henry J. Perkinson, ed., Teachers Without Goals, Students Without Purposes (New York: McGraw-Hill, 1997), pp. 81-93; Peter W. Airasian and Mary E. Walsh, "Constructivist Cautions," Phi Delta Kappan, February 1997, pp. 444-49; Peter Airasian and George F. Madaus, "Linking Testing and Instruction: Policy Issues," Journal of Educational Measurement, vol. 20, 1983, pp. 103-18; and Hamilton, Nussbaum, and Snow, op. cit.
76. For a discussion of this problem of teaching higher-order thinking skills, see Peter Airasian, "Empiricism and Values: Two Faces of Educational Change," International Journal of Educational Research, vol. 27, 1997, pp. 433-45. See also idem. "Critical Pedagogy and the Realities of Teaching," in Henry J. Perkinson, ed., Teachers Without Goals, Students Without Purposes (New York: McGraw-Hill, 1997), pp. 81-93; Peter W. Airasian and Mary E. Walsh, "Constructivist Cautions," Phi Delta Kappan, February 1997, pp. 444-49; Peter Airasian and George F. Madaus, "Linking Testing and Instruction: Policy Issues," Journal of Educational Measurement, vol. 20, 1983, pp. 103-18; and Hamilton, Nussbaum, and Snow, op. cit.
77. For a discussion of this problem of teaching higher-order thinking skills, see Peter Airasian, "Empiricism and Values: Two Faces of Educational Change," International Journal of Educational Research, vol. 27, 1997, pp. 433-45. See also idem. "Critical Pedagogy and the Realities of Teaching," in Henry J. Perkinson, ed., Teachers Without Goals, Students Without Purposes (New York: McGraw-Hill, 1997), pp. 81-93; Peter W. Airasian and Mary E. Walsh, "Constructivist Cautions," Phi Delta Kappan, February 1997, pp. 444-49; Peter Airasian and George F. Madaus, "Linking Testing and Instruction: Policy Issues," Journal of Educational Measurement, vol. 20, 1983, pp. 103-18; and Hamilton, Nussbaum, and Snow, op. cit.
78. For a discussion of this problem of teaching higher-order thinking skills, see Peter Airasian, "Empiricism and Values: Two Faces of Educational Change," International Journal of Educational Research, vol. 27, 1997, pp. 433-45. See also idem. "Critical Pedagogy and the Realities of Teaching," in Henry J. Perkinson, ed., Teachers Without Goals, Students Without Purposes (New York: McGraw-Hill, 1997), pp. 81-93; Peter W. Airasian and Mary E. Walsh, "Constructivist Cautions," Phi Delta Kappan, February 1997, pp. 444-49; Peter Airasian and George F. Madaus, "Linking Testing and Instruction: Policy Issues," Journal of Educational Measurement, vol. 20, 1983, pp. 103-18; and Hamilton, Nussbaum, and Snow, op. cit.