Risking Frankness in Educational Assessment

Phi Delta Kappan

Volumn 80, Issue 9, 1999, Pages 680-687

  Moss, Pamela A ^a   Schutz, Aaron ^b  

^a UNIVERSITY OF MICHIGAN   (United States)

^b UNIVERSITY OF WISCONSIN MILWAUKEE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0010220301     PISSN: 00317217     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (36)

1. Lee J. Cronbach, "Validity on Parole: How Can We Go Straight?," in Measuring Achievement: Progress over a Decade (San Francisco: Jossey-Bass, 1980), p. 107.
2. Lee J. Cronbach, "Five Perspectives on the Validity Argument," in Howard Wainer, ed., Test Validity (Hillsdale, N.J.: Erlbaum, 1988), p. 3.
3. Website of the National Board for Professional Teaching Standards, www.nbpts.org.
4. Early Adolescence/English Language Arts (EA/ELA) Standards for National Board Certification (Southfield, Mich.: National Board for Professional Teaching Standards, 1996), p. 7.
5. Mari Pearlman, "The Design Architecture of NBPTS Certification Assessments," in Lawrence Ingvarson, ed., Assessing Teachers for Professional Certification (Stamford, Ct.: JAI, forthcoming).
6. Ibid., p. 9.
7. "(1) Teachers are committed to students and their learning. (2) Teachers know the subjects they teach and how to teach those subjects to students. (3) Teachers are responsible for managing and monitoring student learning. (4) Teachers think systematically about their practice and learn from experience. (5) Teachers are members of learning communities." EA/ELA Standards, pp. 2-3.
8. John Hattie, "Validating Teaching Standards," in Ingvarson, op. cit.
9. Ibid., p. 10.
10. EA/ELA Standards, p. 1.
11. Ibid., p. 13.
12. Hattie, p. 14.
13. Educational Testing Service, NBPTS Technical Analysis Report, 1996-97 Administration (Southfield, Mich.: National Board for Professional Teaching Standards, 1998), p. 10.
14. Pearlman, p. 18.
15. Ibid.
16. Mari Pearlman, quoted in Richard M. Jaeger, "Evaluating the Psychometric Qualities of the National Board for Professional Teaching Standards' Assessments: A Methodological Accounting," Journal of Personnel Evaluation in Education, vol. 12, 1998, p. 191.
17. Educational Testing Service, p. 11.
18. One promising study under way is a comparison of National Board evaluations of MC/GEN candidates with evaluations based on classroom observations and interviews. Drew Gitomer, personal communication, 18 May 1998.
19. Educational Testing Service, p. 58.
20. Ibid., App. 1, p. 23.
21. Ibid., p. 46.
22. One notable exception to this observation is a promising set of special studies under way to help in understanding differences in scores between African American and white ELA candidates. See Lloyd Bond, "Disparate Impact and Teacher Certification," Journal of Personnel Evaluation in Education, vol. 12, 1998, pp. 211-20.
23. Mari Pearlman, personal communication, 4 September 1998.
24. Jaeger, op. cit.; and Educational Testing Service, op. cit.
25. Charles Lewis and Man Pearlman, "An Experiment in Standard Setting," paper presented at the annual meeting of the American Educational Research Association, Chicago, 1997, p. 3. For a similar argument, see Ginette Delandshere and Anthony R. Petrosky, "Capturing Teachers' Knowledge," Educational Researcher, June/July 1994, pp. 11-18.
26. Charles Lewis and Man Pearlman, "An Experiment in Standard Setting," paper presented at the annual meeting of the American Educational Research Association, Chicago, 1997, p. 3. For a similar argument, see Ginette Delandshere and Anthony R. Petrosky, "Capturing Teachers' Knowledge," Educational Researcher, June/July 1994, pp. 11-18.
27. A pilot study by assessment developers Lewis and Pearlman explored the feasibility of having standard setters read over multiple exercises from a candidate, stopping when they thought no new information would change their minds. Lewis and Pearlman, op. cit.
28. Pamela A. Moss, "Enlarging the Dialogue in Educational Measurement: Voices from Interpretive Research Traditions," Educational Researcher, January/February 1996, pp. 20-28, 43.
29. Standards for Educational and Psychological Testing (Washington, D.C.: American Educational Research Association, American Psychological Association, and National Council on Measurement in Education, 1985). The Standards are currently under revision.
30. Lee J. Cronbach, "Construct Validation After Thirty Years," in Robert L. Linn, ed., Intelligence: Measurement, Theory, and Public Policy (Urbana: University of Illinois Press, 1989), p. 152; and idem, "Five Perspectives," p. 13.
31. Lee J. Cronbach, "Construct Validation After Thirty Years," in Robert L. Linn, ed., Intelligence: Measurement, Theory, and Public Policy (Urbana: University of Illinois Press, 1989), p. 152; and idem, "Five Perspectives," p. 13.
32. Gitomer, op. cit.; Lewis and Pearlman, op. cit; and Bond, op. cit. For a more extensive review of the National Board's validity research agenda, see Jaeger, op. cit.; and Pamela A. Moss, "A Critical Review of the Validity Research Agenda of the National Board for Professional Teaching Standards," in Ingvarson, op. cit. Our own work in progress with the assessment of beginning teachers for the Interstate New Teacher Assessment and Support Consortium offers another example of the kinds of alternative sources of evidence that can be assessed. See Pamela A. Moss, Aaron Schutz, and Kathleen Collins, "An Integrative Approach to Portfolio Evaluation for Teacher Licensure," Journal of Personnel Evaluation in Education, vol. 12, 1998, pp. 139-62.
33. Gitomer, op. cit.; Lewis and Pearlman, op. cit; and Bond, op. cit. For a more extensive review of the National Board's validity research agenda, see Jaeger, op. cit.; and Pamela A. Moss, "A Critical Review of the Validity Research Agenda of the National Board for Professional Teaching Standards," in Ingvarson, op. cit. Our own work in progress with the assessment of beginning teachers for the Interstate New Teacher Assessment and Support Consortium offers another example of the kinds of alternative sources of evidence that can be assessed. See Pamela A. Moss, Aaron Schutz, and Kathleen Collins, "An Integrative Approach to Portfolio Evaluation for Teacher Licensure," Journal of Personnel Evaluation in Education, vol. 12, 1998, pp. 139-62.
34. Hans-Georg Gadamer, Truth and Method (New York: Seabury, 1975); and Moss, "Enlarging the Dialogue."
35. Hans-Georg Gadamer, Truth and Method (New York: Seabury, 1975); and Moss, "Enlarging the Dialogue."
36. Samuel Messick, "Validity," in Robert L. Linn, ed., Educational Measurement, 3rd ed. (Washington, D.C.: American Council on Education and National Council on Measurement in Education, 1989), p. 88.