Standard Setting

The Future of Test-Based Educational Accountability

Volumn , Issue , 2008, Pages 139-154

  Haertel, Edward H ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84908915209     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4324/9780203895092     Document Type: Chapter

References (46)

1. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2. Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508-600). Washington, DC: American Council on Education.
3. Beaton, A. E. & Allen, N. (1992). Interpreting scales through scale anchoring. Journal of Educational Statistics, 17 (2), 191-204.
4. Berk, R. A. (1986). A consumer's guide to setting performance standards on criterion-referenced tests. Review of Educational Research, 56 (1), 137-172.
5. Berk, R. A. (1995). Standard setting-the next generation. Proceedings of the Joint Conference on Standard Setting for Large-Scale Assessments, Vol. II (pp. 161-181). Washington, DC:National Assessment Governing Board, National Center for Education Statistics.
6. Braun, H. I. & Qian, J. (2007). An enhanced method for mapping state standards onto the NAEP scale. In N. J. Dorans, M. Pommerich, & P. W. Holland (Eds.), Linking and aligning scores and scales (pp. 313-338). New York: Springer.
7. Chang, L. (1999). Judgmental item analysis of the Nedelsky and Angoff standard-setting methods. Applied Measurement in Education, 12 (2), 151-165.
8. Cizek, G. J. (1993). Reactions to National Academy of Education report, "Setting Performance Standards for Student Achievement." Paper commissioned by the National Assessment Governing Board (ERIC Document Reproduction Service No. ED360397).
9. Glass, G. V. (1978). Standards and criteria. Journal of Educational Measurement, 15 (4), 237-261.
10. Goodwin, L. D. (1999). Relations between observed item difficulty levels and Angoff minimum passing levels for a group of borderline examinees. Applied Measurement in Education, 12 (1), 13-28.
11. Haertel, E. H. (2001). Comment by Edward Haertel [Discussion of "The controversy over the National Assessment Governing Board Standards" by Mark D. Reckase]. In D. Ravitch (Ed.), Brookings Papers on Education Policy: 2001 (pp. 255-262). Washington, DC: Brookings Institute Press.
12. Haertel, E. H. (2002). Standard setting as a participatory process: Implications for validation of standards-based accountability programs. Educational Measurement: Issues and Practice, 21 (1), 16-22.
13. Haertel, E. H. & Lorié, W. A. (2004). Validating standards-based test score interpretations. Measurement:Interdisciplinary Research and Perspectives, 2 (12), 61-103.
14. Hambleton, R. K. (1980). Test score validity and standard-setting methods. In R. A. Berk (Ed.), Criterion-referenced measurement: The state of the art (pp. 80-123). Baltimore: The Johns Hopkins University Press.
15. Hambleton, R. K. & Novick, M. R. (1973). Toward an integration of theory and method for criterion-referenced tests. Journal of Educational Measurement, 10 (3), 159-170.
16. Hambleton, R. K., Brennan, R. L., Brown, W., Dodd, B., Forsyth, R. A., Mehrens, W. A., et al. (2000). A response to "Setting reasonable and useful performance standards" in the National Academy of Sciences' Grading the Nation's Report Card. Educational Measurement:Issues and Practice, 19 (2), 5-14.
17. Ho, A. D. & Haertel, E. H. (2006). Metric-free measures of test score trends and gaps with policyrelevant examples (CSE Report No. 665). Los Angeles: University of California, National Center for Research on Evaluation, Standards and Student Testing (CRESST).
18. Holland, P. (2002). Two measures of change in the gaps between the CDFs of test-score distributions. Journal of Educational and Behavioral Statistics, 27 (1), 3-17.
19. Hoover, H. D. (2003). Some common misconceptions about tests and testing. Educational Measurement:Issues and Practice, 22 (1), 5-14.
20. Impara, J. C. & Plake, B. S. (1998). Teachers' ability to estimate item difficulty: A test of the assumptions in the Angoff standard setting method. Journal of Educational Measurement, 35 (1), 69-81.
21. Kane, M. (1992). An argument-based approach to validation. Psychological Bulletin, 112 (3), 527-535.
22. Kane, M. (1994). Validating the performance standards associated with passing scores. Review of Educational Research, 64 (3), 425-461.
23. Kelly, D. L., Mullis, I. V. S., & Martin, M. O. (2000). Profiles of student achievement in mathematics at the TIMSS international benchmarks: U.S. performance and standards in an international context. Chestnut Hill, MA: Boston College.
24. Kingston, N. M., Kahl, S. R., Sweeney, K. P., & Bay, L. (2001). Setting performance standards using the Body of Work method. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods, and perspectives (pp. 219-248). Mahwah, NJ: Erlbaum.
25. Linn, R. L. (2003a). Accountability: Responsibility and reasonable expectations (CSE Report No. 601). Los Angeles: University of California, National Center for Research on Evaluation, Standards and Student Testing (CRESST).
26. Linn, R. L. (2003b). Performance standards: Utility for different uses of assessments. Education Policy Analysis Archives, 11 (31). Online, available at: http://epaa.asu.edu/epaa/v11n31/ (accessed August 2, 2007).
27. Linn, R. L. & Baker, E. L. (1995). What do international assessments imply for world-class standards? Educational Evaluation and Policy Analysis, 17 (4), 405-418.
28. Linn, R. L. & Shepard, L. A. (1997). Item-by-item standard setting: Misinterpretations of judges' intentions due to less than perfect item inter-correlations. Unpublished manuscript, University of Colorado at Boulder.
29. Livingston, S. A. & Zieky, M. J. (1982). Passing scores: A manual for setting standards of performance on educational and occupational tests. Princeton, NJ: Educational Testing Service.
30. McLaughlin, D. & Bandeira de Mello, V. (2002, April). Comparison of state elementary school mathematics achievement standards using NAEP 2000. Paper presented at the meeting of the American Educational Research Association, New Orleans, LA.
31. McLaughlin, D. H., DuBois, P. A., Eaton, M. S., Ehrlich, D. E., Stancavage, F. B., O'Donnell, C. A., et al. (1993). Comparison of teachers' and researchers' ratings for students' performance in mathematics and reading with NAEP measurement of achievement levels. In R. Glaser, R. Linn, and G. Bohrnstedt (Eds.), Setting performance standards for student achievement: Background studies (pp. 283-364). Stanford, CA: National Academy of Education.
32. Mislevy, R. J. (1998). Implications of market-basket reporting for achievement-level setting. Applied Measurement in Education, 11 (1), 49-63.
33. Mitzel, H. C., Lewis, D. M., Patz, R. J., & Green, D. R. (2001). The Bookmark procedure: Psychological perspectives. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods, and perspectives (pp. 249-281). Mahwah, NJ: Erlbaum.
34. Musick, M. (1997, June). Setting standards high enough. Paper presented at the CCSSO Annual Assessment Conference, Colorado Springs, CO. (ERIC Document Reproduction Service No. ED414309).
35. National Assessment Governing Board. (2004). Mathematics framework for the 2005 National Assessment of Educational Progress. Washington, DC: U.S. Government Printing Office. Online, available at: http://www.nagb.org/pubs/m_framework_05/761607-Math%20Framework.pdf (accessed January 12, 2007).
36. National Center for Education Statistics (2007). Mapping 2005 State Proficiency Standards Onto the NAEP Scales (NCES 2007-482). U.S. Department of Education. Washington, DC:National Center for Education Statistics. Online, available at: http://nces.ed.gov/nationsreportcard/pdf/studies/2007482.pdf (accessed August 2, 2007).
37. Pellegrino, J. W., Jones, L. R., & Mitchell, K. J. (Eds.). Committee on the Evaluation of National and State Assessments of Educational Progress, Board on Testing and Assessment, Commission on Behavioral and Social Sciences and Education, National Research Council. (1999). Grading the nation's report card: Evaluating NAEP and transforming the assessment of educational progress. Washington, DC: National Academy Press.
38. Rosenberg, B. (2004). What's proficient? The No Child Left Behind Act and the many meanings of proficiency. Washington, DC: American Federation of Teachers. Online, available at:http://www.aft.org/pubs-reports/downloads/teachers/WhatsProficient.pdf (accessed January 12, 2007).
39. Schulz, E. M. (2006). Commentary: A response to Reckase's conceptual framework and examples for evaluating standard setting methods. Educational Measurement: Issues and Practice, 25 (3), 4-13.
40. Shepard, L. A., Glaser, R., Linn, R., & Bohrnstedt, G. (1993). Setting performance standards for student achievement (Report of the NAE Panel on the Evaluation of the NAEP Trial State Assessment: An Evaluation of the 1992 Achievement Levels). Stanford, CA: National Academy of Education.
41. Thorndike, E. L. (1918). Specific uses of measurement in the solution of school problems. In G. M. Whipple (Ed.), The seventeenth yearbook of the National Society for the Study of Education, Part II: The measurement of educational products. Bloomington, IL: Public School Publishing.
42. Tversky, A. & Kahneman, D. (1993). Probabilistic reasoning. In A. I. Goldman (Ed.), Readings in philosophy and cognitive science (pp. 43-68). Cambridge, MA: MIT Press.
43. U.S v. South Carolina, 445 F. Supp. 1094 (D.S.C. 1977), aff'd, 434 U.S. 1026 (1978).
44. Webb, N. L. (1997). Criteria for alignment of expectations and assessments in mathematics and science education (Research Monograph No. 6). Madison, WI: University of Wisconsin-Madison, National Institute for Science Education.
45. Webb, N. L. (2002). Alignment study in language arts, mathematics, science, and social studies of state standards and assessments for four states. Washington, DC: Council of Chief State School Officers.
46. Wheeler, P. (1991, November). The relationship between modified Angoff knowledge estimation judgments and item difficulty values for seven NTE specialty area tests. Paper presented at the meeting of the California Educational Research Association, San Diego, CA (available from the ERIC Document Reproduction Service, No. ED340745).