Writing performance level descriptors and setting performance standards for assessments of modified achievement standards: The role of innovation and importance of following conventional practice

Peabody Journal of Education

Volumn 84, Issue 4, 2009, Pages 552-577

  Egan, Karla L ^a   Ferrara, Steve ^a   Schneider, M Christina ^a   Barton, Karen E ^a  

^a CTB MCGRAW HILL   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 75249105464     PISSN: 0161956X     EISSN: None     Source Type: Journal    
DOI: 10.1080/01619560903241028     Document Type: Article

References (41)

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2. Barton, K. (2006, April). Approaches to developing modified alternate assessments. Paper presented in H. Huynh (Chair), Alternate and modified assessments for accountability and AYP requirements: Policy, technology, and implementation considerations. Symposium conducted at the annual meeting of the National Council on Measurement in Education, San Francisco.
3. Bejar, I. I., Braun, H. I., & Tannenbaum, R. J. (2007). A prospective, progressive, and predictive approach to standard setting. In R. Lissitz (Ed.), Assessing and modeling cognitive development in school (pp. 31-63). Maple Grove, MN: JAM Press.
4. Burstein, L., Koretz, D., Linn, R., Sugrue, B., Novak, J., Baker, E. L., et al. (1996). Describing performance standards: Validity of the 1992 National Assessment of Educational Progress PLDs as characterizations of mathematics performance. Educational Assessment, 3(1), 9-51.
5. Cizek, G. J., & Bunch, M. B. (2007). Standard setting: A guide to establishing and evaluating performance standards on tests. Thousand Oaks, CA: Sage.
6. Crane, E. W., & Winter, P. C. (2006). Setting coherent performance standards. Washington, DC: Chief Council of State School Officers.
7. Elliott, S. N., Kettler, R. J., & Roach, A. T. (2008). Alternate assessments of modified achievement standards: More accessible and less difficult tests to advance assessment practices? Journal of Disability Policy Studies, 19, 140-152.
8. Ercikan, K., & Julian, M. (2002). Classification accuracy of assigning student performance to performance levels: Guidelines for assessment development. Applied Measurement in Education, 15(3), 269-294.
9. Ferrara, S., Perie, M., & Johnson, E. (2008). Matching the judgmental task with standard setting panelist expertise: The Item-Descriptor (ID)Matching procedure. Journal of Applied Testing Technology, 9(1). Retrieved February 11, 2009, from http://www.testpublishers.org/Documents/JATT2008_Ferrara%20et%20al.%20IDM.pdf
10. Ferrara, S., Phillips, G., Williams, P., Leinwand, S., Mahoney, S., & Ahadi, S. (2007). Vertically articulated performance standards: An exploratory study of inferences about achievement and growth. In R. Lissitz (Ed.), Assessing and modeling cognitive development in school (pp. 31-63). Maple Grove, MN: JAM Press.
11. Ferrara, S., Swaffield, S., & Mueller, L. (2009). Conceptualizing and setting performance standards for alternate assessments. In W. D. Schafer & R. W. Lissitz (Eds.), Alternate assessments based on alternate achievement standards: Policy, practice, and potential (pp. 93-111). Baltimore: Paul Brooks Publishing.
12. Ferrara, S., & Wright, L. (2007, July 26). Alternate assessments based on modified achievement standards: Following standard design and development principles enables valid interpretations and decisions. Invited presentation in a meeting of the Special Education Partnership, Washington, DC.
13. Filbin, J. (2008). Lessons from the initial peer review of Alternate Assessments Based onModified Achievement Standards. Washington, DC: USDE.
14. Fincher, M. (2008, June). Georgia's journey towards an AA-MAS. In 1%, 2%: skim or whole assessments. Paper presented at the National Conference on Student Assessment, Orlando, FL. Retrieved November 6, 2008, from http://www.ccsso.org/content/PDFs/14%5FALL.pdf
15. Hambleton, R. K., & Pitoniak, M. (2006). Setting performance standards. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 433-470). Westport, CT: Praeger.
16. HB 05-1246 Study Committee. (1995, December 31). Assessing "students in the gap" in Colorado: Report from the HB 05-1246 Study Committee. Minneapolis, MN: National Center on Educational Outcomes. Retrieved February 11, 2009, from the National Center on Educational OutcomesWeb site: http://cehd.umn.edu/NCEO/Teleconferences/tele11/ColoradoStudy.pdf
17. Kane, M. (2001). So much remains the same: Conception and status of validation in setting standards. In G. Cizek (Ed.), Standard setting: Concepts, methods, and perspectives (pp. 53-88). Mahwah, NJ: Erlbaum.
18. Karantonis, A., & Sireci, S.G. (2006). The Bookmark standard setting method: A literature review. Educational Measurement: Issues and Practice, 25(1), 4-12.
19. Koretz, D., Stecher, B. M., Klein, S. P., & McCaffrey, D. F. (1994). The Vermont portfolio assessment program: Findings and implications. Educational Measurement: Issues and Practice, 13(3), 5-16.
20. Lazarus, S. S., Thurlow, M. L., Christensen, L. L., & Cormier, D. (2007). States' alternate assessments based on modified achievement standards (AA-MAS) in 2007 (Synthesis Rep. No. 67). Minneapolis: University of Minnesota, National Center on Educational Outcomes.
21. Lewis, D. M., & Green, R. (1997, June). The validity of PLDs. Paper presented at the National Conference on Large Scale Assessment, Colorado Springs, CO.
22. Lewis, D. M., Green, D. R., Mitzel, H. C., Baum, K., & Patz, R. J. (1998). The Bookmark Standard Setting Procedure: Methodology and recent implementations. Paper presented at the annual meeting of the Council of State School Officers, Phoenix, AZ.
23. Lewis, D. M., & Haug, C. A. (2005). Aligning policy and methodology to achieve consistent across-grade performance standards. Applied Measurement in Education, 18(1), 11-34.
24. Lewis, D. M., Mitzel, H. C., & Green, D. R. (1996, June). Standard setting: A bookmark approach. Paper presented at the National Conference on Large-Scale Assessment, Phoenix, AZ.
25. Loomis, S. C. (2001, June). Judging evidence of the validity of the National Assessment of Educational Progress Achievement Levels. Paper presented at the National Conference on Student Assessment, Houston, TX.
26. Mercado, R. L., & Egan, K. L. (2005). Performance level descriptors. Paper presented at the National Council on Measurement in Education, Montréal, Quebec, Canada.
27. National Assessment Governing Board. (1995). Developing student performance levels for the National Assessment of Educational Progress: Policy statement. Washington, DC: Author. Retrieved from http://www.nagb.org/policies/PoliciesPDFs/Technical%20Methodology/developing-student-performance.pdf
28. Nickerson, R. S. (2004). Cognition and chance: The psychology of probabilistic reasoning. Mahwah, NJ: Erlbaum.
29. Perie, M. (2008). A guide to understanding and developing PLDs. Educational Measurement: Issues and Practice, 27(4), 15-29.
30. Plake, B. S. (2008). Standard setters: Stand up and take a stand! Educational Measurement: Issues and Practice, 27(1), 3-9.
31. Plous, S. (1993). The psychology of judgment and decision making. New York: McGraw-Hill.
32. Rigney, S. (2008). Federal policy & statewide assessments for students with disabilities. Minneapolis, MN: National Center on Educational Outcomes. Retrieved February 11, 2009, from the National Center on Educational Outcomes Web site http://www.cehd.umn.edu/nceo/Presentations/OSEP2008/OSEPprojDir708.ppt
33. Schneider, M. C., Egan, K. L., Kim, D., & Brandstrom, A. (2008, March). Stability of achievement level descriptors across time and equating methods. Paper presented at the annual meeting of the National Council on Measurement in Education, New York.
34. Shepard, L. A., Glaser, R., Linn, R., & Bohrnstedt, G. (1993). Setting performance standards for student achievement. Report of the NAE Panel on the Evaluation of the NAEP Trial State Assessment: An Evaluation of the 1992 Achievement Levels. Washington, DC: National Academy of Education.
35. Surowiecki, J. (2004). The wisdom of crowds. Boston: Little Brown.
36. U.S. Department of Education. (2007a). Modified academic achievement standards: Non-regulatory guidance. Washington, DC: Author.
37. U.S. Department of Education. (2007b). Standards and assessments peer review guidance: Information and examples
38. for meeting the requirements of the No Child Left Behind Act of 2001. Revised December 21, 2007.Washington, DC: Author.
39. Wang, N. (2003). Use of the Rasch IRT model in standard setting: An item mapping method. Journal of Educational Measurement, 40(3), 231-253.
40. Webb, N., Alt, M., Ely, R., Cormier, M., & Vesperman, B. (2006). The Webb alignment tool: Development, refinement, and dissemination. In Aligning assessments to guide the learning of all students (pp. 1-30). Washington, DC: Council of Chief State School Officers. Retrieved February 11, 2009, from http://www.ccsso.org/publications/details.cfm?PublicationID=293
41. Zieky, M. J., Perie, M., & Livingston, S. A. (2008). Cutscores: A manual for setting standards of performance on educational and occupational tests. Princeton, NJ: Educational Testing Service.