Modifying achievement test items: A theory-guided and data-based approach for better measurement of what students with disabilities know

Peabody Journal of Education

Volumn 84, Issue 4, 2009, Pages 529-551

  Kettler, Ryan J ^a   Elliott, Stephen N ^a   Beddow, Peter A ^a  

^a VANDERBILT UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 75249106343     PISSN: 0161956X     EISSN: None     Source Type: Journal    
DOI: 10.1080/01619560903240996     Document Type: Article

References (39)

1. Abedi, J., Leon, S., & Kao, J. C. (2007, January). Examining differential distractor functioning in reading assessments for students with disabilities.Minneapolis: University of Minnesota, Partnership for Accessible Reading Assessment.
2. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington, DC: Author.
3. American Psychological Association. (1985). Standards for educational and psychological testing. Washington, DC: Author.
4. Assistive Technology Act of 1998, Pub. L. 105-394, Nov. 13, 1998, 112 Stat. 3627 Short title, see 29 U.S.C. 3001 note.
5. Beddow, P. A., Kettler, R. J., & Elliott, S. N. (2008). Test Accessibility and Modification Inventory. Nashville, TN: Vanderbilt University. Retrieved July 18, 2009, from http://peabody.vanderbilt.edu/tami.xml
6. Bennett, R. E. (2001). How the Internet will help large-scale assessment reinvent itself. Education Policy Archives, 9(5). Retrieved July 18, 2009, from http://epaa.asu.edu/epaa/v9n5.html
7. Center for Universal Design. (1997). The principles of universal design. Raleigh: University of North Carolina. Retrieved August 4, 2008, from http://www.design.ncsu.edu/cud
8. Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of instruction. Cognition and Instruction, 8, 293-332.
9. Clark, R., Nguyen, F., & Sweller, J. (2006). Efficiency in learning: Evidence-based guidelines to manage cognitive load. San Francisco, CA: Pfeiffer.
10. Egan, K. L., Ferrara, S., Schneider, M. C., & Barton, K. E. (2009/this issue). Writing performance level descriptors and setting performance standards for assessments of modified achievement standards: The role of innovation and importance of following conventional practice. Peabody Journal of Education, 84(4), 552-577.
11. Elliott, S. N., Kettler, R. J., Beddow, P. A., Compton, E., McGrath, D., Bruen, C., et al. (2009). Using modified items to test students with and without persistent academic difficulties: Effects on groups and individual students. Manuscript submitted for publication
12. Elliott, S. N., Kettler, R. J., & Roach, A. T. (2008). Alternate assessments of modified achievement standards: More accessible and less difficult tests to advance assessment practices? Journal of Disability Policy Studies, 19, 140-152.
13. Elliott, S. N., & Marquart, A. M. (2004). Extended time as a testing accommodation: Its effects and perceived consequences. Exceptional Children, 70(3), 349-367.
14. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (Revised edition). Cambridge, MA: MIT Press.
15. Fulk, C. L., & Smith, P. J. (1995). Students' perceptions of teachers' instructional and management adaptations for students with learning or behavior problems. The Elementary School Journal, 95(5), 409-419.
16. Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309-344.
17. Hess, K., McDivitt, P., & Fincher, M. (2008, June). Who are the 2%students and how do we design items and assessments that provide greater access for them? Results from a pilot study with Georgia students. Paper presented at the CCSSO National Conference on Student Assessment, Orlando, FL.Available from http://www.nciea.org/publications/CCSSO_KHPMMF08.pdf
18. Johnstone, C. J., Bottsford-Miller, N. A., & Thompson, S. J. (2006). Using the think aloud method (cognitive labs) to evaluate test design for students with disabilities and English language learners (Tech. Rep. No. 44). Minneapolis: University of Minnesota, National Center on Educational Outcomes.
19. Johnstone, C. J., Thurlow, M., Moore, M., & Altman, J. (2006). Using systematic item selection methods to improve universal design of assessments (Policy Directions 18). Minneapolis: University of Minnesota, National Center on Educational Outcomes.
20. Kettler, R. J., & Elliott, S. N. (in press). Assessment accommodations for children with special needs. In B. McGaw, E. Baker, & P. Peterson (Eds.), International encyclopedia of education: 3rd edition. Oxford, UK: Elsevier Limited.
21. Kettler, R. J., Rodriguez, M. R., Bolt, D. M., Elliot, S. N., Beddow, P. A., & Kurz, A. (2009). Modifiedmultiple-choice items for alternate assessments: Reliability, difficulty, and the interaction paradigm. Manuscript submitted for publication.
22. Kettler, R. J., Russell, M., Camacho, C., Thurlow, M., Ketterlin Geller, L., Godin, K., et al. (2009, April). Improving reading measurement for alternate assessment: Suggestions for designing research on item and test alterations. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.
23. Koretz, D. M., & Hamilton, L. S. (2006). Testing for accountability in K-12. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 531-578). Santa Barbara, CA: American Council on Education and Praeger.
24. Kosciolek, S., & Ysseldyke, J.E. (2000). Effects of a reading accommodation on the validity of a reading test (Tech. Rep. No. 28). Minneapolis: University of Minnesota, National Center on Educational Outcomes. Retrieved February 6, 2003, from http://education.umn.edu/NCEO/Online_Pubs/Technical28.htm
25. Lang, S. C., Elliott, S. N., Bolt, D. M., & Kratochwill, T. R. (2008). The effects of testing accommodations on students' performances and reactions to testing. School Psychology Quarterly, 14(3), 107-124.
26. Lazarus, S. S., Thurlow, M. L., Lail, K. E., Eisenbraun, K. D., & Kato, K. (2006). 2005 state policies on assessment participation and accommodations for students with disabilities (NCEO Synthesis Rep. No. 64). Minneapolis: University of Minnesota, National Center on Educational Outcomes. Retrieved July 11, 2007, from http://education.umn.edu/NCEO/OnlinePubs/Synthesis64/
27. McKevitt, B. C., & Elliott, S. N. (2003). The use of testing accommodations on a standardized reading test: Effects on scores and attitudes about testing. School Psychology Review, 32(4), 583-600.
28. Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. The Psychological Review, 63, 81-97.
29. Paas, F., Renkl, A., & Sweller, J. (2003).Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38, 1-4.
30. Phillips, S. E. (1994). High-stakes testing accommodations: Validity versus disabled rights. Applied Measurement in Education, 7, 93-120.
31. th ed., pp. 733-757). Santa Barbara, CA: American Council on Education and Praeger.
32. Roach, A. T., Beddow, P. A., Kurz, A., Kettler, R. J., & Elliott, S. N. (2009). Using student responses and perceptions to inform item development for an alternate assessment based on modified achievement standards.Manuscript submitted for publication.
33. Rodriguez, M. C. (2005). Three options are optimal for multiple-choice items: A meta-analysis of 80 years of research. Educational Measurement: Issues and Practice, 24, 3-13.
34. Rodriguez, M. C. (2009, April). The role of item response attractors in modified items for students with disabilities. In S. E. Elliott (Chair), Designing and evaluating modified items for students with disabilities: Research results. Symposium conducted at the annual meeting of the National Council on Measurement in Education, San Diego, CA.
35. Rose, D. H.,&Meyer, A. (2006). A practical reader in universal design for learning. Cambridge,MA: Harvard University Press.
36. Thompson, S. J., Johnstone, C. J., & Thurlow, M. L. (2002). Universal design applied to large scale assessments (Synthesis Rep. No. 44). Minneapolis: University of Minnesota, National Center on Educational Outcomes.
37. U.S. Department of Education. (2007a). Modified academic achievement standards: Non-regulatory guidance.Washington, DC: Author.
38. U.S. Department of Education. (2007b). Standards and assessments peer review guidance. Washington, DC: Author.
39. Web Accessibility Initiative. (2008). Web Content Accessibility Guidelines (WCAG) Version 2.0. Retrieved November 26, 2008, from http://www.w3.org/TR/WCAG20