Measurement issues in the alignment of standards and assessments: A case study

Applied Measurement in Education

Volumn 20, Issue 1, 2007, Pages 101-126

  Herman, Joan L ^a,b   Webb, Noreen M ^a   Zuniga, Stephen A ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b National Center for Research on Evaluation Standards and Student Testing   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 34248358204     PISSN: 08957347     EISSN: None     Source Type: Journal    
DOI: 10.1207/s15324818ame2001_6     Document Type: Conference Paper

References (37)

1. Academic Senate of the California State University. ( 1997). Statement on competencies in mathematics expected of entering college students. Sacramento, CA: Author.
2. Ananda, S. (2003). Achieving alignment. Leadership, 33, 18-21.
3. Bhola, D. S., Impara, J. C., & Buckendahl, C. W. (2003). Aligning tests with states' content standards: Methods and issues. Educational Measurement: Issues and Practice, 22(3), 21-29.
4. Brennan, R. L. (2001). Generalizability theory. New York: Springer-Verlag.
5. Brennan, R. L., & Kane, M. F. (1977). An index of dependability for mastery tests. Journal of Educational Measurement, 14, 277-289.
6. Buckendahl, C. W., Plake, B. S., Impara, J. C., & Irwin, P. M. (2000, April). Alignment of standardized achievement tests to state content standards: A comparison of publishers 'and teachers 'perspectives. Paper presented at the Annual Meeting of the National Council on Measurement in Education. New Orleans, LA.
7. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284-290.
8. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.
9. California Department of Education. (1997). Mathematics content standards for California public schools, kindergarten through grade twelve. Sacramento, CA: Author.
10. California Department of Education. (2001 a). Golden State Examination in High School Mathematics. Sacramento, CA: Author.
11. California Department of Education. (2001b). High school mathematics blueprint, draft. Sacramento, CA: Author.
12. California Department of Education. (2001c). Overview specifications for high school mathematics, draft. Sacramento, CA: Author.
13. Feiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76, 378-382.
14. Fleiss, J. L. (1981). Balanced incomplete block designs for inter-rater reliability studies. Applied Psychological Measurement, 5, 105-112.
15. Glaser, R., & Baxter, G. P. (2000). Assessing active knowledge (CSE Tech. Rep. No. 516). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.
16. Glaser, R., & Chi, M. T. H. (1988). Overview. In M. T. H. Chi, R. Glaser, & M. J. Farr (Eds.), The nature of expertise (pp. xv-xxviii). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
17. Herman, J. L. (2004). The effects of testing in instruction. In S. Fuhrman & R. Elmore (Eds.), Redesigning accountability systems for education (pp. 141-165). New York: Teachers College Press.
18. Herman, J. L., Webb, N. M., & Zuniga, S. A. (2003). Alignment and college admissions: The match of expectations, assessments, and educator perspectives (CSE Tech. Rep. No. 593). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.
19. Koretz, D. M., Mitchell, K. J., Barron, S., & Keith, S. (1996). Perceived effects of the Maryland State Assessment Program (CSE Tech. Rep. No. 406). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.
20. Lane, S., Stone, C., Parke, C., Hansen, M., & Cerillo, T. (2000, April). Consequential evidence for MSPAP from the teacher, principal and student perspective. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, LA.
21. McDonnell, L. M., & Choisser, C. (1997). Testing and teaching: Local implementation of new state assessments (CSE Tech. Rep. No. 442). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.
22. No Child Left Behind Act of 2001, Pub. L. No. 107-110, 115 Stat. 1425 (2002).
23. Olson, L. (2003, Spring). Standards and tests: Keeping them aligned. Research Points: Essential Information for Education Policy, 1(1).
24. Porter, A. C. (2002). Measuring the content of instruction: Uses in research and practice. Educational Researcher, 31(1), 3-14.
25. Porter, A. C., & Smithson, J. L. (2001). Defining, developing, and using curriculum indicators. CPRE Research Report Series. Philadelphia: Consortium for Policy Research in Education.
26. Rothman, R., Slattery, J. B., Vranek, J. L., & Resnick, L. B. (2002). Benchmarking and alignment of standards and testing (CSE Tech. Rep. No. 566). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.
27. Shavelson, R. J., Baxter, G. P., & Gao, X. (1993). Sampling variability of performance assessments. Journal of Educational Measurement, 30, 215-232.
28. Shavelson, R. J., & Ruiz-Primo, M. A. (2000). On the psychometrics of assessing science understanding. In J. J. Mintzes, J. H. Wandersee, & J. D. Novak (Eds.), Assessing science understanding (pp. 303-341). San Diego, CA: Academic.
29. Shavelson, R. J., Ruiz-Primo, M. A., & Wiley, E. W. (1999). Note on sources of sampling variability in science performance assessments. Journal of Educational Measurement, 36, 61-71.
30. Shavelson, R. J., & Webb, N. M. ( 1991 ). Generalizability theory: A primer. Newbury Park, CA: Sage.
31. Stecher, B., Barron, S., Chun, T., & Ross, K. (2000). The effects of the Washington State Education Reform on schools and classrooms (CSE Tech. Rep. No. 525). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.
32. Watkins, M. W., & Pacheco, M. (2001). Interobserver agreement in behavioral research: Importance and calculation. Journal of Behavioral Education, 10, 205-212.
33. Webb, N. L. (1997). Criteria for alignment of expectations and assessments in mathematics and science education. Council of Chief State School Officers and National Institute for Science Education. Madison: University of Wisconsin, Wisconsin Center for Education Research.
34. Webb, N. L. (1999). Alignment of science and mathematics standards and assessments in four states. Council of Chief State School Officers and National Institute for Science Education, Madison: University of Wisconsin, Wisconsin Center for Education Research.
35. Webb, N. L. (2002, April). An analysis of the alignment between mathematics standards and assessments for three states. Paper presented at the American Educational Research Association Annual Meeting. New Orleans, LA.
36. Webb, N. L. (2005). Alignment analysis of mathematics standards and assessments. Report submitted to the State of Michigan. Madison: Wisconsin Center for Educational Research.
37. Wixson, K. K., Fisk, M. C., Dutro, E., & McDaniel, J. (2002). The alignment of state standards and assessments in elementary reading. Ann Arbor, MI: Center for the Improvement of Early Reading Achievement.