Discrepancies between score trends from NAEP and state tests: A scale-invariant perspective

Educational Measurement: Issues and Practice

Volumn 26, Issue 4, 2007, Pages 11-20

  Ho, Andrew D ^a  

^a UNIVERSITY OF IOWA   (United States)

Author keywords

High stakes testing; National assessment of educational progress; No child left behind; Nonparametric statistics; Score trends

Indexed keywords

EID: 36148975795     PISSN: 07311745     EISSN: 17453992     Source Type: Journal    
DOI: 10.1111/j.1745-3992.2007.00104.x     Document Type: Article

References (31)

1. Braun, H., & Mislevy, R. (2005). Intuitive test theory. Phi Delta Kappan, 86 (7 489 497.
2. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York : Lawrence Erlbaum.
3. Education Week (2006). Quality counts at 10: A decade of standards-based education. 25 (17).
4. Ferrara, S. (2005). Editorial. Educational Measurement: Issues and Practice, 24 (1 1 2.
5. Ferrara, S. (2006). Editorial. Educational Measurement: Issues and Practice, 25 (3 1 3.
6. Fuller, B., Gesicki, K., Kang, E., & Wright, J. (2006). Is the No Child Left Behind Act working? The reliability of how states track achievement. Working Paper 06-1. University of California, Berkeley : Policy Analysis for California Education.
7. Gnanadesikan, R. (1977). Methods for statistical data analysis of multivariate observations. New York : John Wiley & Sons.
8. Haertel, E., Thrash, W., & Wiley, D. (1978). Metric-free distributional comparisons. Report. Chicago : ML-Group for Policy Studies in Education.
9. Hall, D., & Kennedy, S. (2006). Primary progress, secondary challenge. Report. Downloaded in March, 2006 from http://www2.edtrust.org/NR/ rdonlyres/15-B22876 - 20C8 - 47B8 - 9AF4-FAB148A225-AC/0/PPSCreport.pdf
10. Hambleton, R. K., & Pitoniak, M. J. (2006). Setting performance standards. In R. L. Brennan (Ed Educational measurement (4th ed., pp. 531 578). Westport CT : American Council on Education and Praeger Publishers.
11. Hanley, J. A., & McNeil, B. J. (1983). A method of comparing the areas under Receiver Operating Characteristic curves derived from the same cases. Radiology, 178, 839 843.
12. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL : Academic Press.
13. Ho, A. D. (2005). Comparing score trends on high-stakes and low-stakes tests using metric-free statistics and multidimensional item response models. Unpublished Doctoral Dissertation. Stanford University.
14. Ho, A. D., & Haertel, E. H. (2006). Metric-free measures of test score trends and gaps with policy-relevant examples. CSE Technical Report #665. University of California, National Center for Research on Evaluation, Standards and Student Testing (CRESST), Los Angeles, CA.
15. Holland, P. (2002). Two measures of change in the gaps between the CDFs of test-score distributions. Journal of Educational and Behavioral Statistics, 27 (1 3 17.
16. Hoover, H. D. (2003). Some common misconceptions about tests and testing. Educational measurement: Issues and Practice, 22 (1 5 14.
17. Klein, S., Hamilton, L., McCaffrey, D., & Stecher, B. (2000). What do test scores in Texas tell us? Report, Santa Monica, CA : The RAND Corporation.
18. Koretz, D. (2005). Alignment, high stakes, and the inflation of test scores. In J. L. Herman & E. H. Haertel (Eds Uses and misuses of data for educational accountability and improvement (104th Yearbook of the National Society of Education, Part 2, pp. 99 118). Malden, MA : Blackwell.
19. Koretz, D., & Barron, S. (1998). The validity of gains on the Kentucky Instructional Results Information System (KIRIS). Report, Santa Monica, CA : The RAND Corporation.
20. Koretz, D., & Hamilton, L. (2006). Testing for accountability in K-12. In R. L. Brennan (Ed Educational measurement (4th ed., pp. 531 578). Westport, CT : American Council on Education and Praeger Publishers.
21. Koretz, D., McCaffrey, D., & Hamilton, L. (2001). Toward a framework for validating gains under high-stakes conditions. CSE Technical Report #551, University of California, National Center for Research on Evaluation, Standards and Student Testing (CRESST), Los Angeles, CA.
22. Lee, J. (2006). Tracking achievement gaps and assessing the impact of NCLB on the gaps: An in-depth look into national and state reading and math outcome trends. Civil Rights Project. Cambridge, MA : Harvard University.
23. Linn, R. L. (2003). Performance standards: Utility for different uses of assessments. Education Policy Analysis Archives, 11 (31). Retrieved March 15, 2007 from http://epaa.asu.edu/epaa/v11n31/.
24. Linn, R. L., Baker, E. L., & Betebenner, D. W. (2002). Accountability systems: Implications of requirements of the No Child Left Behind Act of 2001. Educational Researcher, 31 (6 3 16.
25. Linn, R., Graue, M., & Sanders, N. (1990). Comparing state and district results to national norms: The validity of claims that "everyone is above average". Educational Measurement: Issues and Practice, 9 (3 5 14.
26. Livingston, S. A. (2006). Double P-P Plots for Comparing Differences Between Two Groups. Journal of Educational and Behavioral Statistics, 31 (4 431 435.
27. Macmillan, N. A., & Creelman, C. D. (1996). Triangles in ROC space: History and theory of "nonparametric" measures of sensitivity and response bias. Psychonomic Bulletin and Review, 3 (2 164 170.
28. Marzban, C. (2004). A comment on the ROC curve and the area under it as performance measures. Downloaded in June, 2005 from http://www.csee.wvu.edu/ ∼ross/courses/sp06/biom693/reading/MarzbanROC_2004.pdf
29. Spencer, B. (1983). On interpreting test scores as social indicators: Statistical considerations. Journal of Educational Measurement, 20, 317 333.
30. Thomas B. Fordham Foundation (2005). Gains on state reading tests evaporate on 2005 NAEP. Report. Downloaded in November, 2005 from http://www.edexcellence.net/foundation/about/press_release.cfm?id=19.
31. Wei, X., Lukoff, B., Shen, X., Ho, A. D., & Haertel, E. H. (2006). Using test content to address trend discrepancies between NAEP and California state tests. Paper presented at the 2006 meeting of the American Educational Research Association. San Francisco, CA.