The impact of vertical scaling decisions on growth interpretations

Educational Measurement: Issues and Practice

Volumn 28, Issue 4, 2009, Pages 3-14

  Briggs, Derek C ^a   Weeks, Jonathan P ^a  

^a UNIVERSITY OF COLORADO   (United States)

Author keywords

Accountability; Growth models; Item response theory; Vertical scaling

Indexed keywords

EID: 72249110724     PISSN: 07311745     EISSN: 17453992     Source Type: Journal    
DOI: 10.1111/j.1745-3992.2009.00158.x     Document Type: Article

References (76)

1. Ackerman, T. A. (1992). A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. Journal of Educational Measurement, 29 (1 67 91.
2. Baker, F. B., & Kim, S.-H. (2004). Item response theory: Parameter estimation techniques (2nd ed.). New York : Marcel Dekker.
3. Ballou, D. (2009, in press). Test scaling and value-added measurement Special Issue Education Finance & Policy, 4 (4).
4. Ballou, D., Sanders, W. L., & Wright, P. (2004). Controlling for student background in value-added assessment of teachers. Journal of Educational and Behavioral Statistics, 29 (1 37 66.
5. Béguin, A. A., Hanson, B. A., & Glas, C. A. W. (2000). Effect of multidimensionality on separate and concurrent estimation in IRT equating. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA.
6. Béguin, A. A., & Hanson, B. A. (2001). Effect of noncompensatory multidimensionality on separate and concurrent estimation in IRT observed score equating. Paper presented at the Annual Meeting of the National Council on Measurement in Education, Seattle, WA.
7. Betebenner, D. (2009 : this issue). Norm- and criterion-referenced student growth. Educational Measurement: Issues and Practice, this issue, 42 51.
8. Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds Statistical theories of mental test scores (pp. 397 479). Reading, MA : Addison-Wesley.
9. Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: An application of an EM algorithm. Psychometrika, 46 (4 443 459.
10. Bock, R. D., & Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6 (4 431 444.
11. Bock, R. D., & Zimowski, M. (1997). Multi-group IRT. In W. J. van der Linden and R. K. Hambleton (Eds Handbook of modern item response theory. New York : Springer-Verlag.
12. Braun, H. (2005, September). Using student progress to evaluate teachers: A primer on value-added models [Policy Information Perspective]. Princeton, NJ : Educational Testing Service.
13. Briggs, D. C., & Weeks, J. P. (2009, in press). The sensitivity of value-added modeling to the creation of a vertical scale [Special Issue]. Education Finance & Policy, 4 (4).
14. Briggs, D. C., & Wiley, E. (2008). Causes and effects. In K. Ryan & L. Shepard (Eds The future of test-based educational accountability. New York : Routledge.
15. Camilli, G. (1987). Scale shrinkage and the estimation of latent distribution parameters. Journal of Educational Statistics, 13 (3 227 241.
16. Camilli, G., Yamamoto, K., & Wang, M. M. (1993). Scale shrinkage in vertical equating. Applied Psychological Measurement, 17 (4 379 388.
17. Colorado Department of Education (2006). Colorado Student Assessment Program Technical Report. Accessed January 5, 2009 at http://www.cde.state.co. us/cdeassess/documents/reports/2006/Complete-CSAP-2006-Technical-Report-2006- FINAL.pdf.
18. Divgi, D. R. (1981). Model-free evaluation of equating and scaling. Applied Psychological Measurement, 5 (2 203 208.
19. Gustafsson, J.-E. (1979). The Rasch model in vertical equating of tests: A critique of Slinde and Linn. Journal of Educational Measurement, 16 (3 153 158.
20. Hanson, B. A. (2002). IRT command language. Accessed March 23, 2006 at http://www.b-a-h.com/software/irt/icl/index.html.
21. Hanson, B. A., & Béguin, A. A. (2002). Obtaining a common scale for item response theory item parameters using separate versus concurrent estimation in the common-item equating design. Applied Psychological Measurement, 26 (1 3 24.
22. Holland, P., & Dorans, N. (2006). Linking and equating. In R. L. Brennan (Ed. Educational measurement (4th ed., pp. 187 220). Westport, CT : Praeger.
23. Holmes, S. E. (1982). Unidimensionality and vertical equating with the Rasch model. Journal of Educationa1 Measurement, 19 (2 139 147.
24. Ito, K., Sykes, R., & Yao, L. (2008). Concurrent and separate grade-groups linking procedures for vertical scaling Applied Measurement in Education, 21, 187 206.
25. Karkee, T., Lewis, D. M., Hoskins, M., Yao, L., & Haug, C. (2003). Separate versus concurrent calibration methods in vertical scaling. Paper presented at the Annual Meeting of the National Council on Measurement in Education, Chicago, IL.
26. Kim, S.-H., & Cohen, A. S. (1998). A comparison of linking and concurrent calibration under item response theory. Applied Psychological Measurement, 22 (2 131 143.
27. Kim, J. K., Lee, W.-C., & Kim, D.-I. (2008). The effect of choosing a base grade on the vertical scale using various IRT calibration methods. Paper presented at the Annual Meeting of the National Council on Measurement in Education, New York.
28. Kolen, M. J. (1981). Comparison of traditional and item response theory methods for equating tests. Journal of Educational Measurement, 18 (1 1 11.
29. Kolen, M. J. (2006). Scaling and norming. In R. L. Brennan (Ed Educational measurement (4th ed., pp. 155 186). Westport, CT : Praeger.
30. Kolen, M. J., & Brennan, R. L. (2004). Test equating, scaling and linking (2nd ed.). New York : Springer-Verlag.
31. Lewis, D. M., & Haug, C. A. (2005). Aligning policy and methodology to achieve consistent across-grade performance standards. Applied Measurement in Education, 18 (1 11 43.
32. Li, Y. H., & Lissitz, R. W. (2000). An evaluation of the accuracy of multidimensional IRT linking. Applied Psychological Measurement, 24 (2 115 138.
33. Linacre, J. M., & Wright, B. D. (1998). A user's guide to Bigsteps/Winsteps. Chicago : MESA Press.
34. Linn, R. L. (1993). Linking results of distinct assessments. Applied Measurement in Education, 6 (1 83 102.
35. Lissitz, R. W., & Huynh H. (2003). Vertical equating for state assessments: Issues and solutions in determination of adequate yearly progress and school accountability. Practical Assessment, Research & Evaluation, 8 (10). Accessed January 5, 2009 at http://PAREonline.net/getvn.asp?v=8&n=10.
36. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ : Erlbaum.
37. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA : Addison-Wesley.
38. Loyd, B. H., & Hoover, H. D. (1980). Vertical equating using the Rasch model. Journal of Educational Measurement, 17 (3 179 193.
39. Marco, G. L., Petersen, N. S., & Stewart, E. E. (1983). A test of the adequacy of curvilinear score equating models. In D. Weiss (Ed. New horizons in testing: Latent trait theory and computerized adaptive testing. New York : Academic Press.
40. Martineau, J. A. (2006). Distorting value added: The use of longitudinal, vertically scaled student achievement data for growth-based, value-added accountability. Journal of Educational and Behavioral Statistics, 31 (1 35 62.
41. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47 (2 149 174.
42. McCaffrey, D. F., Lockwood, J. R., Koretz, D. M., & Hamilton, L. S. (2003). Evaluating value-added models for teacher accountability. Santa Monica, CA : RAND Corporation.
43. McCaffrey, D. F., Lockwood, J. R., Koretz, D. M., Louis, T. A., & Hamilton, L. S. (2004). Models for value-added modeling of teacher effects. Journal of Educational and Behavioral Statistics, 29 (1 67 101.
44. Michaelides, M. P., & Haertel, E. (2004). Sampling of common items: An unrecognized source of error in test equating (CSE Report 636). Los Angeles : University of California Center for the Study of Evaluation.
45. Mislevy, R. J. (1992). Linking educational assessments: Concepts, issues, methods, and prospects. Princeton, NJ : ETS Policy Information Center.
46. Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16 (2 159 176.
47. Oshima, T. C., Davey, T., & Lee, K. (2000). Multidimensional linking: Four practical approaches. Journal of Educational Measurement, 37 (4 357 373.
48. Petersen, N. S., Cook, L. L., & Stocking, M. L. (1983). IRT versus conventional equating methods: A comparative study of scale stability. Journal of Educational Statistics, 8 (2 137 156.
49. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen : Danish Institute for Educational Research.
50. Raudenbush, S. W. (2004). Schooling, statistics, and poverty: Can we measure school improvement Paper presented at the William H. Angoff Memorial Lecture Series, Princeton, NJ. Available at http://www.ets.org/Media/Education- Topics/pdf/angoff9.pdf).
51. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA : Sage.
52. Reckase, M. D., & Martineau, J. A. (2004). The vertical scaling of science achievement tests : East Lansing, MI : Center for Education and National Research Council.
53. Rubin, D., Stuart, A., & Zannato, E. (2004). A potential outcome view of value-added assessment in education. Journal of Educational and Behavioral Statistics, 29 (1 103 116.
54. Sanders, W. L., Saxton, A. M., & Horn, S. P. (1997). The Tennessee Value-Added Assessment System: A quantitative outcomes-based approach to educational assessment. In J. Millman (Ed Grading teachers, grading schools: Is student achievement a valid measure? (pp. 137 162). Thousand Oaks, CA : Corwin Press.
55. Seltzer, M., Frank, K., & Bryk, A. (1994). The metric matters: The sensitivity of conclusions concerning growth in student achievement to choice of metric. Educational Evaluation and Policy Analysis, 16 (1 41 49.
56. Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York : Oxford University Press.
57. Skaggs, G., & Lissitz, R. W. (1986). IRT test equating: Relevant issues and a review of recent research. Review of Educational Research, 56 (4 495 529.
58. Slinde, J. A., & Linn, R. L. (1978). An exploration of the adequacy of the Rasch model for the problem of vertical equating. Journal of Educational Measurement, 15 (1 23 35.
59. Slinde, J. A., & Linn, R. L. (1979a). A Note on vertical equating via the Rasch Model for groups of Quite different ability and tests of quite different difficulty. Journal of Educational Measurement, 16 (3 159 165.
60. Slinde, J. A., & Linn, R. L. (1979b). The Rasch model, objective measurement, equating, and robustness. Applied Psychological Measurement, 3 (4 437 452.
61. Spellings, M. (2005, November 18). Secretary Spellings announces Growth Model Pilot, address Chief State School Officers' Annual Policy Forum in Richmond. U.S. Department of Education press release. Accessed January 5, 2009 at http://www.ed.gov/news/pressreleases/2005/11/1182005.html.
62. Spellings, M. (2007, December 8). Secretary Spellings invites eligible states to submit innovative models for expanded Growth Model Pilot. Accessed January 5, 2009 at http://www.ed.gov/news/pressreleases/2007/12/12072007.html.
63. Stocking, M. L., & Lord, F. M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7 (2 201 210.
64. Thissen, D., & Orlando, M. (2001). Item response theory for items scored in two categories. In D. Thissen & H. Wainer (Eds Test scoring (pp. 73 140). Mahwah, NJ : Lawrence Erlbaum Associates.
65. Thissen, D., & Wainer, H., eds 2001). Test scoring. Mahwah, NJ : Lawrence Erlbaum Associates.
66. Tong, Y., & Kolen, M. J. (2007). Comparisons of methodologies and results in vertical scaling for educational achievement tests. Applied Measurement in Education, 20 (2 227 253.
67. von Davier, A. A., Holland, P. W., & Thayer, D. T. (2004). The Kernal method of test equating. New York : Springer-Verlag.
68. Weeks, J. P. (2007). plink: IRT separate calibration linking methods (R package version 0.0-4). Accessed September 10, 2008 at http://cran.r-project. org/web/packages/plink/index.html.
69. Williams, V. S., Pommerich, M., & Thissen, D. (1998). A comparison of developmental scales based on Thurstone methods and item response theory. Journal of Educational Measurement, 35 (2 93 107. (Pubitemid 128184220)
70. Wilson, M. (2004). Constructing measures: An item response modeling approach. Mahwah, NJ : Lawrence Erlbaum Associates.
71. Wingersky, M. S., Barton, M. A., & Lord, F. M. (1982). LOGIST 5.0 version 2.5 users' guide. Princeton, NJ : Educational Testing Service.
72. Wright, B. D. (1977). Misunderstanding the Rasch model. Journal of Educational Measurement, 14 (3 219 225.
73. Wright, B. D (1997). A history of social science measurement. Educational Measurement: Issues and Practice, 16 (4). 33 45.
74. Wright, B. D., & Mead, R. J. (1978). BICAL: Calibrating items and scales with the Rasch model (Research Memorandum No. 23A). Statistical Laboratory, Department of Education, University of Chicago.
75. Yen, W. (1985). Increasing item complexity: A possible cause of scale shrinkage for unidimensional item response theory. Psychometrika, 50 (4 399 410.
76. Yen, W. M. (1986). The choice of scale for educational measurement: An IRT perspective. Journal of Educational Measurement, 23 (4 299 325.