A graphical approach to evaluating equating using test characteristic curves

Applied Psychological Measurement

Volumn 35, Issue 3, 2011, Pages 217-234

  Wyse, Adam E ^a   Reckase, Mark D ^a  

^a Office of Educational Assessment and Accountability   (United States)

Author keywords

Difference that matters (DTM); Equating; Equity; Item response theory; Linking; Test characteristic curves

Indexed keywords

EID: 79953664906     PISSN: 01466216     EISSN: 15523497     Source Type: Journal    
DOI: 10.1177/0146621610377082     Document Type: Article

References (52)

1. Ackerman, T.A. (1994). Using multidimensional item response theory to understand what items and tests are measuring. Applied Measurement in Education, 7, 255-278.
2. Angoff, W.H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508-597). Washington, DC: American Council on Education.
3. Béguin, A.A., Hanson, B.A., & Glas, C.A.W. (2000, April). Effect of multidimensionality on separate and concurrent estimation in IRT equating. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
4. Bolt, D.M. (1999). Evaluating the effects of multidimensionality on IRT true-score equating. Applied Measurement in Education, 12, 383-407.
5. Brennan, R.L. (2008). A discussion of population invariance. Applied Psychological Measurement, 32, 102-114.
6. Briggs, D.C., Weeks, J.P., & Wiley, E.W. (2008, April). The sensitivity of value-added modeling to the creation of vertical score scale. Paper presented at the National Conference on Value-Added Modeling. Madison, WI.
7. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erblaum.
8. Cook, L.L., & Petersen, N.S. (1987). Problems related to the use of conventional and item response theory equating methods in less than optimal circumstances. Applied Psychological Measurement, 11, 225-244.
9. DeMarco, G.L. (1977). Item characteristic curve solutions to three intractable testing problems. Journal of Educational Measurement, 14, 139-160.
10. Divgi, D.R. (1981, April). Two procedures for scaling and equating test with item response theory. Paper presented at the annual meeting of the American Educational Research Association, Los Angeles.
11. Dorans, N.J., & Feigenbaum, M.D. (1994). Equating issues engendered by changes to the SAT and PSAT/ NMSQT. In I. M. Lawrence, N. J. Dorans, M. D. Feigenbaum, N. Feryok, A. P. Schmitt, & N. K. Wright (Eds.), Technical issues related to the introduction of the new SAT and PSAT/NMSQT (ETS RM-94-10). Princeton, NJ: Educational Testing Service.
12. Dorans, N.J., & Holland, P.W. (2000). Population invariance and equitability of tests: Basic theory and the linear case. Journal of Educational Measurement, 37, 281-306.
13. Dorans, N.J., Liu, J., & Hammond, S. (2008). Anchor test type and population invariance: An exploration across subpopulations and test administrations. Applied Psychological Measurement, 32, 81-97.
14. Fraser, C. (1988). NOHARM: A Fortran program for fitting unidimensional and multidimensional normal ogive models in latent trait theory. Armidale, Australia: University of New England, Center for Behavioral Studies.
15. Haebara, T. (1980). Equating logistic ability scales by a weighted least squares method. Japanese Psychological Research, 22, 144-149.
16. Hambleton, R.K., Swaminathan, H., & Rogers, H.J. (1991). Fundamentals of item response theory. New-bury Park, CA: Sage.
17. Hanson, B.A., & Béguin, A.A. (2002). Obtaining a common scale for item response theory item parameters using separate versus concurrent estimation in common-item equating design. Applied Psychological Measurement, 26, 3-24.
18. Harris, D.J., & Crouse, J.D. (1993). A study of criteria used in equating. Applied Measurement in Education, 6, 195-240.
19. Holland, P.W., & Dorans, N.J. (2006). Linking and equating. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 187-220). Washington, DC: American Educational Research Association.
20. Hu, H., Roger, T., & Vukmirovic, Z. (2008). Investigation of IRT-based equating methods in the presence of outlier common items. Applied Psychological Measurement, 32, 311-333.
21. Kane, M.T., & Mroch, A. (2005). Equating the MBE. Bar Examiner, 74, 22-27.
22. Kim, S. (2006). A comparative study of IRT fixed parameter calibration methods. Journal of Educational Measurement, 43, 355-381.
23. Kim, S., & Cohen, A. (1998). A comparison of linking and concurrent calibration under item response theory. Applied Psychological Measurement, 22, 131-143.
24. Kim, S., & Cohen, A. (2002). A comparison of linking and concurrent calibration under the graded response model. Applied Psychological Measurement, 26, 25-41.
25. Kolen, M.J., & Brennan, R.L. (2004). Testing equating, scaling, and linking: Methods and practice (2nd ed.). New York, NY: Springer-Verlag.
26. Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
27. Lord, F.M., & Wingersky, M.S. (1984). Comparison of IRT true-score and equipercentile observed-score "equatings. " Applied Psychological Measurement, 8, 453-461.
28. Loyd, B.H., & Hoover, H.D. (1980). Vertical equating using the Rasch model. Journal of Educational Measurement, 17, 179-193.
29. Liu, M., & Holland, P.W. (2008). Exploring population sensitivity of linking functions across three law school admission test administrations. Applied Psychological Measurement, 32, 27-44.
30. Morris, C.N. (1982). On the foundations of test equating. In P. W. Holland & D. B. Rubin (Eds.), Test equating (pp. 169-191). New York, NY: Academic Press.
31. National Conference of Bar Examiners. (2009). The MBE (Multistate Bar Exam) 2010 information booklet. Retrieved from http://www.ncbex.org/uploads/userdocrepos/MBE2010.pdf.
32. Petersen, N.S. (2008). A discussion of population invariance of equating. Applied Psychological Measurement, 32, 98-101.
33. Reckase, M.D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of Educational Statistics, 4, 207-230.
34. Reckase, M.D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9, 401-412.
35. Reckase, M.D., & McKinley, R.L. (1991). The discriminating power of items that measure more than one dimension. Applied Psychological Measurement, 15, 361-373.
36. Skaggs, G., & Lissitz, R.W. (1986). IRT test equating: Relevant issues and a review of recent research. Review of Educational Research, 56, 495-529.
37. Slinde, J.A., & Linn, R.L. (1978). An exploration of the adequacy of the Rasch model for the problem of vertical equating. Journal of Educational Measurement, 15, 23-35.
38. Stocking, M.L., & Lord, F.M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201-210.
39. Stone, C.A., & Yeh, C. (2006). Assessing the dimensionality of factor structure of multiple-choice exams. Educational and Psychological Measurement, 66, 193-214.
40. Thissen, D., Wainer, H., & Thayer, D.T. (1994). Are tests comprising both multiple-choice and free response items necessarily less unidimensional than multiple-choice tests? Journal of Educational Measurement, 31, 113-123.
41. Thomasson, G. (1993). The asymptotic equating methodology and other test equating evaluation procedures (Unpublished doctoral dissertation). University of Illinois, Urbana-Champaign.
42. Tong, Y., & Kolen, M.J. (2005). Assessing equating results on different equating criteria. Applied Psychological Measurement, 29, 418-432.
43. van der Linden, W.J. (2005). Linear models for optimal test design. New York, NY: Springer-Verlag.
44. van der Linden, W.J., & Luecht, R.M. (1998). Observed-score equating as a test assembly problem. Psychometrika, 63, 401-418.
45. von Davier, A.A., & Liu, M. (2008). Population invariance of test equating and linking: Theory extension and applications across exams. Applied Psychological Measurement, 32, 9-10.
46. von Davier, A.A., & Wilson, C. (2008). Investigating the population sensitivity assumption of item response theory true-score equating across two subgroups of examinees and two test formats. Applied Psychological Measurement, 32, 11-26.
47. Wang, M. (1986, April). Fitting a unidimensional model to multidimensional item response data. Paper presented at the Office of Naval Research Contractors Meeting, Gatlinburg, TN.
48. Weeks, J.P. (2007). plink: IRT separate calibration linking methods (R Package Version 0.0-4).
49. Yang, W.L., & Guo, R. (2008). Invariance of score linkings across gender groups for forms of a testletbased college-level examination program examination. Applied Psychological Measurement, 32, 45-61.
50. Yen, W.M., & Fitzpatrick, A.R. (2006). Item response theory. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 111-153). Washington, DC: American Council on Education.
51. Yi, Q., Harris, D.J., & Gao, X. (2008). Invariance of equating functions across different subgroups of examinees taking a science achievement test. Applied Psychological Measurement, 32, 62-80.
52. Zimowski, M.F., Muraki, E., Mislevy, R.J., & Bock, R.D. (1996). BILOG-MG: Multiple group IRT analysis and test maintenance for binary items [Computer program]. Chicago, IL: Scientific Software International.