Using Multidimensional Item Response Theory to Evaluate Educational and Psychological Tests

Educational Measurement: Issues and Practice

Volumn 22, Issue 3, 2003, Pages 37-51

  Ackerman, Terry A ^a   Gierl, Mark J ^b   Walker, Cindy M ^c  

^a UNIVERSITY OF NORTH CAROLINA AT GREENSBORO   (United States)

^b UNIVERSITY OF ALBERTA   (Canada)

^c UNIVERSITY OF WISCONSIN MILWAUKEE   (United States)

Author keywords

Dimensionality; Multidimensional item response theory; Test development and analysis

Indexed keywords

EID: 4644323320     PISSN: 07311745     EISSN: 17453992     Source Type: Journal    
DOI: 10.1111/j.1745-3992.2003.tb00136.x     Document Type: Article

References (62)

1. Ackerman, T. A. (1992) A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. Journal of Educational Measurement, 29, 67–91.
2. Ackerman, T. A. (1994) Using multidimensional item response theory to understand what items and tests are measuring. Applied Measurement in Education, 7, 255–278.
3. Ackerman, T. A. (1996) Graphical representation of multidimensional item response theory analyses. Applied Psychological Measurement, 20, 311–330.
4. Ackerman, T. A., Kelkar, V., Neustel, S., &Simpson, M. (2001) A simulation study examining NOHARM's ability to recover two-dimensional generated item parameters. Paper presented at the annual meeting of the National Council on Measurement in Education, Seattle, WA.
5. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999) Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
6. Balassiano, M., &Ackerman, T. A. (1995) An evaluation of NOHARM estimation accuracy with a two-dimensional latent space. Unpublished manuscript.
7. Bollen, K. A. (1989) Structural equations with latent variables. New York: Wiley.
8. Camilli, G., &Shepard, L. (1994) Methods for identifying biased test items. Newbury Park, CA Sage.
9. Christoffersson, A. (1975) Factor analysis of dichotomized variables. Psychometrika, 40, 5–32.
10. Clauser, B. E., &Mazor, K. M. (1998) Using statistical procedures to identify differentially functioning test items. Educational Measurement: Issues and Practice, 17, 31–44.
11. Computer Associates International, Inc. (1989) DISSPLA 10.0 [Computer software]. Garden City, NY: Authors.
12. de Champlain, A. F., &Tang, K. L. (1997) CHIDIM: A FORTRAN program for assessing the dimensionality of binary item responses based on McDonald's nonlinear factor analytic model. Educational and Psychological Measurement 57, 174–178.
13. Douglas, J., Kim, H. R., Roussos, L., Stout, W., &Zhang, J. (1999) LSAT dimensionality analysis for the December 1991, June 1992, and October, 1992, administrations. (Statistical Report No. 95-05). Newton, PA Law School Admission Council.
14. Fraser, C. (1988) NOHARM: An IBM PC computer program for fitting both unidimensional and multidimensional normal ogive models of latent trait theory. Armidale, Australia: The University of New England.
15. Froelich, A. G. (2000) Assessing the unidimensionality of test items and some asympototics of parametric item response theory. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign, Department of Statistics.
16. Froelich, A. G., &Habing, B. (2001) Refinements of the DIMTEST methodology for testing unidimensionality and local independence. Paper presented at the annual meeting of the National Council on Measurement in Education, Seattle, WA.
17. Gallagher, A. M. (1998) Gender and antecedents of performance in mathematics testing. Teachers College Record, 100, 297–314.
18. Gallagher, A. M., de Lisi, R., Holst, P. C., McGillicuddy-De Lisi, A. V., Morely, M., &Cahalan, C. (2000) Gender differences in advanced mathematical problem solving. Journal of Experimental Child Psychology, 75, 165–190.
19. Gelfand, A. E., &Smith, A. F. M. (1990) Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 85, 398–409.
20. Geman, S., &Geman, D. (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741.
21. Gessaroli, M. E., &De Champlain, Andre F. (1996) Using an approximate chi-square statistic to test the number of dimensions underlying the responses to a set of items. Journal of Educational Measurement, 33, 157–179.
22. Gierl, M. J., Bisanz, J., Bisanz, G., Boughton, K., &Khaliq, S. (2001) Illustrating the utility of differential bundle functioning analyses to identify and interpret group differences on achievement tests. Educational Measurement: Issues and Practice, 20, 26–36.
23. Gierl, M. J., &Khaliq, S. N. (2001) Identifying sources of differential item and bundle functioning on translated achievement tests. Journal of Educational Measurement, 38, 164–187.
24. Hambleton, R. K., &Rovinelli, R. J. (1986) Assessing the dimensionality of a set of test items. Applied Psychological Measurement, 10, 287–302.
25. Harris, D. (1989) Comparison of 1-,2-, and 3-parameter IRT models. Educational Measurement: Issues and Practice, 8, 35–41.
26. Hartz, S., Roussos, L., &Stout, W. (2000) DIMTEST and HCM/CCPROX: Two conditional covariance based dimensionality assessment tools. Handout provided at an instructional workshop conducted at the annual meeting of the National Council on Measurement in Education, New Orleans, LA.
27. Hattie, J. (1984) An empirical study of various indices for determining the unidimensionality. Multivariate Behavioral Research, 19, 49–78.
28. Hattie, J. (1985) Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9, 139–164.
29. Hattie, J., Krakowski, K., Rogers, H. J., &Swaminathan, H. (1996) An assessment of Stout's index of essential unidimensionality. Applied Psychological Measurement, 20, 1–14.
30. Hayduk, L. A. (1987) Structural equation modeling with LISREL. Baltimore, MD: The Johns Hopkins University Press.
31. Kelderman, H., &Rijkes, C. (1994) Loglinear multidimensional IRT models for polytomously scored items. Psychometrika, 59, 149–176.
32. Kim, H. R. (1994) New techniques for the dimensionality assessment of standardized test data. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign, Department of Statistics.
33. Kok, F. (1988) Item bias and test multidimensionality. In R. Langeheine & J. Rost, (Eds.), Latent trait and latent class models (pp. 263–274). New York: Plenum Press.
34. Lord, F. M. (1980) Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
35. McDonald, R. P. (1967) Nonlinear factor analysis. Psychometric Monographs, No. 15.
36. McDonald, R. P. (1997) Normal-ogive multi-dimensional model. In V. J. van der Linden & R. K. Hambleton, (Eds.), Handbook of modern item response theory (pp. 257–269). New York: Springer.
37. McDonald, R. P. (1999) Test theory: A unified approach. Mahwah, NJ: Erlbaum.
38. McDonald, R. P. (2000) A basis for multidimensional item response theory. Applied Psychological Measurement, 24, 99–114.
39. Messick, S. (1989) Validity. In R. L. Linn, (Ed.), Educational measurement (3rd ed., pp. 13–103). New York: American Council on Education/Macmillan.
40. Mislevy, R. J. (1986) Recent developments in the factor analysis of categorical variables. Journal of Educational Statistics, 11, 3–31.
41. Nandakumar, R. (1991) Traditional dimensionality versus essential dimensionality. Journal of Educational Measurement, 28, 99–117.
42. Nandakumar, R., &Stout, W. (1993) Refinement of Stout's procedure for assessing latent trait unidimensionality. Journal of Educational Statistics, 18, 41–68.
43. Nandakumar, R., Yu, F., Li, H., &Stout, W. (1998) Assessing unidimensionality of polytomous data. Applied Psychological Measurement, 22, 99–115.
44. Reckase, M. D. (1985) The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9, 401–412.
45. Reckase, M. D. (1997) The past and future of multidimensional item response theory. Applied Psychological Measurement, 21, 25–36.
46. Reckase, M. D., Carlson, J. E., Ackerman, T. A., &Spray, J. A. (1986) The interpretation of unidimensional IRT parameters when estimated from multidimensional data. Paper presented at the annual meeting of the Psychometric Society, Toronto, ON, Canada.
47. Reckase, M. D., &McKinley, R. L. (1991) The discrimination power of items that measure more than one dimension. Applied Psychological Measurement, 14, 361–373.
48. Roussos, L. (1992) Hierarchical agglomerative clustering computer programs manual. Unpublished manuscript, University of Illinois at Urbana-Champaign, Department of Statistics.
49. Roussos, L., Reese, L., &Harris, V. (1997) Formulation of DETECT'S conditional convariance parameter and evaluation of estimator bias. Unpublished manuscript, Law School Admission Council, Newton, PA.
50. Roussos, L., &Stout, W. (1996) A multidimensionality-based DIF analysis paradigm. Applied Psychological Measurement, 20, 355–371.
51. Shealy, R., &Stout, W. F. (1993) A model based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58, 159–194.
52. Spray, J. A., &Ackerman, T. A. (1986, June) The effect of item response dependency on trait or ability dimensionality. Paper presented at the annual meeting of the Psychometric Society, Toronto, Canada.
53. Stout, W. (1987) A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52, 589–617.
54. Stout, W., Froelich, A. G., &Gao, F. (2001) Using resampling to produce an improved DIMTEST procedure. In A. Boomsma, M. A. J. van Duijn, T. A. B. Snijders, (Eds.), Essays on Item Response Theory (pp. 357–376). New York: Springer-Verlag.
55. Stout, W., Habing, B., Douglas, J., Kim, H. R., Roussos, L., &Zhang, J. (1996) Conditional covariance-based nonparametric multidimensionality assessment. Applied Psychological Measurement, 20, 331–354.
56. Sympson, J. B. (1978) A model for testing with multidimensional items. In D. J. Weiss, (Ed.), Proceedings of the 1977 Computerized Adaptive Testing Conference (pp. 82–98). Minneapolis: University of Minnesota, Department of Psychology, Psychometric Methods Program.
57. Thurstone, L. L. (1947) Multiple factor analysis. Chicago, IL: University of Chicago Press.
58. van der Linden, W. J., &Hambleton, R. K. (1997) Handbook of modern item response theory. New York: Springer.
59. Walker, C. M., &Beretvas, S. N. (2001) An empirical investigation demonstrating the multidimensional DIF paradigm: A cognitive explanation for DIF. Journal of Educational Measurement, 38, 147–163.
60. Walker, C. M., &Beretvas, S. N. (in press). Comparing multidimensional and unidimensional proficiency classifications: Multidimensional IRT as a diagnostic aid. Journal of Educational Measurement.
61. Wilson, D. T., Wood, R., &Gibbons, R. (1991) TESTFACT: Test scoring, item statistics, and item factor analysis. Chicago, IL: Scientific Software International.
62. Zhang, J., &Stout, W. (1999) The theoretical detect index of dimensionality and its application to approximate simple structure. Psychometrika, 64, 231–249.