Monte Carlo studies in item response theory

Applied Psychological Measurement

Volumn 20, Issue 2, 1996, Pages 101-125

  Harwell, Michael ^a   Stone, Clement A ^b   Hsu, Tse Chi ^b   Kirisci, Levent ^b  

^a UNIVERSITY OF PITTSBURGH   (United States)

^b NONE

Author keywords

Analysis of variance; Experimental design; Item response theory; Monte Carlo techniques; Multiple regression

Indexed keywords

EID: 0030494344     PISSN: 01466216     EISSN: None     Source Type: Journal    
DOI: 10.1177/014662169602000201     Document Type: Article

References (97)

1. Ackerman, T. A. (1989). Unidimensional IRT calibration of compensatory and noncompensatory multidimensional items. Applied Psychological Measurement, 13, 113-127.
2. Ankemann, R. D., & Stone, C. A. (1991, April). More results on parameter recovery in the graded model using MULTILOG. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco.
3. Ansley, T. N., & Forsyth, R. A. (1985). An examination of the characteristics of unidimensional IRT parameter estimates derived from two-dimensional data. Applied Psychological Measurement, 9, 37-48.
4. Assessment Systems Corporation. (1988). ASCAL: 2- and 3-parameter IRT calibration program. St. Paul MN: Author.
5. Assessment Systems Corporation. (1995). XCALIBRE: Marginal maximum-likelihood estimation program. St. Paul MN: Author.
6. Babbie, E. (1989). The practice of social research (5th ed.). Belmont CA: Wadsworth.
7. Baker, F. B. (1987). Methodology review: Item parameter estimation under the one-, two-, and three-parameter logistic models. Applied Psychological Measurement, 11, 111-141.
8. Baker, F. B. (1989). GENIRV: A program to generate item response vectors. Unpublished manuscript, University of Wisconsin, Laboratory of Experimental Design, Madison.
9. Baker, F. B. (1990). Some observations on the metric of PC-BILOG results. Applied Psychological Measurement, 14, 139-150.
10. Baker, F. B. (1992). Item response theory: Parameter estimation techniques. New York: Marcel Dekker.
11. Baker, F. B., & Al-Karni, A. (1991). A comparison of two procedures for computing IRT equating coefficients. Journal of Educational Measurement, 28, 147-162.
12. Bartlett, M. S., & Kendall, D. G. (1946). The statistical analysis of variance heterogeneity and the logarithmic transformation. Journal of the Royal Society, (Supplement 8), 128-138.
13. Bavry, J. L. (1991). STAT-POWER: User's guide. Chicago: Scientific Software.
14. Bock, R. D. (1991, April). Item parameter estimation. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago.
15. Bock, R. D., & Lieberman, M. (1970). Fitting a response model for n dichotomously scored items. Psychometrika, 35, 179-197.
16. Bogan, D. E., & Yen, W. M. (1983, April). Detecting multidimensionality and examining its effects on vertical equating with the three-parameter logistic model. Paper presented at the annual meeting of the American Educational Research Association, Montreal.
17. Box, G. E. P. (1954). Some theorems on quadratic forms applied in the study of analysis of variance problems. I: Effects of inequality of variance in the one-way classification. Annals of Mathematical Statistics, 25, 290-302.
18. Box, G. E. P., & Muller, M. E. (1958). A note on the generation of random normal deviates. Journal of the Royal Statistical Society, 29, 610-613.
19. Candell, G. L., & Drasgow, F. (1988). An iterative procedure for linking metrics and assessing item bias in item response theory. Applied Psychological Measurement, 12, 253-260.
20. Carlson, J. (1993). GENER: Program to generate compensatory MIRT data. Unpublished program, Educational Testing Service, Princeton NJ.
21. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Chicago: Rand McNally.
22. De Ayala, R. J. (1993). YeomanDG: A data generation program (Version 1.0). Applied Psychological Measurement, 17, 393.
23. De Ayala, R. J. (1994). The influence of multidimensionality on the graded response model. Applied Psychological Measurement, 18, 155-170.
24. De Ayala, R. J., Dodd, B. G., & Koch, W. R. (1990). A simulation and comparison of flexilevel and Bayesian computerized adaptive testing. Journal of Educational Measurement, 27, 227-239.
25. De Vroye, L. (1986). Non-uniform random variate generation. New York: Springer-Verlag.
26. Draper, N. R., & Smith, H. (1981). Applied regression analysis (2nd ed.). New York: Wiley.
27. Drasgow, F. (1989). An evaluation of marginal maximum likelihood estimation for the two-parameter logistic model. Applied Psychological Measurement, 13, 77-90.
28. Drasgow, F., & Parsons, C. K. (1983). Application of unidimensional item response theory models to multidimensional data. Applied Psychological Measurement, 7, 189-199.
29. Fleishman, A. (1978). A method for simulating non-normal distributions. Psychometrika, 43, 521-532.
30. Furqon, S., & Hsu, T. C. (1993). The use of flexible logistic regression when assessing differential item functioning. Unpublished paper, University of Pittsburgh, Department of Psychology in Education.
31. Gifford, J. A., & Swaminathan, H. (1990). Bias and the effect of priors in Bayesian estimation of parameters of item response models. Applied Psychological Measurement, 14, 33-43.
32. Hambleton, R. K., Jones, R. W., & Rogers, H. J. (1993). Influence of item parameter estimation errors in test development. Journal of Educational Measurement, 30, 143-155.
33. Hambleton, R. K., & Rovinelli, R. J. (1973). A Fortran IV program for generating examinee response data from logistic models. Behavioral Science, 17, 73-74.
34. Hammersley, J. M., & Handscomb, D. C. (1964). Monte Carlo methods. London: Methuen.
35. Hartley, H. O. (1977). Solution of statistical distribution problems by Monte Carlo methods. In K. Enslein, A. Ralston, & H. S. Wilf (Eds.), Statistical methods for digital computers (Vol. III; pp. 16-34). New York: Wiley.
36. Harwell, M. R. (1991, April). Analyzing and reporting the results of Monte Carlo studies in educational and psychological research. Paper presented at the annual meeting of the American Educational Research Association, Chicago.
37. Harwell, M. R., & Janosky. J. E. (1991). An empirical study of the effects of small datasets and varying prior variances on item parameter estimation in BILOG. Applied Psychological Measurement, 15, 279-291.
38. Harwell, M. R., Rubinstein, E. N., Hayes, W. S., & Olds, C. C. (1992). Summarizing Monte Carlo results in methodological research: The one- and two-factor fixed effects ANOVA cases. Journal of Educational Statistics, 17, 315-339.
39. Harwell, M. R., & Serlin, R. C. (1989). A nonparametric test statistic for the general linear model. Journal of Educational Statistics, 14. 351-371.
40. Hauck, W. W., & Anderson, S. (1984). A survey regarding the reporting of simulation studies. The American Statistician, 38, 214-216.
41. Hoaglin, D. C., & Andrews, D. F. (1975). The reporting of computation-based results in statistics. American Statistician, 29, 122-126.
42. Holt, J. A., & Macready, G. B. (1989). A simulation study of the difference chi-square statistic for comparing latent class models under violation of regularity conditions. Applied Psychological Measurement, 13, 221-231.
43. Hsu, T. (1993, July). Contributions of Monte Carlo techniques to IRT research: A methodological review. Paper presented at the European meeting of the Psychometric Society, Barcelona.
44. Hulin, C. L., Lissak, R. I., & Drasgow, F. (1982). Recovery of two- and three-parameter logistic item characteristic curves: A Monte Carlo study. Applied Psychological Measurement, 6, 249-260.
45. IMSL, Inc. (1989). IMSL library reference manuals. Houston TX: Author.
46. Jöreskog, K. G., & Sörbom, D. (1989). LISREL 7 user's guide. Chicago: Scientific Software.
47. Kim, S.-H., Cohen, A. S., Baker, F. B., Subkoviak, M. J., & Leonard, T. (1994). An investigation of hierarchical Bayes procedures in item response theory. Psychometrika, 59, 405-421.
48. Kirisci, L. (1992). ITMGNR: A Fortran IV computer program to generate item response data. Unpublished program, University of Pittsburgh.
49. Kirisci, L., Hsu, T., & Furqon, S. (1993, July). Computer resources for Monte Carlo-based IRT research. Paper presented at the European meeting of the Psychometric Society, Barcelona.
50. Lehman, R. S., & Bailey, D. E. (1968). Digital computing: Fortran IV and its applications in behavioral science. New York: Wiley.
51. Levine, M. V., & Rubin, D. B. (1979). Measuring the appropriateness of multiple-choice test scores. Journal of Educational Statistics, 4, 269-290.
52. Lewis, P. A. W., & Orav, E. J. (1989). Simulation methodology for statisticians, operations analysts, and engineers (Vol. 1). Pacific Grove CA: Wadsworth and Brooks/Cole.
53. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading MA: Addison-Wesley.
54. Marsaglia, G. (1964). Generating a variable from the tail of the normal distribution. Technometrics, 6, 101-102.
55. McCauley, C. D., & Mendoza, J. (1985). A simulation study of item bias using a two-parameter item response model. Applied Psychological Measurement, 9, 389-400.
56. McKinley, R. L., & Mills, C. N. (1985). A comparison of several goodness-of-fit statistics. Applied Psychological Measurement, 9, 49-57.
57. Metropolis, N., & Ulam, S. (1949). The Monte Carlo method. Journal of the American Statistical Association, 44, 335-341.
58. Minitab, Inc. (1989). MINITAB handbook. Boston: Duxbury Press.
59. Mislevy, R. J. (1986). Bayes modal estimation in item response models. Psychometrika, 51, 177-194.
60. Mislevy, R. J., & Bock, R. D. (1986). PC-BILOG: Item analysis and test scoring with binary logistic models. Mooresville IN: Scientific Software.
61. Muraki, E. (1992). RESGEN: Item response generator. Unpublished program, Educational Testing Service, Princeton NJ.
62. Nandakumar, R. (1991). Traditional dimensionality versus essential dimensionality. Journal of Educational Measurement, 28, 99-117.
63. Narayanan, P., & Swaminathan, H. (1994). Performance of the Mantel-Haenszel and simultaneous item bias procedures for detecting differential item functioning. Applied Psychological Measurement, 18, 315-328.
64. Naylor, T. H., Balintfy, J. L., Burdick, D. S., & Chu, K. (1968). Computer simulation techniques. New York: Wiley.
65. Newman, T. G., & Odell, P. L. (1971). The generation of random variates. New York: Hafner Press.
66. Oshima, T. C. (1994). The effect of speededness on parameter estimation in item response theory. Journal of Educational Measurement, 31, 200-219.
67. Plake, B. S., & Kane, M. T. (1991). Comparison of methods for combining the minimum passing levels for individual items into a passing score for a test. Journal of Educational Measurement, 28, 249-256.
68. Press, W. H., Flannery, B. P., Teukolsky, S. A., & Vetterling, W. T. (1986). Numerical recipes. Boston: Cambridge University Press.
69. Psychometric Society. (1979). Publication policy regarding Monte Carlo studies. Psychometrika, 44, 133-134.
70. Quails, A. L., & Ansley, T. N. (1985, April). A comparison of item and ability parameter estimates derived from LOGIST and BILOG. Paper presented at the Annual Meeting of the National Conference on Measurement in Education, Chicago.
71. Reckase, M. D., & McKinley, R. L. (1983, April). The definition of difficulty and discrimination for multi-dimensional item response theory models. Paper presented at the annual meeting of the American Educational Research Association, Montreal.
72. Reise, S. P., & Yu, J. (1990). Parameter recovery in the graded response model using MULTILOG. Journal of Educational Measurement, 27, 133-144.
73. Ripley, B. D. (1987). Stochastic simulation. New York: Wiley.
74. Rogers, H. J., & Swaminathan, H. (1993). A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17, 105-116.
75. Rogosa, D. R. (1980). Comparing non-parallel regression lines. Psychological Bulletin, 88, 307-321.
76. Rosnow, R. L., & Rosenthal, R. (1989). Definition and interpretation of interaction effects. Psychological Bulletin, 105, 143-146.
77. Rubinstein, R. V. ( 1981 ). Simulation and the Monte Carlo method. New York: Wiley.
78. Rudner, L. M., Getson, P. M., & Knight, D. L. (1980). Biased item detection techniques. Journal of Educational Statistics, 5, 213-233.
79. SAS Institute, Inc. (1988). SAS user's guide: Statistics. Cary NC: Author
80. Skaggs, G., & Lissitz, R. W. (1988). Effect of examinee ability on test equating invariance. Applied Psychological Measurement, 12, 69-82.
81. Smith, V. K. (1975). Monte Carlo methods: Their role for econometrics. Lexington MA: Lexington Books.
82. Spence, I. (1983). Monte Carlo simulation studies. Applied Psychological Measurement, 7, 405-425.
83. SPSS, Inc. (1993). SPSS for windows: Base system user's guide. Chicago: Author.
84. Stone, C. A. (1992). Recovery of marginal maximum likelihood estimates in the two-parameter logistic response model: An evaluation of MULTILOG. Applied Psychological Measurement, 16, 1-16.
85. Stone, C. A. (1993, July). The use of multiple replications in IRT based Monte Carlo research. Paper presented at the European Meeting of the Psychometric Society, Barcelona.
86. Swaminathan, H., & Rogers, J. A. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361-370.
87. Sympson, J. B. (1978). A model for testing with multidimensional items. In D. J. Weiss (Ed.), Proceedings of the 1977 computerized adaptive testing conference (pp. 82-98). Minneapolis: University of Minnesota, Department of Psychology, Psychometric Methods Program.
88. SYSTAT, Inc. (1991). SYSTAT: The system for statistics. Evanston IN: Author.
89. Thissen, D. (1986). MULTILOG user's guide. Mooresville IN: Scientific Software.
90. Timm, N. H. (1975). Multivariate analysis with applications in education and psychology. Monterey CA: Brooks/Cole.
91. Toothaker, L. E. (1991). Multiple comparisons for researchers. Newbury Park CA: Sage.
92. Vale, C. D., & Maurelli, V. A. (1983). Simulating multi-variate nonnormal distributions. Psychometrika, 48, 465-471.
93. von Neumann, J., & Ulam, S. J. (1949). Various techniques used in connection with random digits. Journal of Research of the National Bureau of Standards, 12, 16-38.
94. Wilcox, R. R. (1988). Simulation as a research technique. In J. P. Keeves (Ed.), Educational research, method ology, and measurement: An international handbook (pp. 134-137). New York: Pergamon.
95. Wingersky, M. S., Barton, M. A., & Lord, F. M. (1982). LOGIST 5.0 version 1.0 user's guide. Princeton NJ: Educational Testing Service.
96. Yen, W. M. (1987). A comparison of the efficiency and accuracy of BILOG and LOGIST. Psvchometrika, 52, 275-291.
97. Zwick, R., Donoghue, J. R., &Grima, A. (1993). Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30, 233-251.