Effects of calibration sample size and item bank size on ability estimation in computerized adaptive testing

Kuram ve Uygulamada Egitim Bilimleri

Volumn 15, Issue 6, 2015, Pages 1585-1595

  Şahin, Alper ^a   Weiss, David J ^b  

^a CANKAYA UNIVERSITY   (Turkey)

^b UNIVERSITY OF MINNESOTA   (United States)

Author keywords

Ability estimation accuracy; Calibration sample size; Computerized adaptive testing; Item response theory; Pretest item calibration

Indexed keywords

EID: 84953375121     PISSN: 13030485     EISSN: None     Source Type: Journal    
DOI: 10.12738/estp.2015.6.0102     Document Type: Article

References (26)

1. Akour, M., & Al Omari, H. (2013). Empirical investigation of the stability of IRT item-parameters estimation. International Online Journal of Educational Sciences, 5(2), 291–301.
2. Baker, F. B. (1998). An investigation of the item parameter recovery of a Gibbs sampling procedure. Applied Psychological Measurement, 22(2), 153–169. doi:10.1177/01466216980222005
3. Baker, F. B., & Kim, S. H. (2004). Item response theory: Parameter estimation techniques (2nd ed.). New York, NY: Marcel Dekker.
4. Chuah, S. C., Drasgow F., & Luecht, R. (2006). How big is big enough? Sample size requirements for CAST item parameter estimation. Applied Measurement in Education, 19(3), 241–255. doi:10.1207/s15324818ame1903_5
5. Gao, F., & Chen, L. (2005). Bayesian or non-Bayesian: A comparison study of item parameter estimation in the three-parameter logistic model. Applied Measurement in Education, 18(4), 351–380. doi:10.1207/s15324818ame1804_2
6. Goldman, S. H., & Raju, N. S. (1986). Recovery of one- and two-parameter logistic item parameters: An empirical study. Educational and Psychological Measurement, 46(1), 11–21. doi:10.1177/0013164486461002
7. Guyer, R., & Thompson, N. A. (2011). User’s manual for Xcalibre 4.1. St. Paul, MN: Assessment Systems Corporation.
8. Hambleton, R. K. (1989). Principles and selected applications of item response theory. In R. L. Linn (Ed.), Educational measurement (pp. 147–200). Washington, DC: American Council of Education.
9. Hambleton, R. K., & Swaminathan H. (1985). Item response theory: Principals and applications. Norwell, MA: Kluwer Nijhoff.
10. Harwell, M. R., & Janosky, J. E. (1991). An empirical study of the effects of small datasets and varying prior variances on item parameter estimation in BILOG. Applied Psychological Measurement, 15(3), 279–291. doi:10.1177/014662169101500308
11. Hulin, C. L., Lissak, R. I., & Drasgow, F. (1982). Recovery of two and three-parameter logistic item characteristic curves: A Monte Carlo study. Applied Psychological Measurement, 6(3), 249–260. doi:10.1177/014662168200600301
12. IBM Corporation. (2011). IBM SPSS Statistics for Windows, Version 20.0. Armonk, NY: Author.
13. Lord, F. M. (1968). An analysis of the verbal scholastic aptitude test using Birnbaum’s three-parameter logistic model. Educational and Psychological Measurement, 28(4), 989–1020. doi:10.1177/001316446802800401
14. Patsula, L. N., & Gessaroli M. E. (1995, April). A comparison of item parameter estimates and ICCs produced with TESTGRAF and BILOG under different test lengths and sample sizes. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco.
15. Ree, M. J. (1981). The effects of item calibration sample size and item pool size on adaptive testing. Applied Psychological Measurement, 5(1), 11–19. doi:10.1177/014662168100500102
16. Ree, M. J., & Jensen, H. E. (1980). Effects of sample size on linear equating of item characteristic curve parameters. In D. J. Weiss (Ed.), Proceedings of the 1979 Computerized Adaptive Testing Conference (pp. 249–260). Minneapolis: University of Minnesota.
17. Rudner, L. M. (1998). An on-line, interactive, computer adaptive testing tutorial. Retrieved November 10, 2014 from http://echo.edres.org:8080/scripts/cat/catdemo.htm
18. Swaminathan, H., & Gifford, J. A. (1979). Estimation of parameters in the three-parameter latent trait model (Report No. 90). Amherst, MA: University of Massachusetts, School of Education, Laboratory of Psychometric and Evaluation Research.
19. Swaminathan, H., Hambleton, R. K., Sireci, S. G., Xing, D., & Rizavi, S. M. (2003). Small sample estimation in dichotomous item response models: Effect of priors based on judgmental information on the accuracy of item parameter estimates. Applied Psychological Measurement, 27(1), 27–51. doi:10.1177/0146621602239475
20. Tang, K. L., Way, W. D., & Carey, P. A. (1993). The effect of small calibration sample sizes on TEOFL IRT-based equating (TOEFL technical report TR-7). Princeton, NJ: Educational Testing Service.
21. Thissen, D., & Wainer, H. (1982). Some standard errors in item response theory. Psychometrika, 47(4), 397–412. doi:10.1007/BF02293705
22. Weiss, D. J., & Guyer, R. (2012a). CATSim (version 4.0.6) [Software]. St. Paul, MN: Assessment Systems Corporation.
23. Weiss, D. J., & Guyer, R. (2012b). Manual for CATSim: Comprehensive simulation of computerized adaptive testing. St. Paul, MN: Assessment Systems Corporation.
24. Weiss, D. J., & von Minden, S. V. (2012). A comparison of item parameter estimates from Xcalibre 4.1 and Bilog-MG. St. Paul, MN: Assessment Systems Corporation.
25. Yen, W. M. (1987). A comparison of the efficiency and accuracy of BILOG and LOGIST. Psychometrika, 52(2), 275–291. doi:10.1007/BF02294241
26. Yoes, M. (1995). An updated comparison of micro-computer based item parameter estimation procedures used with the 3-parameter IRT model. Saint Paul, MN: Assessment Systems Corporation.