Item selection rules in computerized adaptive testing: Accuracy and security

Methodology

Volumn 5, Issue 1, 2009, Pages 7-17

  Barrada, Juan Ramón ^a   Olea, Julio ^b   Ponsoda, Vicente ^b   Abad, Francisco José ^b  

^a UNIVERSITAT AUTÒNOMA DE BARCELONA   (Spain)

^b UNIVERSIDAD AUTÓNOMA DE MADRID   (Spain)

Author keywords

Computerized adaptive testing; Item exposure control; Item selection; Test security

Indexed keywords

EID: 67549132336     PISSN: 16141881     EISSN: 16142241     Source Type: Journal    
DOI: 10.1027/1614-2241.5.1.7     Document Type: Article

References (48)

1. Barrada, J. R., Mazuela, P., & Olea, J. (2006). Maximum information stratification method for controlling item exposure in computerized adaptive testing. Psicothema, 18, 156-159.
2. Barrada, J. R., Olea, J., & Ponsoda, V. (2004). Reglas de selección de items en tests adaptativos informatizados [Item selection rules in computerized adaptive testing]. Metodología de las Ciencias del Comportamiento, Volumen Especial, 55-61.
3. Barrada, J. R., Olea, J., Ponsoda, V., & Abad, F. J. (2006). Estrategias de seleccio'n de i'tems en un test adaptativo informatizado para la evaluación de inglés escrito [Item selection rules in a computerized adaptive test for the assessment of written English]. Psicothema, 18, 828-834.
4. Barrada, J. R., Olea, J., Ponsoda, V., & Abad, F. J. (2008). Incorporating randomness in Fisher information for improving item exposure control in CATs. British Journal of Mathematical and Statistical Psychology, 61, 493-513.
5. Barrada, J. R., Veldkamp, B. P., & Olea, J. (in press). Multiple maximum exposure rates in computerized adaptive testing. Applied Psychological Measurement.
6. Birnbaum, A. (1968). Some latent ability models and their use in inferring an examinee's ability. In F. M. Lord, & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 392-479). Reading, MA: Addison-Wesley.
7. Bock, R. D., & Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444.
8. Bradley, R. A., & Gart, J. J. (1962). The asymptotic properties of ML estimators when sampling from associated populations. Biometrika, 4, 205-214.
9. Chang, H. H. (2004). Understanding computerized adaptive testing - from Robbins-Monro to Lord and beyond. In David Kaplan (Ed.), The SAGE handbook of quantitative methodology for the social sciences (pp. 117-133). Sage Publications.
10. Chang, H. H., Qian, J., & Ying, Z. (2001). a-Stratified multistage computerized adaptive testing with b blocking. Applied Psychological Measurement, 25, 333-341.
11. Chang, H. H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213-229.
12. Chang, H. H., & Ying, Z. (1999). a-Stratified multistage computerized adaptive testing. Applied Psychological Measurement, 23, 211-222.
13. Chang, H. H., & Ying, Z. (2008). To weight or not to weight? Balancing influence of initial items in adaptive testing. Psychometrika, 73, 441-450.
14. Chang, H. H., & Zhang, J. (2002). Hypergeometric family and item overlap rates in computerized adaptive testing. Psychometrika, 67, 387-398.
15. Chang, S. W., & Ansley, T. N. (2003). A comparative study of item exposure control methods in computerized adaptive testing. Journal of Educational Measurement, 40, 71-103.
16. Chen, S. Y., & Ankenmann, R. D. (2004). Effects of practical constraints on item selection rules at the early stages of computerized adaptive testing. Journal of Educational Measurement, 41, 149-174.
17. Chen, S. Y., Ankenmann, R. D., & Chang, H. H. (2000). A comparison of item selection rules at the early stages of computerized adaptive testing. Applied Psychological Measurement, 24, 241-255.
18. Chen, S. Y., Ankenmann, R. D., & Spray, J. A. (2003). The relationship between item exposure and test overlap in computerized adaptive testing. Journal of Educational Measurement, 40, 129-145.
19. Chen, S. Y., & Lei, P. W. (2005). Controlling item exposure and test overlap in computerized adaptive testing. Applied Psychological Measurement, 29, 204-217. rrada et al.: Item Selection Rules in CATs Methodology 2009; Vol. 5(1):7-17 b 2009 Hogrefe & Huber Publishers
20. Cheng, P. E., & Liou, M. (2000). Estimation of trait level in computerized adaptive testing. Applied Psychological Measurement, 24, 257-265.
21. Cheng, P. E., & Liou, M. (2003). Computerized adaptive testing using the nearest-neighbors criterion. Applied Psychological Measurement, 27, 204-216.
22. Davey, T., & Fan, M. (2000, April). Specific information item selection for adaptive testing. Paper presented at the annual meeting of National Council on Measurement in Education, New Orleans, LA.
23. Davey, T., & Parshall, C. G. (1995, April). New algorithms for item selection and exposure control with adaptive testing. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
24. Dodd, B. G. (1990). The effect of item selection procedure and stepsize on computerized adaptive attitude measurement using the rating scale model. Applied Psychological Measurement, 14, 355-366.
25. Eggen, T. J. H. M. (1999). Item selection in adaptive testing with the sequential probability ratio test. Applied Psychological Measurement, 23, 249-261.
26. Georgiadou, E., Triantafillou, E., & Economides, A. (2007). A review of item exposure control strategies for computerized adaptive testing developed from 1983 to 2005. Journal of Technology, Learning, and Assessment, 5(8), Retrieved June 28, 2007, from http://www.jtla.org.
27. Li, Y. H., & Schafer, W. D. (2005). Increasing the homogeneity of CAT's item-exposure rates by minimizing or maximizing varied target functions while assembling shadow tests. Journal of Educational Measurement, 42, 245-269.
28. Lord, F. M. (1971). Robbins-Monro procedures for tailored testing. Educational and Psychological Measurement, 31, 3-31.
29. Lord, F. M. (1977). A broad-range tailored test of verbal ability. Applied Psychological Measurement, 1, 95-100.
30. Lord, F. M. (1983). Unbiased estimators of ability parameters, of their variance, and their parallel-form reliability. Psychometrika, 48, 233-245.
31. McBride, J. R., & Martin, J. T. (1983). Reliability and validity of adaptive ability tests in a military setting. In D. J. Weiss (Ed.), New horizons in testing (pp. 223-236). New York: Academic Press.
32. Olea, J., & Ponsoda, V. (2003). Tests adaptativos informatizados [Computerized adaptive testing]. Madrid: UNED.
33. Owen, R. J. (1975). A Bayesian sequential procedure for quantal response in the context of adaptive mental testing. Journal of the American Statistical Association, 70, 351-356.
34. Revuelta, J., & Ponsoda, V. (1998). A comparison of item exposure control methods in computerized adaptive testing. Journal of Educational Measurement, 35, 311-327.
35. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika, 34, 100.
36. Samejima, F. (1977). A use of the information function in tailored testing. Applied Psychological Measurement, 1, 233-247.
37. Stocking, M. L.,&Lewis, C. L. (1998). Controlling item exposure conditional on ability in computerized adaptive testing. Journal of Educational and Behavioral Statistics, 23, 57-75.
38. Stocking, M. L., & Lewis, C. L. (2000). Methods of controlling the exposure of items in CAT. In W. J. van der Linden, & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 163-182). Dordrecht, The Netherlands: Kluwer Academic.
39. Sympson, J. B., & Hetter, R. D. (1985, Ocobter). Controlling item exposure rates in computerized adaptive testing. Proceedings of the 27th annual meeting of the Military Testing Association (pp. 973-977). San Diego, CA.
40. van der Linden, W. J. (1998). Bayesian item selection criteria for adaptive testing. Psychometrika, 63, 201-216.
41. van der Linden, W. J., & Glas, C. A. W. (Eds.). (2000a). Computerized adaptive testing: Theory and practice. Norwell, MA: Kluwer.
42. van der Linden, W. J., & Glas, C. A. W. (2000b). Capitalization on item calibration error in adaptive testing. Applied Measurement in Education, 12, 35-53.
43. van der Linden, W. J., & Veldkamp, B. P. (2004). Constraining item exposure in computerized adaptive testing with shadow tests. Journal of Educational and Behavioral Statistics, 29, 273-291.
44. Veldkamp, B. P. (2003). Item selection in polytomous CAT. In H. Yanai, A. Okada, K. Shigemasu, Y. Kano, & J. J. Meulman (Eds.), New developments in psychometrics (pp. 207-214). Tokyo, Japan: Springer-Verlag.
45. Veerkamp, W. J. J., & Berger, M. P. F. (1997). Some new item selection criteria for adaptive testing. Journal of Educational and Behavioral Statistics, 22, 203-226.
46. Wang, T., & Vispoel, W. P. (1998). Properties of ability estimation methods in computerized adaptive testing. Journal of Educational Measurement, 35, 109-135.
47. Way, W. D. (1998). Protecting the integrity of computerized testing item pools. Educational Measurement: Issues and Practice, 17, 17-27.
48. Wingersky, M. S., & Lord, F. M. (1984). An investigation of methods for reducing sampling error in certain IRT procedures. Applied Psychological Measurement, 8, 347-364.