A comparison of item selection techniques and exposure control mechanisms in CATs using the generalized partial credit model

Applied Psychological Measurement

Volumn 26, Issue 2, 2002, Pages 147-163

  Pastor, Dena A ^a   Dodd, Barbara G ^a   Chang, Hua Hua ^b  

^a UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^b National Board of Medical Examiners   (United States)

Author keywords

A stratified design; Computerized adaptive testing; Exposure control; Polytomous items; Test security

Indexed keywords

EID: 0037496166     PISSN: 01466216     EISSN: None     Source Type: Journal    
DOI: 10.1177/01421602026002003     Document Type: Article

References (39)

1. Allen, N.L., Kline, D.L., & Zelenak, C.A. (1996). The NAEP 1994 technical report. Washington, DC: National Center for Educational Statistics.
2. Andrich, D. (1978a). Application of a psychometric model to ordered categories which are scored with successive integers. Applied Psychological Measurement, 2, 581-594.
3. Andrich, D. (1978b). A rating formulation for ordered response categories. Psychometrika, 43, 561-573.
4. Bergstrom, B. A., Lunz, M. E., & Gershon, R. C. (1992). Altering the level of difficulty in computer adaptive testing. Applied Measurement in Education, 5, 137-149.
5. Chang, H. H., & Mazzeo, J. (1994). The unique correspondence of the item response function and item category response functions in polytomously scored item response models. Psychometrika, 59(3), 391-404.
6. Chang, H. H., Qian, J., & Ying, Z. (1999). A-stratified multistage CAT with b-blocking. Manuscript accepted for publication.
7. Chang, H. H., & Ying, Z. (1999). A-stratified multistage computerized adaptive testing. Applied Measurement in Education, 23(3), 211-222.
8. Chang, H. H., & Zhang, J. (1999). Hypergeometric family and test overlap rates in computerized adaptive testing. Manuscript submitted for publication.
9. Chang, S. (1998). A comparative study of item exposure control methods in a computerized setting. Unpublished doctoral dissertation, The University of Iowa, Iowa City.
10. Chen, S. (1998). A comparison of maximum likelihood estimation and expected a posteriori estimation in computerized adaptive testing using the generalized partial credit model (Doctoral dissertation, University of Texas). Dissertation Abstracts International: Section B: The Sciences & Engineering, 58(1-B), Jul 1997, 0453.
11. Chen, S.-Y., Ankenmann, R. D., & Spray, J. A. (1999). Exploring the relationship between item exposure rate and test overlap rate in computerized adaptive testing. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada.
12. Davey, T., & Parshall, C. G. (1995, April). New algorithms for item selection and exposure control with computerized adaptive testing. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.
13. Davis, L. L., Pastor, D. A., Chiang, C., Dodd, B. G., & Fitzpatrick, S. (2000). An examination of exposure control and content balancing restrictions on item selection in CATs using the partial credit model. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
14. Dodd, B. G. (1990). The effect of item selection procedure and stepsize on computerized adaptive attitude measurement using the rating scale model. Applied Psychological Measurement, 14, 355-366.
15. Dodd, B. G., & De Ayala, R. J. (1994). Item information as a function of threshold values in the rating scale model. In M. R. Wilson (Ed.), Objective measurement: Theory into practice (Vol. 2, pp. 299-315). Norwood, NJ: Ablex.
16. Dodd, B. G., & Koch, W. R. (1987). Effects of variations in item step values on item and test information in the partial credit model. Applied Psychological Measurement, 11(4), 371-384.
17. Hau, K. T., & Chang, H. H. (1998). Item selection in computerized adaptive testing: Should more discriminating items be used first? Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.
18. Hetter, R. D., & Sympson, J. B. (1997). Item exposure control in CAT-ASVAB. In W. A. Sands, B. K. Waters, & J. R. McBride (Eds.), Computerized adaptive testing: From inquiry to operation (pp. 141-144). Washington, DC: American Psychological Association.
19. Holmes, R. M., & Segall, D. O. (1999). Reducing item exposure without reducing precision (much) in computerized adaptive testing. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada.
20. Koch, W. R., & Dodd, B. G. (1989). An investigation of procedures for computerized adaptive testing using partial credit scoring. Applied Measurement in Education, 2(4), 335-357.
21. Leung, C., Chang, H. H., & Hau, K. (1999). An enhanced stratified computerized adaptive testing design. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada.
22. Leung, C., Chang, H. H., & Hau, K. (2000). Content balancing in stratified computerized adaptive testing designs. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
23. Luecht, R. M. (1998). A framework for exploring and controlling risks associated with test item exposure over time. Paper presented at the annual meeting of the National Council for Measurement in Education, San Diego, CA.
24. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
25. McBride, J. R., & Martin, J. T. (1983). Reliability and validity of adaptive ability tests in a military setting. In D. J. Weiss (Ed.), New horizons in testing (pp. 223-236). New York: Academic Press.
26. Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159-176.
27. Muraki, E., & Bock, R. D. (1993). The PARSCALE computer program [Computer program], Chicago, IL: Scientific Software International.
28. Nering, M. L., Davey, T., & Thompson, T. (1998). A hybrid method for controlling item exposure in computer adaptive testing. Paper presented at the annual meeting of the Psychometric Society, Urbana, IL.
29. Parshall, C. G., Davey, T., & Nering, M. L. (1998). Test development exposure control for adaptive testing. Paper presented at the annual meeting of the National Council on Measurement in Education, San Diego, CA.
30. Parshall, C. G., Hogerty, K. Y., & Kromrey, J. D. (1999). Item exposure in adaptive tests: An empirical investigation of control strategies. Paper presented at the annual meeting of the Psychometric Society, Lawrence, KS.
31. Pastor, D. A., Chiang, C., Dodd, B. G., & Yockey, R. (1999, April). Performance of the Sympson-Hetter exposure control algorithm with a polytomous item bank. Paper presented at the annual meeting of American Educational Research Association, Montreal, Canada.
32. Stocking, M. L. (2000). A framework for comparing adaptive test designs. Unpublished manuscript.
33. Stocking, M. L., & Lewis, C. (1995). A new method of controlling item exposure in computer adaptive testing (Research Report 95-25). Princeton, NJ: Educational Testing Service.
34. Stocking, M. L., & Lewis, C. L. (1998, Spring). Controlling item exposure conditional on ability in computer adaptive testing. Journal of Educational and Behavioral Statistics, 25(1), 57-75.
35. Stocking, M. L., & Lewis, C. L. (2000). Methods of controlling the exposure of items in CAT. In W. J. van der Linden & C.A.W. Glass (Eds.), Computerized adaptive testing: Theory and practice (pp. 163-182). Dordrecht, The Netherlands: Kluwer Academic.
36. Tang, L., Jiarig, H., & Chang, H. (1998, April). A comparison of two methods of controlling item exposure in computerized adaptive testing. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.
37. Thomasson, G. L. (1998). CAT Item exposure control: New evaluation tools, alternate methods and integration into a total CAT program. Paper presented at the annual meeting of the National Council of Measurement in Education, San Diego, CA.
38. Way, W., Zara, A., & Leahy, J. (1996, April). Modifying the NCLEX™ CAT item selection algorithm to improve item exposure. Paper presented at the annual meeting of the American Educational Research Association, New York.
39. Way, W. D. (1998). Protecting the integrity of computerized testing item pools. Educational Measurement: Issues and Practice, 17(4), 17-27.