Optimal design in educational testing

Applied Optimal Designs

Volumn , Issue , 2005, Pages 1-19

  Buyske, Steven ^a  

^a RUTGERS UNIVERSITY   (United States)

Author keywords

CAT calibration; Computerized adaptive testing (CAT); Educational testing; Online item calibration; Optimal design; Optimal design theory; Paper and pencil calibration; Polytomous response; Test design

Indexed keywords

EID: 40749155385     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/0470857005.ch1     Document Type: Chapter

References (71)

1. Adema, J. J. (1992). Methods and models for the construction of weakly parallel tests. Applied Psychological Measurement, 16, 53-63.
2. Adema, J. J., Boekkooi-Timminga, E. and van der Linden, W. J. (1991). Achievement test construction using 0-1 linear programming. European Journal of Operational Research, 55, 103-111.
3. Ahuja, R. K., Magnanti, T. L. and Orlin, J. B. (1993). Network Flows: Theory, Algorithms, and Applications. Prentice Hall.
4. Armstrong, R. D., Jones, D. H. and Wu, I. L. (1992). An automated test development of parallel tests from a seed test. Psychometrika, 57, 271-288.
5. Armstrong, R. D., Jones, D. H. and Wang, Z. (1995). Network optimization in constrained standardized test construction. In Lawrence, K. and Reeves, G. R. (eds), Applications of Management Science: Network Applications, vol. 8. JAI Press, pp. 189-212.
6. Armstrong, R. D., Jones, D. H., Li, X. and Wu, I. L. (1996). A study of a network-flow algorithm and a noncorrecting algorithm for test assembly. Applied Psychological Measurement, 20, 89-98.
7. Armstrong, R. D., Jones, D. H. and Kunce, C. S. (1998a). IRT test assembly using networkflow programming. Applied Psychological Measurement, 22, 237-247.
8. Armstrong, R. D., Jones, D. H. and Wang, Z. (1998b). Optimization of classical reliability in test construction. Journal of Educational and Behavioral Statistics, 23, 1-17.
9. Berger, M. P. F. (1991). On the efficiency of IRT models when applied to different sampling designs. Applied Psychological Measurement, 15, 293-306.
10. Berger, M. P. F. (1992). Sequential sampling designs for the two-parameter item response theory model. Psychometrika, 57, 521-538.
11. Berger, M. P. F. (1994a). D-Optimal sequential sampling designs for item response theory models. Journal of Educational Statistics, 19, 43-56.
12. Berger, M. P. F. (1994b). A general approach to algorithmic design of fixed-form tests, adaptive tests and testlets. Applied Psychological Measurement, 18, 141-153.
13. Berger, M. P. F. (1998). Optimal design of test with dichotomous and polytomous items. Applied Psychological Measurement, 22, 248-258.
14. Berger, M. P. F. and Mathijssen, E. (1997). Optimal test designs for polytomously scored items. British Journal of Mathematical and Statistical Psychology, 50, 127-141.
15. Berger, M. P. F. and van der Linden, W. J. (1992). Optimality of sampling designs in item response theory models. In Wilson, M. (ed.), Objective Measurement: Theory into Practice, vol. 1. Ablex, pp. 274-288.
16. Berger, M. P. F., King, C. Y. J. and Wong, W. K. (2000). Minimax D-optimal designs for item response theory models. Psychometrika, 65, 377-390.
17. Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In Lord, F. M. and Novick, M. R. Statistical Theories of Mental Test Scores. Addison-Wesley.
18. Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51.
19. Boekkooi-Timminga, E. (1987). Simultaneous test construction by zero-one programming. Methodika, 1, 101-112.
20. Boekkooi-Timminga, E. (1990). The construction of parallel tests from IRT-based item banks. Journal of Educational Statistics, 15, 129-145.
21. Buyske, S. G. (1998a). Optimal design for item calibration in computerized adaptive testing: the 2PL case. In Flournoy, N., Rosenberger, W. F. and Wong, W. K (eds), New Developments and Applications in Experimental Design. Berkeley: Institute of Mathematical Statistics.
22. Buyske, S. G. (1998b). Optimal designs for item calibration in computerized adaptive testing. Ph.D. dissertation, Rutgers University.
23. Chang, H. H. and Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213-229.
24. Chang, H. H. and Ying, Z. (1999). α-stratified multistage computerized adaptive testing. Applied Psychological Measurement, 23, 211-222.
25. Chang, H. H. and Ying, Z. L. (2004). Nonlinear sequential designs for logistic item response theory models with applications to computerized adaptive tests. Annals of Statistics, to appear.
26. Chang, H. H., Qian, J. and Ying, Z. (2001). α-stratified multistage computerized adaptive testing with b blocking. Applied Psychological Measurement, 25, 333-341.
27. Chen, S. Y., Ankenmann, R. D. and Chang, H. H. (2000). A comparison of item selection rules at the early stages of computerized adaptive testing. Applied Psychological Measurement, 24, 241-255.
28. Feuerman, M. and Weiss, H. (1973). A mathematical programming model for test construction and scoring. Management Science, 19, 961-966.
29. Ford, I. (1976). Optimal static and sequential design: a critical review. Ph.D. thesis, University of Glasgow.
30. Hambleton, R. K. and Swaminathan, H. (1985). Item Response Theory: Principles and Applications. Kluwer.
31. Hambleton, R. K., Jones, R. W. and Rogers, H. J. (1993). Influence of item parameter estimation errors in test development. Journal of Educational Measurement, 30, 143-155.
32. Holman, R. and Berger, M. P. F. (2001). Optimal calibration designs for tests of polytomously scored items described by item response theory models. Journal of Educational and Behavioral Statistics, 26, 361-380.
33. Jones, D. H. and Jin, Z. (1994). Optimal sequential designs for on-line item estimation. Psychometrika, 59, 59-75.
34. Jones, D. H. and Nediak, M. S. (2004). Optimal online calibration of testlets, Chapter 2, this volume.
35. Jones, D. H., Chiang, J. and Jin, Z. (1997). Optimal designs for simultaneous item estimation. Nonlinear Analysis. Theory, Methods and Applications, 30(7), 4051-4058.
36. Jones, D. H., Nediak, M. and Wang, X. B. (1999). Sequential Optimal Designs for On-line Item Calibration. Tech. rept. Rutgers University. Rutcor Research Report 2-99.
37. Lord, F. M. (1962). Estimating norms by item-sampling. Educational and Psychological Measurement, 22, 259-267.
38. Lord, F. M. (1980). Applications of Item Response Theory to Practical Testing Problems. Lawrence Erlbaum Associates.
39. Lord, F. M. and Wingersky, M. S. (1985). Sampling variances and covariances of parameter estimates in item response theory. In Weiss, D. J. (ed.), Proceedings of the 1982 IRT/CAT Conference. University of Minnesota, Department of Psychology, CAT Laboratory.
40. Luecht, R. M. (1998). Computer-assisted test assembly using optimization heuristics. Applied Psychological Measurement, 22, 224-236.
41. Luecht, R. M. and Hirsch, T. M. (1992). Item selection using an average growth approximation of target information functions. Applied Psychological Measurement, 16, 41-51.
42. Nocedal, J. and Wright, S. (1999). Numerical Optimization. Springer-Verlag.
43. Pandey, T. N. and Carlson, D. (1976). Assessing payoffs in the estimation of the mean using multiple matrix sampling designs. In de Gruijter, D. N. M. and van der Kamp, L. J. (eds), Advances in Psychological and Educational Measurement. John Wiley & Sons, Ltd.
44. Parshall, C. G., Spray, J. A., Kalohn, J. C. and Davey, T. (2002). Practical Considerations in Computer-based Testing. Springer-Verlag.
45. Sands, W. A., Waters, B. K. and McBride, J. R. (eds). (1997). Computerized Adaptive Testing: From Inquiry to Operation. American Psychological Association.
46. Stocking, M. L. (1990). Specifying optimum examinees for item parameter estimation in item response theory. Psychometrika, 55, 461-475.
47. Stocking, M. L. and Lewis, C. (2000). Methods of controlling the exposure of items in CAT. In van der Linden, W. J. and Glas, C. A.W. (eds), Computerized Adaptive Testing: Theory and Practice. Kluwer Academic Publishers.
48. Stocking, M. L. and Swanson, L. (1993). A method for severely constrained item selection in adaptive testing. Applied Psychological Measurement, 17, 277-292.
49. Swanson, L. and Stocking, M. L. (1993). A model and heuristic for solving very large item selection problems. Applied Psychological Measurement, 17, 151-166.
50. Sympson, J. B. and Hetter, R. D. (1985). Controlling item-exposure rates in computerized adaptive testing. In Proceedings of the 27th Annual Meeting of the Military Testing Association. San Diego: Navy Personnel Research and Development Centre.
51. Theunissen, T. J. (1985). Binary programming and test design. Psychometrika, 50, 411-420.
52. Theunissen, T. J. (1986). Some applications of optimization algorithms in test design and adaptive testing. Applied Psychological Measurement, 10, 381-389.
53. Thissen, D. and Orlando, M. (2001). Item response theory for items scored in two categories. In Thissen, D. and Wainer, H. Test Scoring. Lawrence Erlbaum Associates, Inc.
54. Thissen, D. and Steinberg, L. (1986). A taxonomy of item response models. Psychometrika, 51, 567-577.
55. Thissen, D, Nelson, L, Rosa, K. and McLeod, L. D. (2001). Item response theory for items scored in more than two categories. In Thissen, D. and Wainer, H. Test Scoring. Lawrence Erlbaum Associates, Inc.
56. Timminga, E. (1998). Solving infeasibility problems in computerized test assembly. Applied Psychological Measurement, 22, 280-291.
57. van der Linden, W. J. (1998). Optimal assembly of psychological and educational tests. Applied Psychological Measurement, 22, 195-211.
58. van der Linden, W. J. (2000). Optimal assembly of tests with item sets. Applied Psychological Measurement, 24, 225-240.
59. van der Linden, W. J. and Adema, J. J. (1998). Simultaneous assembly of multiple test forms. Journal of Educational Measurement, 35, 185-198.
60. van der Linden, W. J. and Boekkooi-Timminga, E. (1989). A maximin model for test design with practical constraints. Psychometrika, 53, 237-247.
61. van der Linden, W. J. and Glas, C. A. W. (2000a). Capitalization on item calibration error in adaptive testing. Applied Measurement in Education, 13, 35-53.
62. van der Linden, W. J. and Hambleton, R. K. (eds). (1997a). Handbook of Modern Item Response Theory. Springer-Verlag.
63. van der Linden, W. J. and Hambleton, R. K. (1997b). Item response theory: brief history, common models, and extensions. In van der Linden, W. J. and Hambleton, R. K. (eds), Handbook of Modern Item Response Theory. Springer-Verlag.
64. van der Linden, W. J. and Reese, L. M. (1998). A model for optimal constrained adaptive testing. Applied Psychological Measurement, 22, 259-270.
65. Veerkamp, W. J. J. and Berger, M. P. F. (1997). Some new item selection criteria for adaptive testing. Journal of Educational and Behavioral Statistics, 22, 203-226.
66. Votaw, D. F. (1952). Methods of solving some personnel problems. Psychometrika, 17, 255-266.
67. Wainer, H. (2000). Computerized Adaptive Testing: a Primer, 2nd edn. Lawrence Erlbaum Associates.
68. Wainer, H. and Einor, D. (2000). Caveats, pitfalls and unexpected consequences of implementing large-scale computerized testing. In Wainer, H., Computerized Adaptive Testing: a Primer, 2nd edn. Lawrence Erlbaum Associates.
69. White, L. V. (1975). The optimal design of experiments for estimation in nonlinear models. Ph.D. thesis, University of London.
70. Wingersky, M. S. and Lord, F. M. (1984). An investigation of methods for reducing sampling error in certain IRT procedures. Applied Psychological Measurement, 8, 347-364.
71. Wynn, H. P. (1970). The sequential generation of D-optimum experimental designs. Annals of Mathematical Statistics, 41, 1655-1664.