Optimal assembly of psychological and educational tests

Applied Psychological Measurement

Volumn 22, Issue 3, 1998, Pages 195-211

  Van Der Linden, Wim J ^a  

^a UNIVERSITY OF TWENTE   (Netherlands)

Author keywords

0 1 linear programming; Item banking; Item response theory; Network flow programming; Optimal design; Optimal test assembly

Indexed keywords

EID: 0032350534     PISSN: 01466216     EISSN: None     Source Type: Journal    
DOI: 10.1177/01466216980223001     Document Type: Article

References (98)

1. Crowder, H., Johnson, E. L., & Padberg, M. (1983). Solving large-scale linear programming problems: Some algorithmic techniques and computational results. Operations Research, 31, 803-834.
2. Fedorov, V. V. (1972). Theory of optimal experiments. New York: Academic Press.
3. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale NJ: Erlbaum.
4. Michalewicz, Z. (1994). Genetic algorithms + data structures = evolution programs (2nd ed.). New York: Springer-Verlag.
5. Nemhauser, G., & Wolsey, L. (1988). Integer and combinatorial optimization. New York: Wiley.
6. Rao, S. S. (1985). Optimization: Theory and applications. New Delhi, India: Wiley.
7. Schrage, L. (1989). Linear, integer and quadratic programming with LINDO. Redwood City CA: Scientific Press.
8. Wagner, H. M. (1972). Principles of operations research, with applications to managerial decisions. London: Prentice-Hall.
9. Ackerman, T. (1989, March). An alternative methodology for creating parallel test forms using the IRT information function. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco.
10. Adema, J. J. (1988). A note on solving large-scale zero-one programming problems (Research Report No. 88-4). Enschede, The Netherlands: University of Twente, Department of Educational Measurement and Data Analysis.
11. Adema, J. J. (1990a). The construction of customized two-staged tests. Journal of Educational Measurement, 27, 241-253.
12. Adema, J. J. (1990b). Models and algorithms for the construction of achievement tests. Unpublished doctoral dissertation, University of Twente, Enschede, The Netherlands.
13. Adema, J. J. (1992a). Implementations of the branch-and-bound method for test construction. Methodika, 6, 99-117.
14. Adema, J. J. (1992b). Methods and models for the construction of weakly parallel tests. Applied Psychological Measurement, 16, 53-63.
15. Adema, J. J., Boekkooi-Tilmminga, E., & Gademan, A. J. R. M. (1992). Computerized test construction. In M. Wilson (Ed.), Objective measurement: Theory into practice (Vol. 1, pp. 261-273). Norwood NJ: Ablex.
16. Adema, J. J., Boekkooi-Timminga, E., & van der Linden, W. J. (1991). Achievement test construction using 0-1 linear programming. European Journal of Operations Research, 55, 103-111.
17. Adema, J. J., & van der Linden, W. J. (1989). Algorithms for computerized test construction using classical item parameters. Journal of Educational Statistics, 14, 279-290.
18. Armstrong, R. D., & Jones, D. H. (1992). Polynomial algorithms foritem matching. Applied Psychological Measurement, 16, 365-373.
19. Armstrong, R. D., Jones, D. H., & Kunce, C. S. (1998). IRT test assembly using network-flow programming. Applied Psychological Measurement, 22, 237-247.
20. Armstrong, R. D., Jones, D. H., Li, X., & Wu, I.-L. (1996). A study of a network-flow algorithm and a noncorrecting algorithm for test assembly. Applied Psychological Measurement, 20, 89-98.
21. Armstrong, R. D., Jones, D. H., & Wang, Z. (1994). Automated parallel test construction using classical test theory. Journal of Educational Statistics, 19, 73-90.
22. Armstrong, R. D., Jones, D. H., & Wang, Z. (1995). Network optimization in constrained standardized test construction. In K. D. Lawrence (Ed.), Applications of management science: Network optimization applications (Vol. 8, pp. 189-212). Greenwich CT: JAI Press.
23. Armstrong, R. D., Jones, D. H., & Wu, I.-L. (1992). An automated test development of parallel tests. Psychometrika, 57, 271-288.
24. Baker, F. B., Cohen, A. S., & Barmish, B. R. (1988). Item characteristics of tests constructed by linear programming. Applied Psychological Measurement, 12, 189-200.
25. Berger, M. P. F. (1994). A general approach to algorithmic design of fixed-form tests, adaptive tests, and testlets. Applied Psychological Measurement, 18, 141-153.
26. Berger, M. P. F. (1997). Optimal designs for latent variable models: A review. In J. Rost & R. Langeheine (Eds.), Applications of latent trait and latent class models in the social sciences (pp. 71-79). Münster, Germany: Waxmann.
27. Berger, M. P. F. (1998). Optimal design of tests with dichotomous and polytomous items. Applied Psychological Measurement, 22, 248-258.
28. Berger, M. P. F., & Mathijssen, E. (1997). Optimal test designs for polytomously scored items. British Journal of Mathematical and Statistical Psychology, 50, 127-141.
29. Berger, M. P. F., & van der Linden, W. J. (1992). Optimality of sampling design in item response theory models. In M. Wilson (Ed.), Objective measurement: Theory into practice (pp. 274-288). Norwood NJ: Ablex.
30. Berger, M. P. F., & van der Linden, W. J. (1995). Het optimaal ontwerpen van tests met verschillende optimaliteitscriteria [Designing educational tests with different criteria of optimality]. Tijdschrift voor Onderwijsresearch, 20, 79-92.
31. Berger, M. P. F., & Veerkamp, W. J. J. (1996). A review of selection methods for optimal test design. In G. Engelhard, Jr. & M. Wilson (Eds.), Objective measurement: Theory into practice (Vol. 3, pp. 437-456). Norwood NJ: Ablex.
32. Birnbaum, A. (1986). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord and M. R. Novick, Statistical theories of mental test scores. Reading MA: Addison-Wesley.
33. Boekkooi-Timminga, E. (1987a). Simultaneous test construction by zero-one programming. Methodika. 1, 1101-112.
34. Boekkooi-Timminga, E. (1987b). Some methods for simultaneous test construction. In W. J. van der Linden (Ed.), IRT-based test construction (Research Report No. 87-2). Enschede, The Netherlands: University of Twente, Department of Educational Measurement and Data Analysis.
35. Boekkooi-Timminga, E. (1989). Models for computerized test construction. Unpublished doctoral dissertation, University of Twente, Enschede, The Netherlands.
36. Boekkooi-Timminga, E. (1990a). A cluster-based method for test construction. Applied Psychological Measurement, 15, 129-145.
37. Boekkooi-Timminga, E. (1990b). The construction of parallel tests from IRT-based item banks. Journal of Educational Statistics, 15, 129-145.
38. Boekkooi-Timminga, E. (1991, June). A method for designing Rasch model based item banks. Paper presented at the annual meeting of the Psychometric Society, Princeton NJ.
39. Boekkooi-Timminga, E. (1993). Computer-assisted test construction. Social Science Computer Review, 11, 292-300.
40. Boekkooi-Timminga, E., & Sun, L. (1991). CONTEST: A computerized test construction system. In J. Hoogstraten & W. J. van der Linden (Eds.), Methodologie [Methodology] (pp. 69-76). Amsterdam: Stichting Centruum voor Onderwijsonderzoek.
41. Boekkooi-Timminga, E., & van der Linden, W. J. (1988). Algorithms for automated test design. In F. J. Maarse, L. J. M. Mulder, W. P. B. Sjouw, & A. E. Akkerman (Eds.), Computers in psychology: Methods, instrumentation and psychodiagnostics (pp. 171-176). Berwyn PA: Swets Publishing.
42. Boomsma, Y. (1986). Item selection by mathematical programming. Unpublished master's thesis, University of Twente, Enschede, The Netherlands.
43. Cordova, M. J. (1997). Optimization methods in computerized adaptive testing. Unpublished doctoral dissertation, Rutgers University, New Brunswick NJ.
44. de Gruijter, D. N. M. (1990). Test construction by means of linear programming. Applied Psychological Measurement, 14, 175-181.
45. Fan, M. (1997, June). A comparison of computerized test assembly programs for constructing parallel test forms. Paper presented at the annual meeting of the Psychometric Society, Gatlinburg TN.
46. Feuerman, F., & Weiss, H. (1973). A mathematical programming model for test construction and scoring. Management Science, 19, 961-966.
47. Gademann, A. J. R. M. (1987). Item selection using multi-objective programming (OIS Report No. 1). Arnhem, The Netherlands: Cito.
48. Glas, C. A. W. (1988). Psychometric aspects of maintaining standards of examinations. Educational Psychology, 8, 257-270.
49. Kelderman, H. (1987). Some procedures to assess target information functions. In W. J. van der Linden (Ed.), IRT-based test construction (Research Report 87-2). Enschede, The Netherlands: University of Twente, Department of Educational Measurement and Data Analysis.
50. Kester, J. G. (1988). Various mathematical programming approaches toward item selection (OIS Project Report No. 3). Arnhem, The Netherlands: Cito.
51. Luecht, R. D. (1998). Computer-assisted test assembly using optimization heuristics. Applied Psychological Measurement, 22, 224-236.
52. Luecht, R. M., & Hirsch, T. M. (1992). Computerized test construction using an average growth approximation of target information functions. Applied Psychological Measurement, 16, 41-52.
53. Miyaji, I., Nakagawa, Y., & Ohno, K. (1995). Decision support system for composition of the examination problem. European Journal of Operational Research, 80, 130-138.
54. Razoux Schultz, A. F. (1987). Item selection using heuristics (IOS Report No. 2). Arnhem, The Netherlands: Cito.
55. Sanders, P. F., & Verschoor, A. J. (1998). Parallel test construction using classical item parameters. Applied Psychological Measurement, 22, 212-223.
56. Stocking, M. L., & Swanson, L. (1993). A method for severely constrained item selection in adaptive testing. Applied Psychological Measurement, 17, 277-292.
57. Stocking, M. L., & Swanson, L. (1998). Optimal design of item banks for computerized adaptive tests. Applied Psychological Measurement, 22, 271-279.
58. Stocking, M. L., Swanson, L., & Pearlman, M. (1993). Application of an automated item selection method to real data. Applied Psychological Measurement, 17, 167-176.
59. Swanson, L., & Stocking, M. L. (1993). A model and heuristic for solving very large item selection problems. Applied Psychological Measurement, 17, 151-166.
60. Theunissen, T. J. J. M. (1985). Binary programming and test design. Psychometrika, 50, 411-420.
61. Theunissen, T. J. J. M. (1986). Optimization algorithms in test design. Applied Psychological Measurement, 10, 381-389.
62. Theunissen, T. J. J. M. (1996). Combinatorial issues in test construction. Unpublished doctoral dissertation, University of Amsterdam.
63. Theunissen, T. J. J. M., & Verstralen, H. H. F. M. (1986). Algoritmen voor het samenstellen van toetsen [Algorithms for test assembly]. In W. J. van der Linden (Ed.), Moderne methoden van toetsconstructie-en gebruik [Modern methods of test assembly and administration] (pp. 32-39). Amsterdam: Swets & Zeitlinger.
64. Timminga, E. (1998). Solving infeasibility problems in computerized test assembly. Applied Psychological Measurement, 22, 280-291.
65. Timminga, E., & Adema, J. J. (1995). Test construction from item banks (pp. 111-127). In G. H. Fischer & I. W. Molenaar (Eds.), The Rasch model: Foundations, recent developments, and applications. New York: Springer-Verlag.
66. Timminga, E., & Adema, J. J. (1996). An interactive approach to modifying infeasible 0-1 linear programming models for test construction. In G. Engelhard Jr. & M. Wilson (Eds.), Objective measurement: Theory into practice (Vol. 3, pp. 419-436). Norwood NJ: Ablex.
67. Timminga, E., van der Linden, W. J., & Schweizer, D. A. (1996). ConTEST 2.0: A decision support system for item banking and optimal test assembly [Computer software and manual]. Groningen, The Netherlands: iec ProGAMMA.
68. Timminga, E., van der Linden, W. J., & Schweizer, D. A. (1997). ConTEST 2.0 Modules: A decision support system for item banking and optimal test assembly [Computer software and manual]. Groningen, The Netherlands: iec ProGAMMA.
69. van der Linden, W. J. (1987a). Automated test construction using minimax programming. In W. J. van der Linden (Ed.), IRT-based test construction (Research Report 87-2). Enschede, The Netherlands: University of Twente, Department of Educational Measurement and Data Analysis.
70. van der Linden, W. J. (Ed.) (1987b). IRT-based test construction (Research Report 87-2). Enschede, The Netherlands: University of Twente, Department of Educational Measurement and Data Analysis.
71. van der Linden, W. J. (1987c). Models for use in computerized test systems. In J. Moonen & T. Plomp (Eds.), Developments in educational software and courseware (pp. 299-307). Oxford: Pergamon Press.
72. van der Linden, W. J. (1989). Optimaliseringsmodellen voor klassieke toetsconstructie uit een gecalibreerde itembank [Optimization models for classical test construction from a calibrated item bank]. In W. J. van der Linden & L. J. Th. van der Kamp (Eds.), Meetmethoden en data-analyse [Methods of measurement and data analyses] (pp. 33-42). Amsterdam: Swets & Zeitlinger.
73. van der Linden, W. J. (1991). Toetsconstructie als een voorbeeldig ontwerpprobleem [Test construction as an exemplary design problem]. In S. Dijkstra, H. P. M. Krammer, & J. M. Pieters (Eds.), De onderwijskundig ontwerper [The educational designer] (pp. 61-70). Amsterdam: Swets & Zeitlinger.
74. van der Linden, W. J. (1992). Selecting passage-based items for achievement tests (Internal report). Iowa City IA: American College Testing.
75. van der Linden, W. J. (1994a). Computerized educational measurement. In T. Husen & T. N. Postlethwaite (Eds.), International encyclopedia of education (2nd ed., pp. 992-998). Oxford England: Pergamon Press.
76. van der Linden, W. J. (1994b). Optimum design in item response theory: Applications to test assembly and item calibration. In G. H. Fischer & D. Laming (Eds.), Contributions to mathematical psychology, psychometrics, and methodology (pp. 308-318). New York: Springer-Verlag.
77. van der Linden, W. J. (1995). Advances in measurement: Computer applications. In T. Oakland & R. K. Hambleton (Eds.), International perspectives on academic assessment (pp. 105-124). Boston: Kluwer-Nijhof Publishing.
78. van der Linden, W. J. (1996). Assembling tests for the measurement of multiple traits. Applied Psychological Measurement, 20, 373-388.
79. van der Linden, W. J. (1997). Assembling test forms for use in large-scale assessments. Proceedings of the National Assessment Governing Board Achievement Levels Workshop. Washington DC: National Assessment Governing Board.
80. van der Linden, W. J. (1998a). Optimal assembling of tests with item sets. Manuscript submitted for publication.
81. van der Linden, W. J (1998b). Optimal assembly of psychological and educational tests. Applied Psychological Measurement, 22, 195-211.
82. van der Linden, W. J., & Adema, J. J. (in press). Simultaneous assembly of multiple test forms. Journal of Educational Measurement.
83. van der Linden, W. J., & Boekkooi-Timminga, E. (1988). A zero-one programming approach to Gulliksen's matched random subsets method. Applied Psychological Measurement, 12, 201-209.
84. van der Linden, W. J., & Boekkooi-Timminga, E. (1989). A maximin model for test design with practical constraints. Psychometrika, 54, 237-247.
85. van der Linden, W. J., & Carlson, J. E. (1997). Optimizing incomplete block designs for large-scale educational assessments. Unpublished manuscript, University of Twente, Department of Educational Measurement, and Data Analysis, Enschede, The Netherlands.
86. van der Linden, W. J., & Luecht, R. M. (1996). An optimization model for test assembly to match observed-score distributions. In G. Engelhard Jr. & M. Wilson (Eds.), Objective measurement: Theory into practice (Vol. 3, pp. 405-418). Norwood NJ: Ablex.
87. van der Linden, W. J., & Luecht, R. M. (in press). Observed-score equating as a test assembly problem. Psychometrika.
88. van der Linden, W. J., & Reese, L. M. (1998). A model for optimal constrained adaptive testing. Applied Psychological Measurement, 22, 259-270.
89. van der Linden, W. J., Scrams, D. J., & Schnipke, D. L. (1998). Using response-time constraints to control for differential speededness in computerized adaptive testing. Manuscript submitted for publication.
90. van der Linden, W. J., Veldkamp, B. P., & Reese, L. M. (1998). An integer programming approach to item pool design. Manuscript in preparation.
91. van der Linden, W. J., & Zwarts, M. A (1989). Some procedures for computerized ability testing. International Journal of Educational Research, 13, 175-187.
92. Veldkamp, B. P. (1998). Multidimensional test construction using Lagrangian relaxation techniques. Manuscript submitted for publication.
93. Verschoor, A. (1991). OTD: Optimal test design [Computer software and manual]. Arnhem, The Netherlands: Cito.
94. Verschoor, A. (1998). Test construction using genetic algorithms. Unpublished manuscript. Arnhem, The Netherlands: Cito.
95. Votaw, D. F. (1952). Methods of solving some personnel classification problems. Psychometrika, 17, 255-266.
96. Wang, C.-S., & Ackerman, T. A. (1998). Two item selection algorithms for creating parallel test forms. Unpublished manuscript. University of Illinois at Urbana-Champaign, Department of Educational Psychology.
97. Wightman, L. F. (1998). Practical issues in computerized test assembly. Applied Psychological Measurement, 22, 292-302.
98. Yen, W. M. (1983). Use of the three-parameter model in the development of standardized achievement tests. In R. K. Hambleton (Ed.), Applications of item response theory. Vancouver: Educational Research Institute of British Columbia.