An NCME instructional module on multistage testing

Educational Measurement: Issues and Practice

Volumn 26, Issue 2, 2007, Pages 44-52

  Hendrickson, Amy ^a  

^a The College Board   (United States)

Author keywords

Adaptive; Multistage; Testlet

Indexed keywords

EID: 34248185569     PISSN: 07311745     EISSN: 17453992     Source Type: Journal    
DOI: 10.1111/j.1745-3992.2007.00093.x     Document Type: Article

References (40)

1. Adema, J. J. (1990). The construction of customized two-stage tests. Journal of Educational Measurement, 27, 241 253.
2. Armstrong, R. D., Jones, D. H., Koppel, N. B., & Pashley, P. J. (2004). Computerized adaptive testing with multiple-form structures. Applied Psychological Measurement, 28, 147 164.
3. Berger, M. P. F. (1994). A general approach to algorithmic design of fixed-form tests, adaptive tests, and testlets. Applied Psychological Measurement, 18, 141 153.
4. Betz, N. E., & Weiss, D. J. (1973). An empirical study of computer-administered two-stage ability testing. Psychometric Methods Program, Research Rep. No. 73-4, Department of Psychology, University of Minnesota, Minneapolis.
5. Birnbaum, A. (1968). Statistical theory for logistic mental test models with a prior distribution of ability. Journal of Mathematical Psychology, 6, 258 276.
6. Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29 51.
7. Bock, R. D., & Zimowski, M.F. (1998). Feasibility studies of two-stage testing in large-scale educational assessment: Implications for NAEP. Commissioned by the NAEP Validity Studies Panel, American Institute for Research in the Behavioral Sciences. Washington, D.C NCES.
8. Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston : Kluwer Nijhoff Publishing.
9. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA : Sage.
10. Kingsbury, G. G. (1996). Item review and adaptive testing. Paper presented at the Annual Meeting of the National Conference on Measurement in Education, New York.
11. Lewis, C., & Sheehan, K. (1990). Using Bayesian decision theory to design a computerized mastery test. Applied Psychological Measurement, 14, 367 386.
12. Linn, R. L., Rock, D. A. Cleary, T. (1969). The development and evaluations of several programmed testing methods. Educational and Psychological Measurement, 29 129 146.
13. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ : Lawrence Erlbaum.
14. Loyd, B. H. (1984). Efficiency and precision in two-stage adaptive testing. West Palm Beach, FL : Eastern ERA.
15. Luecht, R. M. (2003). Exposure control using adaptive multistage item bundles. Paper presented at the Annual Meeting of the National Council on Measurement in Education, Chicago.
16. Luecht, R. M., & Nungester, R. J. (1998). Some practical examples of computer-adaptive sequential testing. Journal of Educational Measurement, 35, 229 249.
17. Luecht, R. M., Nungester, R. J., & Hadidi, A. (1996). Heuristic-based CAT: Balancing item information, content and exposure. Paper presented at the Annual Meeting of the National Council on Measurement in Education, New York.
18. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monograph No. 17). Iowa City, IA : Psychometric Society.
19. Schnipke, D. L., & Reese, L. M. (1997). Comparison of testlet-based test designs for computerized adaptive testing. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago.
20. Sireci, S. G., Thissen, D., & Wainer, H. (1991). On the reliability of testlet-based tests Journal of Educational Measurement, 28, 237 247.
21. Thissen, D., Steinberg, L., & Mooney, J. A. (1989). Trace lines for testlets: A use of multiple-categorical-response models. Journal of Educational Measurement, 26, 247 260.
22. van der Linden, W. J., & Adema, J. J. (1998). Simultaneous assembly of multiple test forms. Journal of Educational Measurement, 35, 185 198.
23. van der Linden, W. J., & Hambleton, R. K. (1997 Handbook of modern item response theory. New York : Springer.
24. Vispoel, W. P. (1998). Reviewing and changing answers on computer-adaptive and self-adaptive vocabulary tests Journal of Educational Measurement, 35, 328 345.
25. Vispoel, W., Hendrickson, A. B., & Bleiler, T. (2000). Limiting answer review and change on computerized adaptive vocabulary tests: Psychometric and attitudinal results. Journal of Educational Measurement, 37, 21 38.
26. Wainer, H. (1990). Computerized adaptive testing: A primer. Hillsdale, NJ : Lawrence Erlbaum.
27. Wainer, H. (1993). Some practical considerations when converting a linearly administered test to an adaptive format. Educational Measurement: Issues and Practice, 12(1), 15 20.
28. Wainer, H. (1995). Precision and differential item functioning on a testlet-based test: The 1991 Law School Admissions Test as an example. Applied Measurement in Education, 8, 157 186.
29. Wainer, H., Bradlow, E. T. Du, Z. (2000). Testlet response theory: An analog for the 3PL model useful in testlet-based adaptive testing. In W. J. van der Linden and C. A. W. Glas (Eds Computerized adaptive testing: Theory and practice (pp. 245 269). Dordrecht, the Netherlands : Kluwer Academic Publishers.
30. Wainer, H., Kaplan, B., & Lewis, C. (1992). A comparison of the performance of simulated hierarchical and linear testlets. Journal of Educational Measurement, 29, 243 251.
31. Wainer, H., Lewis, C., Kaplan, B., & Braswell, J. (1990). An adaptive algebra test: A testlet-based, hierarchically-structured test with validity-based scoring. ETS Technical Report 90-92. Princeton, NJ : ETS.
32. Wainer, H., & Lukhele, R. (1997). How reliable are TOEFL scores? Educational and Psychological Measurement, 57, 741 758.
33. Weiss, D. J. (1974). Strategies of adaptive ability measurement. Psychometric Methods Program, Research Report 74-5, Department of Psychology, University of Minnesota, Minneapolis.
34. Weiss, D. J. (1982). Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement, 6, 473 492.
35. Wise, S. L. (1996). A critical analysis of the arguments for and against item review in computerized adaptive testing. Paper presented at the Annual Meeting of the National Conference on Measurement in Education, New York.
36. Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item dependence Journal of Educational Measurement, 30, 187 214.
37. Kim, H., & Plake, B. S. (1993). Monte Carlo simulation comparison of two-stage testing and computerized adaptive testing. Paper presented at the Annual Meeting of the National Council on Measurement in Education, Atlanta. A study comparing eighteen simulated two-stage and item-level adaptive tests. Shows how characteristics of the first-stage test influence measurement precision of ability. Reviews limitations of item-level CATs and advantages of two-stage tests.
38. Lord, F. M. (1971). A theoretical study of two-stage testing. Psychometrika, 36, 227 242. Discusses considerations in designing a two-stage test and use of the information function for measuring the effectiveness of the test procedure. Provides theoretical results for 5 testing procedures - both two-stage and item-level adaptive tests. Discusses the affects of the test characteristics including possibility of guessing, length of first-stage tests, number of second-stage tests, and difficulty of stages.
39. Lord, F. M. (1974). Practical methods for redesigning a homogeneous test, also for designing a multilevel test. Educational Testing Service RB-74-30. Presents practical methods for designing a multi-level test. Reviews important characteristics of a multistage test including the number and relationship of stages and how to route examinees to levels.
40. Wainer, H., & Kiely, G. L. (1987). Item clusters and computerized adaptive testing: A case for testlets. Journal of Educational Measurement, 24, 185 201. Reviews the development of computerized adaptive testing and the limitations of such tests, including context effects, lack of robustness, and item difficulty ordering. Then develops a model for testlets, both hierarchical and linear, and discusses the relative advantages of testlet-based adaptive tests.