Best Design for Multidimensional Computerized Adaptive Testing With the Bifactor Model

Educational and Psychological Measurement

Volumn 75, Issue 6, 2015, Pages 954-978

  Seo, Dong Gi ^a   Weiss, David J ^b  

^a National Registry of Emergency Medical Technicians (NREMT)   (United States)

^b UNIVERSITY OF MINNESOTA   (United States)

Author keywords

bifactor model; computerized adaptive testing; full information item factor analysis; multidimensional item response theory

Indexed keywords

EID: 84947460548     PISSN: 00131644     EISSN: 15523888     Source Type: Journal    
DOI: 10.1177/0013164415575147     Document Type: Article

References (56)

1. T.A.Ackerman (1989). Unidimensional IRT calibration of compensatory and noncompensatory multidimensional items. Applied Psychological Measurement, 13, 113-127.
2. T.W.Anderson (1984). An introduction to multivariate statistical analysis (2nd ed.). New York, NY: John Wiley.
3. R.A.AnsleyT.N.Forsyth (1985). An examination of the characteristics of unidmensional IRT parameter estimates derived from two-dimensional data. Applied Psychological Measurement, 9, 37-48.
4. F.B.Baker (1992). Item response theory parameter estimation techniques. New York, NY: Marcel Dekker.
5. B.M.BloxomC.D.Vale (1987, June). Multidimensional adaptive testing: A procedure for sequential estimation of the posterior centroid and dispersion of theta. Paper presented at the annual meeting of the Psychometric Society, Montreal, Quebec, Canada.
6. R.D.BockM.Aitkin (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443-459.
7. R.D.BockR.J.Mislevy (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444.
8. M.W.Browne (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavior Research, 36, 111-150.
9. L.CaiJ.YangM.Hansen (2011). Generalized full-information item bifactor analysis. Psychological Methods, 16, 221-248.
10. G.B.CassanoS.MicheliniM.K.ShearE.ColiJ.D.MaserE.Frank (1997). The panic-agoraphobic spectrum: A descriptive approach to the assessment and treatment of subtle symptoms. American Journal of Psychiatry, 154, 27-38.
11. R.P.Chalmers (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1-29. Retrieved from http://www.jstatsoft.org/v48/i06
12. H.-H.ChangW.J.van der Linden (2003). Optimal stratification of item pools in a-stratified computerized adaptive testing. Applied Psychological Measurement, 27, 262-274.
13. F.F.ChenS.G.WestK.H.Sousa (2006). A comparison of bifactor and second- order models of quality of life. Multivariate Behavioral Research, 41, 189-224.
14. P.T.CostaR.R.McCrae (1992). NEO PI-R. Professional manual. Odessa, FL: Psychological Assessment Resources.
15. C.E.DeMars (2007). “Guessing” parameter estimates for multidimensional item response theory models. Educational and Psychological Measurement, 67, 433-446.
16. V.G.FolkB.F.Green (1989). Adaptive estimation when the unidimensionality assumption of IRT is violated. Applied Psychological Measurement, 13, 373-390.
17. A.GenzF.Bretz (2009). Computation of multivariate normal and t probabilities. Lecture notes in statistics (Vol. 195). Heidelberg, Germany: Springer-Verlag.
18. R.D.GibbonsR.D.BockD.HedekerD.J.WeissE.SegawaD.K.BhaumikA.. Stover (2007). Full-information item bifactor analysis of graded response data. Applied Psychological Measurement, 31, 4-19.
19. R.D.GibbonsD.R.Hedeker (1992). Full-information item bifactor analysis. Psychometrika, 57, 423-436.
20. R.D.GibbonsD.J.WeissD.J.KupferE.FrankA.FagioliniV.J.GrochocinskiJ.C.. Immekus (2008). Using computerized adaptive testing to reduce the burden of mental health assessment. Psychiatric Services, 59(4), 49-58.
21. J.E.GustafssonL.Aberg-Bengtsson (2010). Unidimensionality and the interpretability of psychological instruments. In S.E.Embretson (Ed.), Measuring psychological constructs (pp. 97-121). Washington, DC: American Psychological Association.
22. H.B.Heywood (1931). On finite sequences of real numbers. Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character, 134, 486-501.
23. K.J.HolzingerF.Swineford (1937). The bifactor method. Psychometrika, 2, 41-54.
24. J.L.Horn (1986). Intellectual ability concepts. In R.J.Sternberg (Ed.), Advances in the psychology of human intelligence (Vol. 3, pp. 35-78). Hillsdale, NJ: Lawrence Erlbaum.
25. H.-Y.HuangP.-H.ChenW.-C.Wang (2012). Computerized adaptive testing using a class of high-order item response theory models. Applied Psychological Measurement, 36, 689-706.
26. R.JennrichP.Bentler (2012). Exploratory bi-factor analysis: the oblique case. Psychometrika, 77, 442-454.
27. K.G.JöreskogD.Sörbom (1995). LISREL 8 user’s reference guide [Computer software]. Chicago, IL: Scientific Software.
28. G.G.KingsburyD.J.Weiss (1980). An alternate-forms reliability and concurrent validity comparison of Bayesian adaptive and conventional ability tests (Research Report 80-5). Minneapolis: University of Minnesota.
29. G.G.KingsburyD.J.Weiss (1983). A comparison of IRT-based adaptive mastery testing and a sequential mastery testing procedure. In D.J.Weiss (Ed.), New horizons in testing: Latent trait theory and computerized adaptive testing (pp. 257-283). New York, NY: Academic Press.
30. Y.H.LiW.D.Schafer (2005). Trait parameter recovery using multidimensional computerized adaptive testing in reading and mathematics. Applied Psychological Measurement, 29, 3-25.
31. F.M.Lord (1980). Application of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
32. R.M.Luecht (1996). Multidimensional computerized adaptive testing in a certification or licensure context. Applied Psychological Measurement, 20, 389-404.
33. K.E.MorenoD.O.Segall (1992). CAT-ASVAB precision. Proceedings of the 34th Annual Conference of the Military Testing Association, 1, 22-26.
34. J.MulderW.J.van der Linden (2009). Multidimensional adaptive testing with optimal design criteria for item selection. Psychometrika, 74, 273-296.
35. L.K.MuthénB.O.Muthén (2004). Mplus user’s guide (Version 3) [Computer software]. Los Angeles, CA: Muthén & Muthén.
36. R.J.Owen (1975). A Bayesian sequential procedure for quantal response in the context of adaptive mental testing. Journal of the American Statistical Association, 70, 351-356.
37. R Development Core Team. (2012). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for statistical Computing.
38. M.D.Reckase (1974). An interactive computer program for tailored testing based on the one-parameter logistic model. Behavior Research Methods and Instrumentation, 6, 208-212.
39. M.D.Reckase (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9, 401-412.
40. S.P.ReiseT.M.MooreM.G.Haviland (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92, 544-559.
41. S.P.ReiseJ.MorizotR.D.Hays (2007). The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Quality of Life Research, 16, 19-31.
42. W.Revelle (2015). Psych: Procedures for personality and psychological research. Evanston, IL: Northwestern University. Retrieved from http://cran.r-project.org/package=psych
43. D.O.Segall (1996). Multidimensional adaptive testing. Psychometrika, 61, 331-354.
44. S.D.Silvey (1980). Optimal design. London, England: Chapman & Hall.
45. A.H.StroudD.Sechrest (1966). Gaussian quadrature formulas. Englewood Cliffs, NJ: Prentice Hall.
46. B.D.StuckyD.ThissenM.O.Edelen (2013). Using logistic approximations of marginal trace lines to develop short assessments. Applied Psychological Measurement, 37, 41-57.
47. S.S.Tam (1992). A comparison of methods for adaptive estimation of a multidimensional trait (Unpublished doctoral dissertation). Columbia University, New York, NY.
48. W.J.van der Linden (1998). Bayesian item selection criteria for adaptive testing. Psychometrika, 63, 201-216.
49. B.P.VeldkampW.J.van der Linden (2002). Multidimensional adaptive testing with constraints on test content. Psychometrika, 67, 575-588.
50. W.C.WangP.H.Chen (2004). Implementation and measurement efficiency of multidimensional computerized adaptive testing. Applied Psychological Measurement, 28, 295-316.
51. D.J.Weiss (1982). Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement, 6, 473-492.
52. D.J.Weiss (1985). Adaptive testing by computer. Journal of Consulting and Clinical Psychology, 53, 774-789.
53. D.J.WeissR.D.Gibbons (2007, June). Computerized adaptive testing with the bifactor model. Paper presented at the New CAT Models session at the 2007 GMAC Conference on Computerized Adaptive Testing. Retrieved from http://publicdocs.iacat.org/cat2010/cat07weiss&gibbons.pdf
54. H.WolffK.Preising (2005). Exploring item and higher order factor structure with the Schmid-Leiman solution: Syntax codes for SPSS and SAS. Behavioral Research Methods, 37, 48-58.
55. R.WoodD.WilsonR.GibbonsS.SchillingE.MurakiR.D.Bock (2003). TESTFACT 4 [Computer software]. Lincolnwood, IL: Scientific Software.
56. L.Yao (2013). Comparing the performance of five multidimensional CAT selection procedure with different stopping rules. Applied Psychological Measurement, 37, 3-23.