Dimensionality Assessment using the full-information item bifactor analysis for graded response data: An illustration with the state metacognitive inventory

Educational and Psychological Measurement

Volumn 68, Issue 4, 2008, Pages 695-709

  Immekus, Jason C ^a   Imbrie, P K ^b  

^a CALIFORNIA STATE UNIVERSITY   (United States)

^b PURDUE UNIVERSITY   (United States)

Author keywords

Dimensionality assessment; Full information item bifactor analysis; Graded response data; Item response theory; Test score validity

Indexed keywords

EID: 47749095937     PISSN: 00131644     EISSN: 15523888     Source Type: Journal    
DOI: 10.1177/0013164407313366     Document Type: Article

References (56)

1. Ackerman, T.A. (1987, April). The robustness of LOGIST and BILOG IRT estimation programs to violations of local independence. Paper presented at the annual meeting of the American Educational Research Association, Washington, DC. (ERIC Documentation Reproduction Service No. ED284902)
2. Ackerman, T.A. (2005). Multidimensional item response theory modeling. In A. Maydeu-Olivares & J. J. McArdle (Eds.), Contemporary psychometrics (pp. 3-26). Mahwah, NJ: Lawrence Erlbaum.
3. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
4. Ansley, T.N., & Forsyth, R.A. (1985). An examination of the characteristics of unidimensional IRT parameter estimates derived from two-dimensional data. Applied Psychological Measurement, 9, 37-48.
5. Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W. H. Freeman.
6. Bartholomew, D.J. (1980). Factor analysis for categorical data. Journal of the Royal Statistical Society, Series B, 42, 293-321.
7. Beck, C.T., & Gable, R.K. (2001). Item response theory in affective instrument development: An illustration. Journal of Nursing Measurement, 9, 5-22.
8. Bock, R.D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443-459.
9. Bock, R.D., & Gibbons, R.D. (in press). Factor analysis of categorical item responses. In R. Ostini & M. Nering (Eds.), Handbook of polytomous item response theory models: Development and applications. Thousand Oaks, CA: Sage.
10. Bock, R.D., Gibbons, R.D., & Muraki, E. (1988). Full-information item factor analysis. Applied Psychological Measurement, 12, 261-280.
11. Bock, R.D., Gibbons, R.D., Schilling, S.G., Muraki, E., Wilson, D.T., & Wood, R. (2002). TEST-FACT 4 [Computer software]. Chicago: Scientific Software International.
12. Bock, R.D., & Mislevy, R.J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444.
13. Bollen, K.A. (1989). Structural equations with latent variables. New York: John Wiley.
14. Borsboom, D. (2006). The attack of the psychometricians. Psychometrika, 71, 425-440.
15. Byrne, B.M., & Shavelson, R.J. (1987). Adolescent self-concept: Testing the assumption of equivalent structure across gender. American Educational Research Journal, 24, 365-385.
16. Chen, F.F., West, S.G., & Sousa, K.H. (2006). A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research, 41, 189-225.
17. Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Fort Worth, TX: Harcourt Brace.
18. Demars, C.E. (2006). Application of the bifactor multidimensional item response theory model to testlet-based tests. Journal of Educational Measurement, 43, 145-168.
19. Embretson, S.E. (2004). The second century of ability testing: Some predictions and speculations. Measurement, 2, 1-32.
20. Floyd, F.J., & Widaman, K.F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7, 286-299.
21. Gibbons, R.D., Bock, R.D., Hedeker, D., Weiss, D.J., Segawa, E., Bhaumik, D., et al. (2007). Full-information item bi-factor analysis of graded response data. Applied Psychological Measurement, 31, 4-19.
22. Gibbons, R.D., & Hedeker, D. (1992). Full-information item bifactor analysis. Psychometrika, 57, 423-426.
23. Gibbons, R.D., & Hedeker, D. (n.d.). POLYBIF [Computer software]. Chicago: Center for Health Statistics, University of Illinois. Retrieved August 22, 2007, from http://www.uic.edu/labs/biostat/projects.html
24. Gibbons, R.D., Immekus, J.C., & Bock, R.D. (2007). Didactic workbook: The added value of multidimensional IRT models (Final Report for National Cancer Institute Contract No. 2005-05828-00-00). Retrieved January 3, 2008, from http://www.uic.edu/labs/biostat/papers.html
25. Hambleton, R.K., & Swaminathan, H. (1985). Item response theory: Principle and applications. Boston: Kluwer.
26. Henson, R.K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurement and Evaluation in Counseling and Development, 34, 177-189.
27. Holzinger, K.J., & Swineford, R. (1937). The bifactor method. Psychometrika, 2, 41-54.
28. Jreskog, K.G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183-202.
29. Jreskog, K.G. (1990). New developments in LISREL: Analysis of ordinal variables using polychoric correlations and weighted least squares. Quality and Quantity, 24, 387-404.
30. Leonelli, B.T., Chang, C., Bock, R.D., & Schilling, S.G. (2000). Interpretation of a full-information item-level factor analysis of the MMPI-2: Normative sampling and nonpathognomonic descriptors. Journal of Personality Assessment, 74, 400-422.
31. Li, Y., Bolt, D.M., & Fu, J. (2006). A comparison of alternative methods for testlets. Applied Psychological Measurement, 30, 3-20.
32. Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
33. Lord, F.M., & Novick, M. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
34. Marsh, H.W., & Shavelson, R.J. (1985). Self-concept: Its multifaceted, hierarchical structure. Educational Psychologist, 20, 107-125.
35. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749.
36. Muraki, E., & Carlson, J.E. (1995). Full-information factor analysis for polytomous item responses. Applied Psychological Measurement, 19, 73-90.
37. Múthen, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49, 115-132.
38. Nandakumar, R., & Stout, W. (1993). Refinements of Stout's procedure for assessing latent trait unidimensionality. Journal of Educational Statistics, 18, 41-68.
39. O'Neil, H.F., & Abedi, J. (1996). Reliability and validity of a state metacognitive inventory: Potential for alternative assessment. Journal of Educational Research, 89, 234-245.
40. Pintrich, P.R., & DeGroot, E.V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82, 33-40.
41. Pintrich, P.R., & Schunk, D.H. (1999). Motivation in education: Theory, research, and applications. Englewood Cliffs, NJ: Prentice Hall.
42. Pressley, M., & Afflerbach, P. (1995). What readers can do when they read: A summary of the results from the on-line self-report studies of reading. In M. Pressley & P. Afflerbach, Verbal protocols of reading: The nature of constructively responsive reading (pp. 31-82). Hillsdale, NJ: Lawrence Erlbaum.
43. Reckase, M.D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9, 401-412.
44. Reise, S.P., Morizot, J., & Hays, R.D. (2007). The role of the bi-factor model in resolving dimensionality issues in health outcomes measures. Medical Care, 16, 19-31.
45. Samejima, F. (1969). Estimation of ability using a response pattern of graded scores. Psychometrika Monograph, 17.
46. Samejima, F. (1997). Graded response model. In W. van der Linden & R. Hambleton (Eds.), Handbook of modern item response theory (pp. 85-100). New York: Springer.
47. Schmeiser, C.B., & Welch, C.J. (2006). Test development. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 307-353). Westport, CT: American Council on Education and Praeger.
48. Schunk, D.H., & Ertmer, P.A. (2000). Self-regulation and academic learning: Self-efficacy enhancing interventions. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 631-649). San Diego, CA: Academic Press.
49. Sireci, S.D., Thissen, D., & Wainer, H. (1991). On the reliability of testlet-based tests. Journal of Educational Measurement, 28, 237-247.
50. Stout, W. (1987). A nonparametric approach for assessing latent trait dimensionality. Psychometrika, 57, 423-436.
51. Wainer, H., & Wang, C. (2000). Using a new statistical model for testlets to score TOEFL. Journal of Educational Measurement, 37, 203-220.
52. Way, W.D., & Forsyth, R.A. (1988). The comparative effects of compensatory and noncompensatory two-dimensional data on unidimensional estimates. Applied Psychological Measurement, 12, 239-252.
53. Woods, C.M. (2002). Factor analysis of scales composed of binary items: Illustration with the Maudsley Obsessional Compulsive Inventory. Journal of Psychopathology and Behavioral Assessment, 24, 215-223.
54. Yen, W.M., & Fitzpatrick, A.R. (2006). Item response theory. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 111-153). Westport, CT: American Council on Education and Praeger.
55. Zhang, J., & Stout, W. (1999). The theoretical DETECT index of dimensionality and its application to approximate simple structure. Psychometrika, 64, 213-249.
56. Zimmerman, B.J. (1989). A social cognitive view of self-regulated academic learning. Journal of Educational Psychology, 81, 329-339.