Evaluating bifactor models: Calculating and interpreting statistical indices

Psychological Methods

Volumn 21, Issue 2, 2016, Pages 137-150

  Rodriguez, Anthony ^a   Reise, Steven P ^a   Haviland, Mark G ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b LOMA LINDA UNIVERSITY   (United States)

Author keywords

Bifactor; Explained common variance; Factor determinacy; Measurement; Omega; Reliability

Indexed keywords

FACTOR ANALYSIS; HUMAN; PROCEDURES; PSYCHOLOGICAL MODEL; PSYCHOMETRY; REPRODUCIBILITY; STANDARDS; STATISTICAL MODEL;

FACTOR ANALYSIS, STATISTICAL; HUMANS; MODELS, PSYCHOLOGICAL; MODELS, STATISTICAL; PSYCHOMETRICS; REPRODUCIBILITY OF RESULTS;

EID: 84946010595     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/met0000045     Document Type: Article

References (73)

1. Ackerman, R. A., Donnellan, M. B., & Robins, R. W. (2012). An item response theory analysis of the Narcissistic Personality Inventory. Journal of Personality Assessment, 94, 141-155. http://dx. doi. org/10. 1080/00223891. 2011. 645934
2. Bandalos, D. L. (2002). The effects of item parceling on goodness-of-fit and parameter estimate bias in structural equation modeling. Structural Equation Modeling, 9, 78-102. http://dx. doi. org/10. 1207/S15328007SEM0901_5
3. Bandalos, D. L., & Finney, S. J. (2001). Item parceling issues in structural equation modeling. In G. A. Marcoulides & R. E. Schumacker (Eds.), New developments and techniques in structural equation modeling (pp. 269-296). Mahwah, NJ: Erlbaum
4. Beauducel, A. (2011). Indeterminacy of factor score estimates in slightly misspecified confirmatory models. Journal of Modern Applied Statistical Methods, 10, 583-598
5. Bentler, P. M. (2009). Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika, 74, 137-143. http://dx. doi. org/10. 1007/s11336-008-9100-1
6. Bonifay, W. E., Reise, S. P., Scheines, R., & Meijer, R. R. (2015). When are multidimensional data unidimensional enough for structural equation modeling?: An evaluation of the DETECT multidimensionality index. Structural Equation Modeling. Advance online publication. http://dx. doi. org/10. 1080/10705511. 2014. 938596
7. Canivez, G. L. (in press). Bifactor modeling in construct validation of multifactored tests: Implications for understanding multidimensional constructs and test interpretation. In K. Schweizer & C. DiStefano (Eds.), Principles and methods of test construction: Standards and recent advancements. Gottingen, Germany: Hogrefe Publishers
8. Chen, F. F., Hayes, A., Carver, C. S., Laurenceau, J. P., & Zhang, Z. (2012). Modeling general and specific variance in multifaceted constructs: A comparison of the bifactor model to other approaches. Journal of Personality, 80, 219-251. http://dx. doi. org/10. 1111/j. 1467-6494. 2011. 00739. x
9. Chen, F. F., West, S. G., & Sousa, K. H. (2006). A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research, 41, 189-225. http://dx. doi. org/10. 1207/s15327906mbr4102_5
10. Coffman, D. L., & MacCallum, R. C. (2005). Using parcels to convert path analysis models into latent variable models. Multivariate Behavioral Research, 40, 235-259. http://dx. doi. org/10. 1207/s15327906mbr4002_4
11. Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98-104. http://dx. doi. org/10. 1037/0021-9010. 78. 1. 98
12. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334. http://dx. doi. org/10. 1007/BF02310555
13. Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105, 399-412. http://dx. doi. org/10. 1111/bjop. 12046
14. Gignac, G. E., Palmer, B. R., & Stough, C. (2007). A confirmatory factor analytic investigation of the TAS-20: Corroboration of a five-factor model and suggestions for improvement. Journal of Personality Assessment, 89, 247-257. http://dx. doi. org/10. 1080/00223890701629730
15. Gignac, G. E., & Watkins, M. W. (2013). Bifactor modeling and the estimation of model-based reliability in the WAIS-IV. Multivariate Behavioral Research, 48, 639-662. http://dx. doi. org/10. 1080/00273171. 2013. 804398
16. Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Erlbaum
17. Graham, J. M. (2006). Congeneric and (essentially) tau-equivalent estimates of score reliability: What they are and how to use them. Educational and Psychological Measurement, 66, 930-944. http://dx. doi. org/10. 1177/0013164406288165
18. Grice, J. W. (2001). Computing and evaluating factor scores. Psychological Methods, 6, 430-450. http://dx. doi. org/10. 1037/1082-989X. 6. 4. 430
19. Gustafsson, J. E., & Aberg-Bengtsson, L. (2010). Unidimensionality and the interpretability of psychological instruments. In S. E. Embretson (Ed.), Measuring psychological constructs (pp. 97-121). Washington, DC: American Psychological Association. http://dx. doi. org/10. 1037/12074-005
20. Guttman, L. (1955). The determinacy of factor score matrices with implications for five other basic problems of common factor theory. British Journal of Statistical Psychology, 8, 65-81. http://dx. doi. org/10. 1111/j. 2044-8317. 1955. tb00321. x
21. Hancock, G. R. (2001). Effect size, power, and sample size determination for structured means modeling and MIMIC approaches to betweengroups hypothesis testing of means on a single latent construct. Psychometrika, 66, 373-388. http://dx. doi. org/10. 1007/BF02294440
22. Hancock, G. R., & Mueller, R. O. (2001). Rethinking construct reliability within latent variable systems. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.), Structural equation modeling: Present and future-A Festschrift in honor of Karl Jöreskog (pp. 195-216). Lincolnwood, IL: Scientific Software International
23. Holzinger, K. J., & Harman, H. H. (1938). Comparison of two factorial analyses. Psychometrika, 3, 45-60. http://dx. doi. org/10. 1007/BF02287919
24. Holzinger, K. J., & Swineford, F. (1937). The bi-factor method. Psychometrika, 2, 41-54. http://dx. doi. org/10. 1007/BF02287965
25. Immekus, J. C., & Imbrie, P. K. (2008). Dimensionality assessment using full-information item bifactor analysis for graded response data: An illustration with the State Metacognition Inventory. Educational and Psychological Measurement, 68, 695-709. http://dx. doi. org/10. 1177/0013164407313366
26. Ivarsson, T. (2006). Normative data for the Multidimensional Anxiety Scale for Children (MASC) in Swedish adolescents. Nordic Journal of Psychiatry, 60, 107-113. http://dx. doi. org/10. 1080/08039480600588067
27. Jensen, A. R. (1998). The g factor: The science of mental ability. Westport, CT: Praeger/Greenwood
28. John, O. P., & Soto, C. J. (2007). The importance of being valid. In R. W. Robins, R. C. Fraley, & R. F. Krueger (Eds.), Handbook of research methods in personality psychology (pp. 461-494). New York, NY: Guilford Press
29. Jöreskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36, 109-133. http://dx. doi. org/10. 1007/BF02291393
30. Kievit, R. A., Romeijn, J. W., Waldorp, L. J., Wicherts, J. M., Scholte, H. S., & Borsboom, D. (2011). Mind the gap: A psychometric approach to the reduction problem. Psychological Inquiry, 22, 67-87. http://dx. doi. org/10. 1080/1047840X. 2011. 550181
31. Little, T. D., Cunningham, W. A., Shahar, G., & Widaman, K. F. (2002). To parcel or not to parcel: Exploring the question, weighing the merits. Structural Equation Modeling, 9, 151-173. http://dx. doi. org/10. 1207/S15328007SEM0902_1
32. Little, T. D., Rhemtulla, M., Gibson, K., & Schoemann, A. M. (2013). Why the items versus parcels controversy needn't be one. Psychological Methods, 18, 285-300. http://dx. doi. org/10. 1037/a0033266
33. March, J. (1998). Manual for Multidimensional Anxiety Scale for Children (MASC). Toronto, ON: Multi-Health Systems Inc
34. McCrae, R. R., & Costa, P. T. (2010). NEO Inventories for the NEO Personality Inventory-3 (NEO PI-3), NEO Five-Factor Inventory-3
35. (NEO-FFI-3) and NEO Personality Inventory-revised (NEO PI-R): Professional manual. Lutz, FL: Psychological Assessment Resources
36. McDonald, R. P. (1999). Test theory: A unified approach. Mahwah, NJ: Erlbaum
37. Miller, M. B. (1995). Coefficient alpha: A basic introduction from the perspectives of classical test theory and structural equation modeling. Structural Equation Modeling, 2, 255-273. http://dx. doi. org/10. 1080/10705519509540013
38. Mulaik, S. A. (1976). Comments on "the measurement of factorial indeterminacy". Psychometrika, 41, 249-262. http://dx. doi. org/10. 1007/BF02291842
39. Muthén, B. O., Kaplan, D., & Hollis, M. (1987). On structural equation modeling with data that are not missing completely at random. Psychometrika, 52, 431-462. http://dx. doi. org/10. 1007/BF02294365
40. ólason, D. T., Sighvatsson, M. B., & Smári, J. (2004). Psychometric properties of the Multidimensional Anxiety Scale for Children (MASC) among Icelandic schoolchildren. Scandinavian Journal of Psychology, 45, 429-436. http://dx. doi. org/10. 1111/j. 1467-9450. 2004. 00424. x
41. Osman, A., Williams, J. E., Espenschade, K., Gutierrez, P. M., Bailey, J. R., & Chowdhry, O. (2009). Further evidence of the reliability and validity of the Multidimensional Anxiety Scale for Children (MASC) in psychiatric inpatient samples. Journal of Psychopathology and Behavioral Assessment, 31, 202-214. http://dx. doi. org/10. 1007/s10862-008-9095-z
42. Raykov, T. (1997a). Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21, 173-184. http://dx. doi. org/10. 1177/01466216970212006
43. Raykov, T. (1997b). Scale reliability, Cronbach's coefficient alpha, and violations of essential tau-equivalence with fixed congeneric components. Multivariate Behavioral Research, 32, 329-353. http://dx. doi. org/10. 1207/s15327906mbr3204_2
44. Raykov, T., & Pohl, S. (2013). Essential unidimensionality examination for multicomponent scales: An interrelationship decomposition approach. Educational and Psychological Measurement, 73, 581-600. http://dx. doi. org/10. 1177/0013164412470451
45. R Development Core Team. (2014). R: A language and environment for statistical computing [Software]. Vienna, Austria: R Foundation for Statistical Computing, Retrieved from http://www. R-project. org
46. Reichenheim, M. E., Moraes, C. L., Oliveira, A. S. D., & Lobato, G. (2011). Revisiting the dimension structure of the Edinburgh Postnatal Depression Scale (EPDS): Empirical evidence for a general factor. BMC Medical Research Methodology, 11, 93. http://dx. doi. org/10. 1186/1471-2288-11-93
47. Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47, 667-696. http://dx. doi. org/10. 1080/00273171. 2012. 715555
48. Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95, 129-140. http://dx. doi. org/10. 1080/00223891. 2012. 725437
49. Reise, S. P., & Haviland, M. G. (2005). Item response theory and the measurement of clinical change. Journal of Personality Assessment, 84, 228-238. http://dx. doi. org/10. 1207/s15327752jpa8403_02
50. Reise, S. P., Moore, T. M., & Haviland, M. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92, 544-559. http://dx. doi. org/10. 1080/00223891. 2010. 496477
51. Reise, S. P., Moore, T. M., & Haviland, M. G. (2013). Applying unidimensional item response theory models to psychological data. In K. Geisinger (Ed.), APA handbook of testing and assessment in psychology: Vol. 1. Test theory and testing and assessment in industrial and organizational psychology (pp. 101-119). Washington, DC: American Psychological Association. http://dx. doi. org/10. 1037/14047-006
52. Reise, S. P., Morizot, J., & Hays, R. D. (2007). The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Quality of Life Research, 16, 19-31. http://dx. doi. org/10. 1007/s11136-007-9183-7
53. Reise, S. P., Scheines, R., Widaman, K. F., & Haviland, M. G. (2013). Multidimensionality and structural coefficient bias in structural equation modeling: A bifactor perspective. Educational and Psychological Measurement, 73, 5-26. http://dx. doi. org/10. 1177/0013164412449831
54. Revelle, W. (2014). psych: Procedures for personality and psychological, psychometric, and psychological research [Software]. Evanston, IL: Northwestern University. Retrieved from http://CRAN. R-project. org/package psychVersion 1. 4. 5
55. Revelle, W., & Zinbarg, R. E. (2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika, 74, 145-154. http://dx. doi. org/10. 1007/s11336-008-9102-z
56. Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8, 350-353. http://dx. doi. org/10. 1037/1040-3590. 8. 4. 350
57. Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach's alpha. Psychometrika, 74, 107-120. http://dx. doi. org/10. 1007/s11336-008-9101-0
58. Simms, L. J., Grös, D. F., Watson, D., & O'Hara, M. W. (2008). Parsing the general and specific components of depression and anxiety with bifactor modeling. Depression and Anxiety, 25, E34-E46. http://dx. doi. org/10. 1002/da. 20432
59. Sinharay, S., Haberman, S., & Puhan, G. (2007). Subscores based on classical test theory: To report or not to report. Educational Measurement: Issues and Practice, 26, 21-28. http://dx. doi. org/10. 1111/j. 1745-3992. 2007. 00105. x
60. Sinharay, S., Puhan, G., & Haberman, S. J. (2011). An NCME instructional module on subscores. Educational Measurement: Issues and Practice, 30, 29-40. http://dx. doi. org/10. 1111/j. 1745-3992. 2011. 00208. x
61. Steiger, J. H., & Schönemann, P. H. (1978). A history of factor indeterminacy. In S. Shye (Ed.), Theory construction and data analysis in the behavioral sciences (pp. 136-178). San Francisco, CA: Jossey-Bass
62. Sterba, S. K. (2011). Implications of parcel-allocation variability for comparing fit of item-solutions and parcel-solutions. Structural Equation Modeling, 18, 554-577. http://dx. doi. org/10. 1080/10705511. 2011. 607073
63. Sterba, S. K., & MacCallum, R. C. (2010). Variability in parameter estimates and model fit across repeated allocations of items to parcels. Multivariate Behavioral Research, 45, 322-358. http://dx. doi. org/10. 1080/00273171003680302
64. Stucky, B. D., & Edelen, M. O. (2014). Using hierarchical IRT models to create unidimensional measures from multidimensional data. In S. P. Reise & D. A. Revicki (Eds.), Handbook of item response theory modeling: Applications to typical performance assessment (pp. 183-206). New York, NY: Routledge/Taylor & Francis Group
65. Stucky, B. D., Thissen, D., & Edelen, M. O. (2013). Using logistic approximations of marginal trace lines to develop short assessments. Applied Psychological Measurement, 37, 41-57. http://dx. doi. org/10. 1177/0146621612462759
66. Ten Berge, J. M., & Sočan, G. (2004). The greatest lower bound to the reliability of a test and the hypothesis of unidimensionality. Psychometrika, 69, 613-625. http://dx. doi. org/10. 1007/BF02289858
67. Wood, R., Kandola, P., Wilson, D., Gibbons, R., Schilling, S., Muraki, E., & Bock, D. (2003). TESTFACT for Windows 4. 0. 2 [Software]. Lincolnwood, IL: Scientific Software
68. Yang, F. M., & Jones, R. N. (2008). Measurement differences in depression: Chronic health-related and sociodemographic effects in older Americans. Psychosomatic Medicine, 70, 993-1004. http://dx. doi. org/10. 1097/PSY. 0b013e31818ce4fa
69. Yang, Y., & Green, S. B. (2011). Coefficient alpha: A reliability coefficient for the 21st century? Journal of Psychoeducational Assessment, 29, 377-392. http://dx. doi. org/10. 1177/0734282911406668
70. Zhang, J., & Stout, W. (1999). The theoretical DETECT index of dimensionality and its application to approximate simple structure. Psychometrika, 64, 213-249. http://dx. doi. org/10. 1007/BF02294536
71. Zinbarg, R. E., Barlow, D. H., & Brown, T. A. (1997). Hierarchical structure and general factor saturation of the Anxiety Sensitivity Index: Evidence and implications. Psychological Assessment, 9, 277-284. http://dx. doi. org/10. 1037/1040-3590. 9. 3. 277
72. Zinbarg, R. E., Revelle, W., Yovel, I., & Li, W. (2005). Cronbach's, Revelle's β, and McDonald's ω H: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70, 123-133. http://dx. doi. org/10. 1007/s11336-003-0974-7
73. h. Applied Psychological Measurement, 30, 121-144. http://dx. doi. org/10. 1177/0146621605278814