Comparison of methods for factor extraction for cognitive test-like data: Which overfactor, which underfactor?

Intelligence

Volumn 54, Issue , 2016, Pages 37-54

  Keith, Timothy Z ^a   Caemmerer, Jacqueline M ^a   Reynolds, Matthew R ^b  

^a UNIVERSITY OF TEXAS   (United States)

^b UNIVERSITY OF KANSAS   (United States)

Author keywords

Confirmatory factor analysis; Exploratory factor analysis; Minimum average partial; Number of factors; Parallel analysis; Test interpretation

Indexed keywords

EID: 84949309445     PISSN: 01602896     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.intell.2015.11.003     Document Type: Article

References (74)

1. Barendse M.T., Oort F.J., Timmerman M.E. Using exploratory factor analysis to determine the dimensionality of discrete responses. Structural Equation Modeling: A Multidisciplinary Journal 2014, 22:87-101. 10.1080/10705511.2014.934850.
2. Beauducel A. On the generalizability of factors: The influence of changing contexts of variables on different methods of factor extraction. Methods of Psychological Research Online 2001, 6:69-96.
3. Beauducel A. Problems with parallel analysis in data sets with oblique simple structure. Methods of Psychological Research Online 2001, 6:141-157.
4. Bodin D., Pardini D.A., Burns T.G., Stevens A.B. Higher order factor structure of the WISC-IV in a clinical neuropsychological sample. Child Neuropsychology 2009, 15(5):417-424. 10.1080/09297040802603661.
5. Canivez G.L. Construct validity of the WISC-IV with a referred sample: Direct versus indirect hierarchical structures. School Psychology Quarterly 2014, 29(1):38-51. 10.1037/spq0000032.
6. Canivez G.L., Kush J.C. WAIS-IV and WISC-IV structural validity: Alternate methods, alternate results. Commentary on Weiss et al. (2013a) and Weiss et al. (2013b). Journal of Psychoeducational Assessment 2013, 31:157-169.
7. Canivez G.L., Watkins M.W. Investigation of the factor structure of the Wechsler Adult Intelligence Scale-Fourth edition (WAIS-IV): Exploratory and higher order factor analyses. Psychological Assessment 2010, 22(4):827-836. 10.1037/a0020429.
8. Carroll J.B. Human cognitive abilities: A survey of factor-analytic studies. 1993, Cambridge University Press, New York.
9. Carroll J.B. On methodology in the study of cognitive abilities. Multivariate Behavioral Research 1995, 30:429-452.
10. Chang M. Joint confirmatory factor analysis of the Woodcock-Johnson Tests of Cognitive Abilities, Third Edition, and the Stanford Binet Intelligence Scales, Fifth Edition, with a preschool population. ProQuest Dissertations and Theses database (UMI No. 3466801) 2011, (Doctoral Dissertation).
11. Cohen J. Factors underlying Wechsler-Bellevue performance of three neuropsychiatric groups. Journal of Abnormal and Social Psychology 1952, 47:359-365.
12. Crawford A.V., Green S.B., Levy R., Lo W.J., Scott L., Svetina D., Thompson M.S. Evaluation of parallel analysis methods for determining the number of factors. Educational and Psychological Measurement 2010, 70(6):885-901. 10.1177/0013164410379332.
13. Devena S.E., Gay C.E., Watkins M.W. Confirmatory factor analysis of the WISC-IV in a hospital referral sample. Journal of Psychoeducational Assessment 2013, 31(6):591-599. 10.1177/0734282913483981.
14. DiStefano C., Hess B. Using confirmatory factor analysis for construct validation: An empirical review. Journal of Psychoeducational Assessment 2005, 23(3):225-241. 10.1177/073428290502300303.
15. Dombrowski S.C. Investigating the structure of the WJ-III cognitive at school age. School Psychology Quarterly 2013, 28:154-169. 10.1037/spq0000010.
16. Dombrowski S.C. Exploratory bifactor analysis of the WJ-III cognitive in adulthood via the Schmid-Leiman procedure. Journal of Psychoeducational Assessment 2014, 32:330-341. 10.1177/0734282913508243.
17. Dombrowski S.C., Watkins M.W. Exploratory and higher order factor analysis of the WJ-III full test battery: A school-aged analysis. Psychological Assessment 2013, 25:442-455. 10.1037/a0031335.
18. Dombrowski S.C., Watkins M.W., Brogan M.J. An exploratory investigation of the factor structure of the Reynolds Intellectual Assessment Scales (RIAS). Journal of Psychoeducational Assessment 2009, 27(6):494-507. 10.1177/0734282909333179.
19. Fava J.L., Velicer W.F. The effects of underextraction in factor and component analysis. Educational and Psychological Measurement 1996, 56:907-929.
20. Frazier T.W., Youngstrom E.A. Historical increase in the number of factors measured by commercial tests of cognitive ability: Are we overfactoring?. Intelligence 2007, 35:169-182.
21. Glorfeld L.W. An improvement on Horn's parallel analysis methodology for selecting the correct number of factors to retain. Educational and Psychological Measurement 1995, 55(3):377-393. 10.1177/0013164495055003002.
22. Green S.B., Levy R., Thompson M.S., Lu M., Lo W.J. A proposed solution to the problem with using completely random data to assess the number of factors with parallel analysis. Educational and Psychological Measurement 2011, 72(3):357-374. 10.1177/0013164411422252.
23. Hayashi K., Bentler P.M., Yuan K.-H. On the likelihood ratio test for the number of factors in exploratory factor analysis. Structural Equation Modeling 2007, 14:505-526.
24. Henson R.K. Use of exploratory factor analysis in published research: Common errors and some comment on improved practice. Educational and Psychological Measurement 2006, 66(3):393-416. 10.1177/0013164405282485.
25. Hoelzle J.B., Nelson N.W., Smith C.A. Comparison of Wechsler Memory Scale-fourth edition (WMS-IV) and third edition (WMS-III) dimensional structures: Improved ability to evaluate auditory and visual constructs. Journal of Clinical and Experimental Neuropsychology 2011, 33(3):283-291. 10.1080/13803395.2010.511603.
26. Holdnack J.A., Xiaobin Z., Larrabee G.J., Millis S.R., Salthouse T.A. Confirmatory factor analysis of the WAIS-IV/WMS-IV. Assessment 2011, 18(2):178-191. 10.1177/1073191110393106.
27. Horn J.L. A rationale and test for the number of factors in factor analysis. Psychometrika 1965, 30:179-185.
28. Horn J.L., Cattell R.B. Refinement and test of the theory of fluid and crystallized intelligence. Journal of Educational Psychology 1966, 57:253-270.
29. Hu L., Bentler P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling 1999, 6:1-55.
30. Jia R., Jia H.H. Factorial validity of problematic internet use scales. Computers in Human Behavior 2009, 25(6):1335-1342. 10.1016/j.chb.2009.06.004.
31. Johnson W., Bouchard T.J. Constructive replication of the visual-perceptual-image rotation model in Thurstone's (1941) battery of 60 tests of mental ability. Intelligence 2005, 33:417-430.
32. Johnson W., Bouchard T.J. The structure of human intelligence: It is verbal, perceptual, and image rotation (VPR), not fluid and crystallized. Intelligence 2005, 33:393-416.
33. Jöreskog K.G. Factor analysis and its extensions. Factor analysis at 100: Historical developments and future directions 2007, 47-77. Erlbaum, Muhwah, NJ. R. Cudeck, R.C. MacCallum (Eds.).
34. Keith T.Z. Multiple regression and beyond: An introduction to multiple regression and structural equation modeling 2015, Routledge, New York.
35. Keith T.Z., Reynolds M.R. CHC and cognitive abilities: What we've learned from 20years of research. Psychology in the Schools 2010, 47:635-650. 10.1002/pits.20496.
36. Keith T.Z., Witta E.L. Hierarchical and cross-age confirmatory factor analysis of the WISC-III: What does it measure?. School Psychology Quarterly 1997, 12:89-107.
37. Keith T.Z., Fine J.G., Taub G.E., Reynolds M.R., Kranzler J.H. Higher-order, multi-sample, confirmatory factor analysis of the Wechsler Intelligence Scale for Children-Fourth edition: What does it measure?. School Psychology Review 2006, 35:108-127.
38. Keith T.Z., Low J.A., Reynolds M.R., Patel P.G., Ridley K.P. Higher-order factor structure of the Differential Ability Scales-II: Consistency across ages 4 to 17. Psychology in the Schools 2010, 47:676-697. 10.1002/pits.20498.
39. Keith T.Z., Reynolds M.R., Patel P.G., Ridley K.P. Sex differences in latent cognitive abilities ages 6 to 59: Evidence from the Woodcock-Johnson III tests of cognitive abilities. Intelligence 2008, 36(6):502-525. 10.1016/j.intell.2007.11.001.
40. Loehlin J.C. Component analysis versus common factor analysis: A case of disputed authorship. Multivariate Behavioral Research 1990, 25:29-31.
41. Loehlin J.C. Latent variable models: An introduction to factor, path, and structural analysis 2004, Erlbaum, Mahwah, NJ. 4th ed.
42. Lorenzo-Seva U., Timmerman M.E., Kiers H.A.L. The Hull method for selecting the number of common factors. Multivariate Behavioral Research 2011, 46:340-364. 10.1080/00273171.2011.564527.
43. MacCallum R.C., Widaman K.F., Preacher K.J., Hong S. Sample size in factor analysis: The role of model error. Multivariate Behavioral Research 2001, 36:611-637.
44. Major J.T., Johnson W., Bouchard T.J. The dependability of the general factor of intelligence: Why small, single-factor models do not adequately represent g. Intelligence 2011, 39(5):418-433. 10.1016/j.intell.2011.07.002.
45. Marsh H.W., Hau K.-T., Balla J.R., Grayson D. Is more ever too much? The number of indicators per factor in confirmatory factor analysis. Multivariate Behavioral Research 1998, 33(2):181-220. 10.1207/s15327906mbr3302_1.
46. Mays K.L., Kamphaus R.W., Reynolds C.R. Applications of the Kaufman Assessment Battery for Children, 2nd edition in neuropsychological assessment. Handbook of clinical child neuropsychology 2009, 281-296. Springer, New York. C.R. Reynolds, E. Fletcher-Janzen (Eds.).
47. Mulaik S.A. Foundations of factor analysis 2009, CRC Press, Boca Raton, FL. 2nd ed.
48. Nelson J.M., Canivez G.L. Examination of the structural, convergent, and incremental validity of the Reynolds Intellectual Assessment Scales (RIAS) with a clinical sample. Psychological Assessment 2012, 24:129-140.
49. Nelson J.M., Canivez G.L., Lindstrom W., Hatt C.V. Higher-order exploratory factor analysis of the Reynolds Intellectual Assessment Scales with a referred sample. Journal of School Psychology 2007, 45(4):439-456. 10.1016/j.jsp.2007.03.003.
50. O'Connor B.P. SPSS and SAS programs for determining the number of components using parallel analysis and Velicer's MAP test. Behavior Research Methods, Instruments, & Computers 2000, 32:396-402.
51. Preacher K.J., Zhang G., Kim C., Mels G. Choosing the optimal number of factors in exploratory factor analysis: A model selection perspective. Multivariate Behavioral Research 2013, 48(1):28-56. 10.1080/00273171.2012.710386.
52. Reynolds M.R., Keith T.Z., Flanagan D.P., Alfonso V.C. A cross-battery, reference variable, confirmatory factor analytic investigation of the CHC taxonomy. Journal of School Psychology 2013, 51:535-555.
53. Ruscio J., Roche B. Determining the number of factors to retain in an exploratory factor analysis using comparison data of known factorial structure. Psychological Assessment 2012, 24(2):282-292. 10.1037/a0025697.
54. Sanders S., McIntosh D.E., Dunham M., Rothlisberg B.A., Finch H. Joint confirmatory factor analysis of the Differential Ability Scales and the Woodcock-Johnson Tests of Cognitive Abilities-Third edition. Psychology in the Schools 2007, 44:119-138.
55. Schneider W.J., McGrew K.S. The Cattell-Horn-Carrol model of intelligence. Contemporary intellectual assessment: Theories, tests, and issues 2012, 99-144. Guilford, New York. 3rd ed. D.P. Flanagan, P.L. Harrison (Eds.).
56. Sclove S.L. Applications of some model-selection criteria to some problems in multivariate analysis. Psychometrika 1987, 52:333-343.
57. Strickland T. Structural validity of the Woodcock Johnson III Cognitive in a referred sample. ProQuest Dissertations and Theses database (UMI No. 3505630) 2012, (Doctoral dissertation).
58. Taub G.E., McGrew K.S. A confirmatory factor analysis of Cattell-Horn-Carroll theory and cross-age invariance of the Woodcock-Johnson Tests of Cognitive Abilities III. School Psychology Quarterly 2004, 19:72-87.
59. Taub G.E., McGrew K.S. The Woodcock-Johnson Tests of Cognitive Abilities III's cognitive performance model: Empirical support for intermediate factors within CHC theory. Journal of Psychoeducational Assessment 2014, 32(3):187-201. 10.1177/0734282913504808.
60. Timmerman M.E., Lorenzo-Seva U. Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods 2011, 16:209-220. 10.1037/a0023353.
61. Tucker C.M., Rice K.G., Hou W., Kaye L.B., Nolan S.E.M., Grandoit D.J., Desmond F.F. Development of the Motivators of and Barriers to Health-Smart Behaviors Inventory. Psychological Assessment 2011, 10.1037/a0022299.supp.
62. Turner N.E. The effect of common variance and structure pattern on random data eigenvalues: Implications for the accuracy of parallel analysis. Educational and Psychological Measurement 1998, 58:541-568.
63. Velicer W.F. Determining the number of components from the matrix of partial correlations. Psychometrika 1976, 41:321-327.
64. Velicer W.F., Eaton C.A., Fava J.L. Construct explication through factor or component analysis: A review and evaluation of alternative procedures for determining the number of factors or components. Problems and solutions in human assessment 2000, 41-71. Kluwer, Boston. R.D. Goffin, E. Helmes (Eds.).
65. Watkins M.W., Beaujean A.A. Bifactor structure of the Wechsler Preschool and Primary Scale of Intelligence-Fourth edition. School Psychology Quarterly 2014, 10.1037/spq0000038.supp.
66. Watkins M.W., Canivez G.L., James T., James K., Good R. Construct validity of the WISC-IV UK with a large referred Irish sample. International Journal of School & Educational Psychology 2013, 1(2):102-111. 10.1080/21683603.2013.794439.
67. Wechsler D. Wechsler Intelligence Scale for Children-Fourth edition 2003, Psychological Corporation, San Antonio, TX.
68. Wechsler D. Wechsler Intelligence Scale for Children-Fifth edition 2014, Pearson, San Antonio, TX.
69. Weiss L.G., Keith T.Z., Zhu J., Chen H. WISC-IV and clinical validation of the four- and five-factor interpretive approaches. Journal of Psychoeducational Assessment 2013, 31:114-131.
70. Weiss L.G., Keith T.Z., Zhu J., Chen H. WAIS-IV and clinical validation of the four- and five-factor interpretive approaches. Journal of Psychoeducational Assessment 2013, 31:94-113.
71. Widaman K.F. Common factors versus components: Principals and principles, errors, and misconceptions. Factor analysis at 100: Historical developments and future directions 2007, 177-203. Erlbaum, Mahwah, NJ. R. Cudeck, R.C. MacCallum (Eds.).
72. Wood J.M., Tataryn D.J., Gorsuch R.L. Effects of under- and overextraction on principal axis factor analysis with varimax rotation. Psychological Methods 1996, 1(4):354-365. 10.1037/1082-989X.1.4.354.
73. Zwick W.R., Velicer W.F. Variables influencing four rules for determining the number of components to retain. Psychological Bulletin 1982, 1982:253-269.
74. Zwick W.R., Velicer W.F. Comparison of five rules for determining the number of components to retain. Psychological Bulletin 1986, 99:432-442.