Examining the integrity of measurement of cognitive abilities in the prediction of achievement: Comparisons and contrasts across variables from higher-order and bifactor models

Journal of School Psychology

Volumn 58, Issue , 2016, Pages 1-19

  Benson, Nicholas F ^a   Kranzler, John H ^b   Floyd, Randy G ^c  

^a BAYLOR UNIVERSITY   (United States)

^b UNIVERSITY OF FLORIDA   (United States)

^c UNIVERSITY OF MEMPHIS   (United States)

Author keywords

Achievement; Broad cognitive abilities; Factor scores; General intelligence

Indexed keywords

ACADEMIC ACHIEVEMENT; INTELLIGENCE; PREDICTION; STATISTICAL MODEL; THEORETICAL MODEL; WRITING; ACHIEVEMENT; ADOLESCENT; APTITUDE; CHILD; COGNITION; COMPARATIVE STUDY; FEMALE; HUMAN; INTELLIGENCE TEST; MALE; PHYSIOLOGY; PROCEDURES; PSYCHOMETRY; STATISTICS AND NUMERICAL DATA;

ACHIEVEMENT; ADOLESCENT; APTITUDE; CHILD; COGNITION; FEMALE; HUMANS; INTELLIGENCE; INTELLIGENCE TESTS; MALE; MODELS, STATISTICAL; PSYCHOMETRICS;

EID: 84978264436     PISSN: 00224405     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jsp.2016.06.001     Document Type: Article

References (82)

1. Algina, J., Moulder, B.C., Sample sizes for confidence intervals on the increase in the squared multiple correlation coefficient. Educational and Psychological Measurement 61 (2001), 633–649.
2. Algina, J., Keselman, H., Penfield, R., Confidence intervals for an effect size measure in multiple linear regression. Educational and Psychological Measurement 67 (2007), 207–218.
3. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, Standards for educational and psychological testing. 2014, American Educational Research Association, Washington, DC.
4. Anderson, R.D., Rubin, H., Statistical inference in factor analysis. 5, 1956, Proceedings of the Third Berkeley Symposium of Mathematical Statistics and Probability, 111–150.
5. Beaujean, A.A., Parkin, J., Parker, S., Comparing Cattell–Horn–Carroll models for predicting language achievement: Differences between bi-factor and higher-order factor models. Psychological Assessment 26 (2014), 789–805.
6. Benson, J., Developing a strong program of construct validation: A test anxiety example. Educational Measurement: Issues and Practices 17 (1998), 10–17.
7. Benson, N., Cattell–Horn–Carroll cognitive abilities and reading achievement. Journal of Psychoeducational Assessment 26 (2008), 27–41.
8. Brody, N., Intelligence. 2nd ed., 1992, Academic Press, San Diego, CA.
9. Canivez, G.L., Psychometric versus actuarial interpretation of intelligence and related aptitude batteries. Saklofske, D.H., Reynolds, C.R., Schwean, V.L., (eds.) The Oxford handbook of child psychological assessments, 2013, Oxford University Press, New York, NY, 84–112.
10. Canivez, G.L., Incremental criterion validity of WAIS–IV factor index scores: Relationships with WIAT–II and WIAT–III subtest and composite scores. Psychological Assessment 25 (2013), 484–495.
11. Canivez, G.L., Bifactor modeling in construct validation of multifactored tests: Implications for multidimensionality and test interpretation. Schweizer, K., DiStefano, C., (eds.) Principles and methods of test construction: Standards and recent advancements, 2016, Hogrefe Publishing, Gottingen, Germany, 247–271.
12. Carroll, J.B., Human cognitive abilities: A survey of factor-analytic studies. 1993, Cambridge, New York, NY.
13. Carroll, J.B., On methodology in the study of cognitive abilities. Multivariate Behavioral Research 30 (1995), 429–452.
14. Carroll, J.B., Theoretical and technical issues in identifying a factor of general intelligence. Devlin, B., Fienberg, S.E., Resnick, D.P., Roeder, K., (eds.) Intelligence, genes, and success: Scientists respond to the bell curve, 1997, Springer-Verlag, New York, NY, 125–156.
15. Carroll, J.B., The higher-stratum structure of cognitive abilities: Current evidence supports g and about ten broad factors. Nyborg, H., (eds.) The scientific study of general intelligence: Tribute to Arthur R. Jensen, 2003, Elsevier Science, New York, NY, 5–21.
16. Ceci, S.J., On intelligence … more or less: A bio-ecological treatise on intellectual development. 1990, Prentice-Hall, Englewood Cliffs, NJ.
17. Chen, F.F., West, S.G., Sousa, K.H., A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research 41 (2006), 189–225.
18. Coan, R.W., Facts, factors, and artifacts: The quest for psychological meaning. Psychological Review 71 (1964), 123–140.
19. Cohen, J., Statistical power analysis for the behavioral sciences. 2nd ed., 1988, Erlbaum, Hillsdale, NY.
20. Cronbach, L.J., Meehl, P.E., Construct validity in psychological tests. Psychological Bulletin 52 (1955), 281–302.
21. Current Population Survey, March, 2001. 2001, Bureau of the Census, Washington, DC.
22. DiStefano, C., Zhu, M., Mindrila, D., Understanding and using factor scores: Considerations for the applied researcher. Practical Assessment, Research & Evaluation 14 (2009), 1–11.
23. Esters, I.G., Ittenbach, R.F., Han, K., Today's IQ tests: Are they really better than their historical predecessors?. School Psychology Review 26 (1997), 211–224.
24. Fan, X., Sivo, S., Sensitivity of fit indices to model misspecification and model types. Multivariate Behavioral Research 42 (2007), 509–529.
25. Farmer, R.L., Floyd, R.G., Reynolds, M.R., Kranzler, J.H., IQs are very strong but imperfect indicators of psychometric g: Results from joint confirmatory factor analysis. Psychology in the Schools 51 (2014), 801–813.
26. Flanagan, D.P., Ortiz, S.O., Alfonso, V.C., Dynda, A.M., Integration of response to intervention and norm-referenced tests in learning disability identification: Learning from the Tower of Babel. Psychology in the Schools 43 (2006), 807–825.
27. Flanagan, D.R., Fiorello, C.A., Ortiz, S.O., Enhancing practice through application of Cattell–Horn–Carroll theory and research: A ‘third method’ approach to specific learning disability identification. Psychology in the Schools 47 (2010), 739–760.
28. Flanagan, D.P., Alfonso, V.C., Mascolo, J.T., A CHC-based operational definition of SLD: Integrating multiple data sources and multiple data gathering methods. Flanagan, D.P., Alfonso, V.C., (eds.) Essentials of specific learning disability identification, 2011, Wiley, Hoboken, NJ, 233–298.
29. Flanagan, D.P., Ortiz, S.O., Alfonso, V.C., Essentials of cross-battery assessment. 3rd ed., 2013, Wiley, Hoboken, NJ.
30. Floyd, R.G., Keith, T.Z., Taub, G.E., McGrew, K.S., Cattell–Horn–Carroll cognitive abilities and their effects on reading decoding skills: g has indirect effects, more specific abilities have direct effects. School Psychology Quarterly 22 (2007), 200–233.
31. Floyd, R.G., McGrew, K.S., Barry, A., Rafael, F.A., Rogers, J., General and specific effects on Cattell–Horn–Carroll broad ability composites: Analysis of the Woodcock–Johnson III Normative Update CHC factor clusters across development. School Psychology Review 38 (2009), 249–265.
32. Floyd, R.G., Meisinger, E.B., Gregg, N., Keith, T.Z., An explanation of reading comprehension across development using models from the Cattell–Horn–Carroll theory: Support for integrative models of reading. Psychology in the Schools 48 (2012), 725–743.
33. Freberg, M.E., Vandiver, B.J., Watkins, M.W., Canivez, G.L., Significant factor score variability and the validity of the WISC-III Full Scale IQ in predicting later academic achievement. Applied Neuropsychology 15 (2008), 131–139.
34. Gignac, G.E., Higher-order models versus bifactor models: g as a superordinate or breadth factor?. Psychology Science 50 (2008), 21–43.
35. Gignac, G.E., Watkins, M.W., Bifactor modeling and the estimation of model-based reliability in the WAIS-IV. Multivariate Behavioral Research 48 (2013), 639–662.
36. Glutting, J.J., Watkins, M.W., Konold, T.R., McDermott, P.A., Distinctions without a difference: The utility of observed versus latent factors from the WISC-IV in estimating reading and math achievement on the WIAT-II. Journal of Special Education, 40, 2006, 103114.
37. Gustafsson, J.-E., Measurement from a hierarchical point of view. Braun, H.I., Jackson, D.N., Wiley, D.E., (eds.) The role of constructs in psychological and educational measurement, 2002, Erlbaum, Mahwah, NJ, 73–96.
38. Gustafsson, J.-E., Balke, G., General and specific abilities as predictors of school achievement. Multivariate Behavioral Research 28 (1993), 407–434.
39. Hajovsky, D.B., Reynolds, M.R., Floyd, R.G., Turek, J.J., Keith, T.Z., A multi-group structural equation model of CHC abilities and reading achievement across grades. School Psychology Review 43 (2014), 385–406.
40. Hershberger, S.L., Factor scores. Everitt, B.S., Howell, D.C., (eds.) Encyclopedia of statistics in behavioral science, 2005, Wiley, New York, NY, 636–644.
41. Horn, J.L., Noll, J., Human cognitive capabilities: Gf-Gc theory. Flanagan, D.P., Genshaft, J.L., Harrison, P.L., (eds.) Contemporary intellectual assessment: Theories, tests, and issues, 1997, Guilford Press, New York, NY, 53–91.
42. Hu, L., Bentler, P.M., Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling 6 (1999), 1–55.
43. Hunsley, J., Meyer, G.J., The incremental validity of psychological testing and assessment: conceptual, methodological, and statistical issues. Psychological Assessment 15 (2003), 446–455.
44. Jensen, A.R., The g factor and the design of education. Sternberg, R.J., Williams, W.M., (eds.) Intelligence, instruction, and assessment: Theory into practice, 1998, Erlbaum, Mahwah, NJ, 111–131.
45. Jöreskog, K., Sörbom, D., LISREL 8.8 for Windows [computer software]. 2013, Scientific Software International, Inc., Skokie, IL.
46. Kaufman, A.S., IQ testing 101. 2009, Springer Publishing, New York, NY.
47. Kaufman, A.S., Kaufman, N.L., Kaufman Assessment Battery For Children-Second Edition: Technical manual. 2004, American Guidance Service, Circle Pines, MN.
48. Kaufman, A.S., Kaufman, N.L., Kaufman Test of Educational Achievement-Second Edition: Technical manual. 2004, American Guidance Service, Circle Pines, MN.
49. Keith, T.Z., Multiple regression and beyond: An introduction to multiple regression and structural equation modeling. 2nd ed., 2015, Allyn & Bacon, Boston, MA.
50. Kranzler, J.H., Floyd, R.G., Assessing intelligence in children and adolescents: A practical guide. 2013, Guilford Press, New York, NY.
51. Kranzler, J.H., Benson, N., Floyd, R.G., Using estimated factor scores from a bifactor analysis to examine the unique effects of the latent variables measured by the WAIS-IV on academic achievement. Psychological Assessment 27 (2015), 1402–1416, 10.1037/pas0000119.
52. McFarland, D.J., Modeling general and specific abilities: Evaluation of bifactor models for the WJ-III. Assessment, 2015, 10.1177/1073191115595070.
53. McGill, R.J., Interpretation of KABC-II scores: An evaluation of the incremental validity of Cattell–Horn–Carroll (CHC) factor scores in predicting achievement. Psychological Assessment, 2015, 10.1037/pas0000127.
54. McGrew, K.S., The Cattell–Horn–Carroll theory of cognitive abilities: Past, present, and future. Flanagan, D.P., Harrison, P.L., (eds.) Contemporary intellectual assessment: Theories, tests, and issues, 2nd ed., 2005, Guilford Press, New York, NY, 136–181.
55. McGrew, K.S., Wendling, B.J., Cattell–Horn–Carroll cognitive-achievement relations: What we have learned from the past 20 years of research. Psychology in the Schools 47 (2010), 651–675.
56. Messick, S., Meaning and values in test validation: The science and ethics of assessment. Educational Researcher 18 (1989), 5–11.
57. Messick, S., Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist 50 (1995), 741–749.
58. Mulaik, S.A., Quartetti, D.A., First order or higher order general factor?. Structural Equation Modeling 4 (1997), 193–211.
59. Murray, A.L., Johnson, W., The limitations of model fit in comparing the bi-factor versus higher-order models of human cognitive ability structure. Intelligence 41 (2013), 407–422.
60. Naglieri, J.A., Otero, T.M., The cognitive assessment system: From theory to practice. Flanagan, D.P., Harrison, P.L., (eds.) Contemporary intellectual assessment: Theories, tests, and issues, 2012, Guilford Press, New York, NY, 376–399.
61. Nelson, J.M., Canivez, G.L., Examination of the structural, convergent, and incremental validity of the Reynolds Intellectual Assessment Scales (RIAS) with a clinical sample. Psychological Assessment 24 (2012), 129–140.
62. Oh, H.-J., Glutting, J.J., Watkins, M.W., Youngstrom, E.A., McDermott, P.A., Correct interpretation of latent versus observed abilities: Implications from structural equation modeling applied to the WISC-III and WIAT linking sample. Journal of Special Education 38 (2004), 159–173.
63. Reise, S.P., The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(777-696), 2012.
64. Reise, S.P., Bonifay, W.E., Haviland, M.G., Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment 95 (2013), 129–140.
65. Reynolds, M.R., Keith, T.Z., Spearman's law of diminishing returns in hierarchical models of intelligence for children and adolescents. Intelligence 35 (2007), 267–281.
66. Reynolds, M.R., Keith, T.Z., Measurement and statistical issues in child assessment research. Saklofske, D.H., Reynolds, C.R., Schwean, V.L., (eds.) The Oxford handbook of child psychological assessment, 2013, Oxford University Press, New York, NY, 48–83.
67. Reynolds, M.R., Keith, T.Z., Fine, J.G., Fisher, M.E., Low, J.A., Confirmatory factor structure of the Kaufman Assessment Battery for Children-Second Edition: Consistency with Cattell–Horn–Carroll theory. School Psychology Quarterly 22 (2007), 511–539.
68. Reynolds, M.R., Floyd, R.G., Niileksela, C.R., How well is psychometric g indexed by global composites? Evidence from three popular intelligence tests. Psychological Assessment 25 (2013), 1314–1321.
69. Reynolds, M.R., Keith, T.Z., Flanagan, D.P., Alfonso, V.C., A cross-battery, reference variable, confirmatory factor analytic investigation of the CHC taxonomy. Journal of School Psychology 51 (2013), 535–555.
70. Schmid, J., Leiman, J., The development of hierarchical factor solutions. Psychometrika 22 (1957), 53–61.
71. Schmiedek, F., Li, S., Toward an alternative representation for disentangling age-associated differences in general and specific cognitive abilities. Psychology and Aging 19 (2004), 40–56.
72. Schneider, W.J., What if we took our models seriously? Estimating latent scores in individuals. Journal of Psychoeducational Assessment 31 (2013), 186–201.
73. Schneider, W.J., McGrew, K.S., The Cattell–Horn–Carroll model of intelligence. Flanagan, D.P., Harrison, P., (eds.) Contemporary intellectual assessment, 3rd ed., 2012, Guilford Press, New York, NY, 99–145.
74. Schrank, F.A., McGrew, K.S., Mather, N., Woodcock–Johnson IV Tests of Cognitive Abilities. 2014, Riverside, Rolling Meadows, IL.
75. Silver, N.C., Dunlap, W.P., Averaging correlation coefficients: Should Fisher's z transformation be used?. Journal of Applied Psychology 72 (1987), 146–148.
76. Sternberg, R.J., Beyond IQ: A triarchic theory of human intelligence. 1985, Cambridge University Press, New York, NY.
77. Taub, G., Floyd, R.G., Keith, T.Z., McGrew, K.S., Effects of general and broad cognitive abilities on mathematics achievement from kindergarten through high school. School Psychology Quarterly 23 (2008), 187–198.
78. Thompson, B., Oblique rotations and higher-order factors. Exploratory and confirmatory factor analysis: Understanding concepts and applications, 2004, American Psychological Association, Washington, DC, 67–81.
79. Watkins, M.W., Errors in diagnostic decision making and clinical judgment. Gutkin, T.B., Reynolds, C.R., (eds.) Handbook of school psychology, 4th ed., 2009, Wiley, Hoboken, NJ, 210–229.
80. Watkins, M.W., Beaujean, A.A., Bifactor structure of the Wechsler preschool and primary scale of intelligence-fourth edition. School Psychology Quarterly 29 (2014), 52–63.
81. Wechsler, D., Wechsler intelligence scale for children. 5th ed., 2014, Psychological Corporation, San Antonio, TX.
82. Woodcock, R.W., McGrew, K.S., Mather, N., Woodcock–Johnson III. 2001, Riverside Publishing, Itasca, IL.