Incremental criterion validity of WAIS-IV factor index scores: Relationships with WIAT-II and WIAT-III subtest and composite scores

Psychological Assessment

Volumn 25, Issue 2, 2013, Pages 484-495

  Canivez, Gary L ^a  

^a EASTERN ILLINOIS UNIVERSITY   (United States)

Author keywords

Achievement prediction; Construct validity; Hierarchical multiple regression; Incremental validity; WAIS IV

Indexed keywords

ACADEMIC ACHIEVEMENT; ADULT; ARTICLE; CONSTRUCT VALIDITY; CONTROLLED STUDY; FEMALE; HUMAN; INCREMENTAL CRITERION VALIDITY; INTERMETHOD COMPARISON; MALE; PREDICTION; SCORING SYSTEM; VALIDITY; WECHSLER INDIVIDUAL ACHIEVEMENT TEST; WECHSLER INTELLIGENCE SCALE;

ACHIEVEMENT; ADOLESCENT; EDUCATIONAL MEASUREMENT; FACTOR ANALYSIS, STATISTICAL; FEMALE; HUMANS; MALE; PREDICTIVE VALUE OF TESTS; PSYCHOMETRICS; REPRODUCIBILITY OF RESULTS; WECHSLER SCALES;

EID: 84882571435     PISSN: 10403590     EISSN: 1939134X     Source Type: Journal    
DOI: 10.1037/a0032092     Document Type: Article

References (104)

1. American Educational Research Association, American Psychological Association, and the National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2. Benson, N., Hulac, D. M., & Kranzler, J. H. (2010). Independent examination of the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS- IV): What does the WAIS-IV measure?. Psychological Assessment, 22, 121-130.
3. Binet, A., & Simon, T. (1904). Methodes nouvelle pour le diagnostic du niveau intellectual des anormaux [New methods for the diagnosis of the abnormal intellectual level]. L'Annee Psychologique, 11, 191-244. doi: 10.3406/psy.1904.3675
4. Bodin, D., Pardini, D. A., Burns, T. G., & Stevens, A. B. (2009). Higher order factor structure of the WISC-IV in a clinical neuropsychological sample. Child Neuropsychology, 15, 417- 424. doi:10.1080/ 09297040802603661
5. Brody, N. (2002). g and the one-many problem: Is one enough? In G. R. Bock, J. A. Goode, & K. Webb, The nature of intelligence: Novartis Foundation Symposium 233 (pp. 122-135). New York, NY: Wiley.
6. Brown, R. T., Reynolds, C. R., & Whitaker, J. S. (1999). Bias in mental testing since bias in mental testing. School Psychology Quarterly, 14, 208-238. doi:10.1037/h0089007
7. Canivez, G. L. (2008). Orthogonal higher order factor structure of the Stanford-Binet Intelligence Scales for children and adolescents. School Psychology Quarterly, 23, 533-541. doi:10.1037/a0012884
8. Canivez, G. L. (2010). Review of the Wechsler Adult Intelligence Test-Fourth Edition. In R. A. Spies, J. F. Carlson, & K. F. Geisinger (Eds.), The eighteenth mental measurements yearbook (pp. 684-688). Lincoln, NE: Buros Institute of Mental Measurements.
9. Canivez, G. L. (2011). Hierarchical factor structure of the cognitive assessment system: Variance partitions from the Schmid-Leiman (1957) procedure. School Psychology Quarterly, 26, 305-317. doi:10.1037/ a0025973
10. Canivez, G. L. (2012). Incremental validity of the cognitive assessment system: Predicting achievement with CAS Full Scale and PASS Scale scores. Manuscript submitted for publication.
11. Canivez, G. L., Konold, T. R., Collins, J. M., & Wilson, G. (2009). Construct validity of the Wechsler Abbreviated Scale of Intelligence and Wide Range Intelligence Test: Convergent and structural validity. School Psychology Quarterly, 24, 252-265. doi:10.1037/a0018030
12. Canivez, G. L., & Kush, J. C. (in press). WISC-IV and WAIS-IV structural validity: Alternate methods, alternate results. Commentary on Weiss et al. (2013a) and Weiss et al. (2013b). Journal of Psychoeducational Assessment.
13. Canivez, G. L., & Watkins, M. W. (2010a). Exploratory and higher order factor analyses of the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) adolescent subsample. School Psychology Quarterly, 25, 223-235. doi:10.1037/a0022046
14. Canivez, G. L., & Watkins, M. W. (2010b). Investigation of the factor structure of the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV): Exploratory and higher order factor analyses. Psychological Assessment, 22, 827-836. doi:10.1037/a0020429
15. Carretta, T. R., & Ree, J. J. (2001). Pitfalls of ability research. International Journal of Selection and Assessment, 9, 325-335. doi:10.1111/1468-2389.00184
16. Carroll, J. B. (1993). Human cognitive abilities. Cambridge, England: Cambridge University Press. doi:10.1017/CBO9780511571312
17. Carroll, J. B. (1995). On methodology in the study of cognitive abilities. Multivariate Behavioral Research, 30, 429 -452. doi:10.1207/ s15327906mbr3003_6
18. Carroll, J. B. (1997). Theoretical and technical issues in identifying a factor of general intelligence. In B. Devlin, S. E. Fienberg, D. P. Resnick, & K. Roeder (Eds.), Intelligence, genes, and success: Scientists respond to the bell curve (pp. 125-156). New York, NY: Springer-Verlag. doi: 10.1007/978-1-4612-0669-9_6
19. Carroll, J. B. (2003). The higher stratum structure of cognitive abilities: Current evidence supports g and about ten broad factors. In H. Nyborg (Ed.), The scientific study of general intelligence: Tribute to Arthur R. Jensen (pp. 5-21). New York, NY: Pergamon Press. doi:10.1016/B978-008043793-4/50036-2
20. Cattell, R. B. (1943). The measurement of adult intelligence. Psychological Bulletin, 40, 153-193. doi:10.1037/h0059973
21. Cattell, R. B., & Horn, J. L. (1978). A check on the theory of fluid and crystallized intelligence with description of new subtest designs. Journal of Educational Measurement, 15, 139-164. doi:10.1111/j.1745-3984 .1978.tb00065.x
22. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
23. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334. doi:10.1007/BF02310555
24. Dana, J., & Dawes, R. M. (2007). Comment on Fiorello et al. "Interpreting intelligence test results for children with disabilities: Is global intelligence relevant?" Applied Neuropsychology, 14, 21-25. doi:10.1080/ 09084280701280379
25. Dombrowski, S. C., Watkins, M. W., & Brogan, M. J. (2009). An exploratory investigation of the factor structure of the Reynolds Intellectual Assessment Scales (RIAS). Journal of Psychoeducational Assessment, 27, 494-507. doi:10.1177/0734282909333179
26. Elliott, C. D. (2007). Differential Ability Scales: Introductory and technical handbook (2nd ed.). San Antonio, TX: The Psychological Corporation.
27. Evans, J. J., Floyd, R. G., McGrew, K. S., & Leforgee, M. H. (2002). The relations between measures of Cattell-Horn-Carroll (CHC) cognitive abilities and reading achievement during childhood and adolescence. School Psychology Review, 31, 246-262.
28. Flanagan, D. P. (2000). Wechsler-based CHC cross-battery assessment and reading achievement: Strengthening the validity of interpretations drawn from Wechsler test scores. School Psychology Quarterly, 15, 295-329. doi:10.1037/h0088789
29. Freberg, M. E., Vandiver, B. J., Watkins, M. W., & Canivez, G. L. (2008). Significant factor score variability and the validity of the WISC-III Full Scale IQ in predicting later academic achievement. Applied Neuropsychology, 15, 131-139. doi:10.1080/09084280802084010
30. Gignac, G. E. (2008). Higher order models versus direct hierarchical models: g as superordinate or breadth factor?. Psychology Science Quarterly, 50, 21-43.
31. Glutting, J. J., Adams, W., & Sheslow, D. (2000). Wide Range Intelligence Test: Manual. Wilmington, DE: Wide Range.
32. Glutting, J. J., Watkins, M. W., Konold, T. R., & McDermott, P. A. (2006). Distinctions without a difference: The utility of observed versus latent factors from the WISC-IV in estimating reading and math achievement on the WIAT-II. The Journal of Special Education, 40, 103-114. doi:10.1177/00224669060400020101
33. Glutting, J. J., Youngstrom, E. A., Ward, T., Ward, S., & Hale, R. L. (1997). Incremental efficacy of WISC-III factor scores in predicting achievement: What do they tell us?. Psychological Assessment, 9, 295-301. doi:10.1037/1040-3590.9.3.295
34. Golay, P., & Lecerf, T. (2011). Orthogonal higher order structure and confirmatory factor analysis of the French Wechsler Adult Intelligence Scale (WAIS-III). Psychological Assessment, 23, 143-152. doi: 10.1037/a0021230
35. Golay, P., Reverte, I., Rossier, J., Favez, N., & Lecerf, T. (2012). Further insights on the French WISC-IV factor structure through Bayesian structural equation modeling (BSEM). Psychological Assessment. Advance online publication. doi:10.1037/a0030676
36. Gottfredson, L. S. (1997). Intelligence and social policy. Intelligence, 24, 288-320. doi:10.1016/S0160-2896(97)90010-6
37. Gottfredson, L. S. (2008). Of what value is intelligence? In A. Prifitera, D. Saklofske, & L. G. Weiss (Eds.), WISC-IV clinical assessment and intervention (2nd ed., pp. 545-564). Amsterdam, the Netherlands: Elsevier.
38. Gustafsson, J.-E., & Balke, G. (1993). General and specific abilities as predictors of school achievement. Multivariate Behavioral Research, 28, 407-434. doi:10.1207/s15327906mbr2804_2
39. Gustafsson, J.-E., & Snow, R. E. (1997). Ability profiles. In R. F. Dillon (Ed.), Handbook on testing (pp. 107-135). Westport, CT: Greenwood Press.
40. Gustafsson, J. E., & Undheim, J. O. (1996). Individual differences in cognitive functions. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of educational psychology (pp. 186-242). New York, NY: Macmillan.
41. Hale, J. B., Fiorello, C. A., Kavanagh, J. A., Holdnack, J. A., & Aloe, A. M. (2007). Is the demise of IQ interpretation justified? A response to special issue authors. Applied Neuropsychology, 14, 37-51. doi:10.1080/ 09084280701280445
42. Haynes, S. N., & Lench, H. C. (2003). Incremental validity of new clinical assessment measures. Psychological Assessment, 15, 456-466. doi: 10.1037/1040-3590.15.4.456
43. Holzinger, K. J., & Swineford, F. (1937). The bi-factor method. Psychometrika, 2, 41-54. doi:10.1007/BF02287965
44. Horn, J. L. (1988). Thinking about human abilities. In R. Nesselroade & R. B. Cattell (Eds.), Handbook of multivariate experimental psychology (2nd ed., pp. 645-685). New York, NY: Plenum Press. doi:10.1007/ 978-1-4613-0893-5_19
45. Horn, J. L. (1991). Measurement of intellectual capabilities: A review of theory. In K. S. McGrew, J. K. Werder, & R. W. Woodcock (Eds.), Woodcock-Johnson technical manual (Rev. ed., pp. 197-232). Itasca, IL: Riverside.
46. Horn, J. L., & Cattell, R. B. (1966). Refinement and test of the theory of fluid and crystallized general intelligence. Journal of Educational Psychology, 57, 253-270. doi:10.1037/h0023816
47. Horn, J. L., & Noll, J. (1997). Human cognitive capabilities: Gf-Gc theory. In D. P. Flanagan, J. L. Genshaft, & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 53-91). New York, NY: Guilford Press.
48. Hunsley, J. (2003). Introduction to the special section on incremental validity and utility in clinical assessment. Psychological Assessment, 15, 443-445. doi:10.1037/1040-3590.15.4.443
49. Hunsley, J., & Meyer, G. J. (2003). The incremental validity of psychological testing and assessment: Conceptual, methodological, and statistical issues. Psychological Assessment, 15, 446-455. doi:10.1037/1040-3590.15.4.446
50. Jensen, A. R. (1998). The g factor: The science of mental ability. Westport, CT: Praeger.
51. Jones, W. T. (1952). A history of Western philosophy. New York, NY: Harcourt Brace.
52. Kahana, S. Y., Youngstrom, E. A., & Glutting, J. J. (2002). Factor and subtest discrepancies on the Differential Abilities Scale: Examining prevalence and validity in predicting academic achievement. Assessment, 9, 82-93. doi:10.1177/1073191102009001010
53. Kamphaus, R. W., Winsor, A. P., Rowe, E. W., & Kim, S. (2005). A history of intelligence test interpretation. In D. P. Flanagan & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (2nd ed., pp. 23-38). New York, NY: Guilford Press.
54. Kaufman, A. S., & Kaufman, N. L. (1993). Kaufman Adolescent and Adult Intelligence Test. Circle Pines, MN: American Guidance Service.
55. Kaufman, A. S., & Kaufman, N. L. (2004). Kaufman Assessment Battery for Children (2nd ed.). Circle Pines, MN: American Guidance Service.
56. Kline, P. (1994). An easy guide to factor analysis. London, England: Routledge.
57. Lei, P.-W., & Wu, Q. (2007, Fall). Introduction to structural equation modeling: Issues and practical considerations. ITEMS: Instructional Topics in Educational Measurement, 33-43.
58. Lubinski, D. (2000). Scientific and social significance of assessing individual differences: "Sinking shafts at a few critical points". Annual Review of Psychology, 51, 405-444. doi:10.1146/annurev.psych.51.1 .405
59. Lubinski, D., & Dawis, R. V. (1992). Aptitudes, skills, and proficiencies. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed., Vol. 3, pp. 1-59). Palo Alto, CA: Consulting Psychologists Press.
60. Lubinski, D., & Humphreys, L. G. (1997). Incorporating general intelligence into epidemiology and the social sciences. Intelligence, 24, 159-201. doi:10.1016/S0160-2896(97)90016-7
61. McClain, A. L. (1996). Hierarchical analytic methods that yield different perspectives on dynamics: Aids to interpretation. Advances in Social Science Methodology, 4, 229-240.
62. McFall, R. M. (2005). Theory and utility: Key themes in evidence-based assessment. Comment on the special section. Psychological Assessment, 17, 312-323. doi:10.1037/1040-3590.17.3.312
63. McGrew, K. S. (2005). The Cattell-Horn-Carroll theory of cognitive abilities: Past, present, and future. In D. P. Flanagan & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (2nd ed., pp. 136-181.). New York, NY: Guilford Press.
64. Meehl, P. E. (1959). Some ruminations on the validation of clinical procedures. Canadian Journal of Psychology/Revue canadienne de psychologie, 13, 102-128. doi:10.1037/h0083769
65. Mullins-Sweatt, S. N., & Widiger, T. A. (2009). Clinical utility and DSM-V. Psychological Assessment, 21, 302-312. doi:10.1037/ a0016607
66. Naglieri, J. A., & Bornstein, B. T. (2003). Intelligence and achievement: Just how correlated are they?. Journal of Psychoeducational Assessment, 21, 244-260. doi:10.1177/073428290302100302
67. Naglieri, J. A., & Das, J. P. (1997a). Cognitive Assessment System. Itasca, IL: Riverside.
68. Naglieri, J. A., & Das, J. P. (1997b). Cognitive Assessment System: Interpretive handbook. Itasca, IL: Riverside.
69. NCS Pearson. (2009a). Wechsler Individual Achievement Test-Third Edition. San Antonio, TX: Author.
70. NCS Pearson. (2009b). Wechsler Individual Achievement Test-Third Edition technical manual. San Antonio, TX: Author.
71. Neisser, U., Boodoo, G., Bouchard, Jr, T. J., Boykin, A. W., Brody, N., Ceci, S. J., . . . Urbina, S. (1996). Intelligence: Knowns and unknowns. American Psychologist, 51, 77-101. doi:10.1037/0003-066X.51.2.77
72. Nelson, J. M., & Canivez, G. L. (2012). Examination of the structural, convergent, and incremental validity of the Reynolds Intellectual Assessment Scales (RIAS) with a clinical sample. Psychological Assessment, 24, 129-140. doi:10.1037/a0024878
73. Nelson, J. M., Canivez, G. L., Lindstrom, W., & Hatt, C. (2007). Higher order exploratory factor analysis of the Reynolds Intellectual Assessment Scales with a referred sample. Journal of School Psychology, 45, 439-456. doi:10.1016/j.jsp.2007.03.003
74. Oh, H. J., Glutting, J. J., Watkins, M. W., Youngstrom, E. A., & McDermott, P. A. (2004). Correct interpretation of latent versus observed abilities: Implications from structural equation modeling applied to the WISC-III and WIAT linking sample. Journal of Special Education, 38, 159-173. doi:10.1177/00224669040380030301
75. Pedhazur, E. J. (1997). Multiple regression in behavioral research: Explanation and prediction (3rd ed.). New York, NY: Harcourt Brace.
76. Psychological Corporation. (2002a). Wechsler Individual Achievement Test-Second Edition. San Antonio, TX: Author.
77. Psychological Corporation. (2002b). Wechsler Individual Achievement Test-Second Edition examiner's manual. San Antonio, TX: Author.
78. Ree, M. J., Carretta, T. R., & Green, M. T. (2003). The ubiquitous role of g in training. In H. Nyborg (Ed.), The scientific study of general intelligence: Tribute to Arthur R. Jensen (pp. 262-274). New York, NY: Pergamon Press. doi:10.1016/B978-008043793-4/50051-9
79. Reynolds, C. R., & Kamphaus, R. W. (2003). Reynolds Intellectual Assessment Scales. Lutz, FL: Psychological Assessment Resources.
80. Roid, G. H. (2003). Stanford-Binet Intelligence Scales: Fifth Edition: Technical manual. Itasca, IL: Riverside.
81. Ryan, J. J., Kreiner, D. S., & Burton, D. B. (2002). Does high scatter affect the predictive validity of WAIS-III IQs?. Applied Neuropsychology, 9, 173-178. doi:10.1207/S15324826AN0903_5
82. Schmid, J., & Leiman, J. M. (1957). The development of hierarchical factor solutions. Psychometrika, 22, 53-61. doi:10.1007/BF02289209
83. Schneider, J. (2008). Playing statistical ouija board with commonality analysis (and other errors). Applied Neuropsychology, 15, 44-53. doi: 10.1080/09084280801917566
84. Schraw, G. (2010). Test review of the Wechsler Adult Intelligence Scale-Fourth Edition. In R. A. Spies, J. F. Carlson, & K. F. Geisinger (Eds.), The eighteenth mental measurements yearbook (pp. 688-690). Lincoln, NE: Buros Institute of Mental Measurements.
85. Sechrest, L. (1963). Incremental validity: A recommendation. Educational and Psychological Measurement, 23, 153-158. doi:10.1177/ 001316446302300113
86. Thompson, B. (2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. Washington, DC: American Psychological Association. doi:10.1037/10694-000
87. Thorndike, R. L. (1986). The role of general ability in prediction. Journal of Vocational Behavior, 29, 332-339. doi:10.1016/0001-8791(86)90012-6
88. Ward, L. C., Bergman, M. A., & Hebert, K. R. (2012). WAIS-IV subtest covariance structure: Conceptual and statistical considerations. Psychological Assessment, 24, 328-340. doi:10.1037/a0025614
89. Watkins, M. W. (2006). Orthogonal higher order structure of the Weschsler Intelligence Scale for Children-Fourth edition. Psychological Assessment, 18, 123-125. doi:10.1037/1040-3590.18.1.123
90. Watkins, M. W., Glutting, J. J., & Lei, P.-W. (2007). Validity of the Full-Scale IQ when there is significant variability among WISC-III and WISC-IV factor scores. Applied Neuropsychology, 14, 13-20. doi: 10.1080/09084280701280353
91. Watkins, M. W., Wilson, S. M., Kotz, K. M., Carbone, M. C., & Babula, T. (2006). Factor structure of the Wechsler Intelligence Scale for Children-Fourth edition among referred students. Educational and Psychological Measurement, 66, 975-983. doi:10.1177/0013164406288168
92. Wechsler, D. (1939a). The measurement of adult intelligence. Baltimore, MD: Williams & Wilkins. doi:10.1037/10020-000
93. Wechsler, D. (1939b). Wechsler Bellevue Intelligence Scale. New York, NY: The Psychological Corporation.
94. Wechsler, D. (1991). Wechsler Intelligence Scale for Children (3rd ed.). San Antonio, TX: Psychological Corporation.
95. Wechsler, D. (2003). Wechsler Intelligence Scale for Children-Fourth Edition technical and interpretive manual. San Antonio, TX: Psychological Corporation.
96. Wechsler, D. (2008a). Wechsler Adult Intelligence Scale-Fourth Edition. San Antonio, TX: Pearson.
97. Wechsler, D. (2008b). Wechsler Adult Intelligence Scale-Fourth Edition: Technical and interpretive manual. San Antonio, TX: Pearson.
98. Wechsler, D., & Naglieri, J. A. (2006). Wechsler Nonverbal Scale of Ability. San Antonio, TX: Psychological Corporation.
99. Weiner, I. B. (2003). Prediction and postdiction in clinical decision making. Clinical Psychology: Science and Practice, 10, 335-338. doi: 10.1093/clipsy.bpg030
100. Weiss, L. G., Keith, T. Z., Zhu, J., & Chen, H. (in press). WAIS-IV and clinical validation of the four-and five-factor interpretative approaches. Journal of Psychoeducational Assessment.
101. Weiss, L. G., & Prifitera, A. (1995). An evaluation of differential prediction of WIAT achievement scores from WISC-III FSIQ across ethnic and gender groups. Journal of School Psychology, 33, 297-304. doi: 10.1016/0022-4405(95)00016-F
102. Wiggins, J. S. (1988). Personality and prediction: Principles of personality assessment. Malabar, FL: Krieger.
103. Woodcock, R. W., McGrew, K. S., & Mather, N. (2001). Woodcock-Johnson III Tests of Cognitive Abilities. Itasca, IL: Riverside.
104. Youngstrom, E. A., Kogos, J. L., & Glutting, J. J. (1999). Incremental efficacy of Differential Ability Scales factor scores in predicting individual achievement criteria. School Psychology Quarterly, 14, 26-39. doi:10.1037/h0088996