Increasing the reliability of ipsative interpretations in neuropsychology: A comparison of reliable components analysis and other factor analytic methods

Journal of the International Neuropsychological Society

Volumn 10, Issue 4, 2004, Pages 578-589

  Frazier, Thomas W ^a,b   Youngstorm, Eric A ^a   Chelune, Gordon J ^b   Naugle, Richard I ^b   Lineweaver, Tara T ^b  

^a CASE WESTERN RESERVE UNIVERSITY   (United States)

^b LERNER RESEARCH INSTITUTE   (United States)

Author keywords

Battery interpretation; Difference scores; Ipsative; Reliable components analysis

Indexed keywords

ANALYTIC METHOD; ARTICLE; CLINICAL EXAMINATION; COGNITION; CONFIDENCE INTERVAL; CONTROLLED STUDY; DATA ANALYSIS; FACTORIAL ANALYSIS; HUMAN; INTERMETHOD COMPARISON; LANGUAGE ABILITY; MEMORY; NEUROPSYCHOLOGICAL TEST; PRIORITY JOURNAL; RELIABILITY; SCORING SYSTEM; VALIDATION PROCESS; VISUOMOTOR COORDINATION;

EID: 4043128863     PISSN: 13556177     EISSN: None     Source Type: Journal    
DOI: 10.1017/S1355617704104049     Document Type: Article

References (67)

1. Bauer, R.M. (2000). The flexible battery approach to neuropsychological assessment. In R.D. Vanderploeg (Ed.), Clinician's guide to neuropsychological assessment (pp. 419-448). Mahwah, NJ: Lawrence Erlbaum Associates.
2. Benton, A.L., Hamsher, K., & de S. Sivan, A.B. (1994). Multilingual aphasia examination. Iowa City, IA: AJA Associates.
3. Bowden, S.C., Fowler, K.S., Bell, R.C., Whelan, G., Clifford, C.C., Ritter, A.J., & Long, C.M. (1998). The reliability and internal validity of the Wisconsin Card Sorting Test. Neuropsychological Rehabilitation, 8, 243-254.
4. Buja, A. & Eyuboglu, N. (1992). Remarks on parallel analysis. Multivariate Behavioral Research, 27, 509-540.
5. Carroll, J.B. (1093). Human cognitive abilities: A survey of factor-analytic studies. New York: Cambridge University Press.
6. Caruso, J.C. (2001a). Increasing the reliability of the Fluid/ Crystallized Difference Score from the Kaufman Adolescent and Adult Intelligence Test with reliable component analysis. Assessment, 8, 155-166.
7. Caruso, J.C. (2001b). Reliable components analysis of the Stanford-Binet: Fourth Edition for 2- to 6-year olds. Psychological Assessment, 13, 261-266.
8. Caruso, J.C. & Cliff, N. (1998). The factor structure of the WAIS-R: Replicability across age groups. Multivariate Behavioral Research, 33, 291-308.
9. Caruso, J.C. & Cliff, N. (1999). The properties of equally and differentially weighted WAIS-III factor scores. Psychological Assessment, 11, 198-206.
10. Caruso, J.C. & Cliff, N. (2000). Increasing the reliability of Wechsler Intelligence Scale for Children-Third edition difference scores with reliable component analysis. Psychological Assessment, 12, 89-96.
11. Caruso, J.C. & Cliff, N. (2002). Reliable component analysis: A new/old method for exploratory data reduction. Unpublished manuscript, University of Montana at Missoula and University of Southern California at Los Angeles.
12. Caruso, J.C. & Witkiewitz, K. (2001). Memory and reasoning abilities assessed by the Universal Nonverbal Intelligence Test: A reliable component analysis (RCA) study. Educational and Psychological Measurement, 61, 5-22.
13. Caruso, J.C. & Witkiewitz, K. (2002). Increasing the reliability of ability-achievement difference scores: An example using the Kaufman Assessment Battery for Children. Journal of Educational Measurement, 39, 39-58.
14. Cerhan, J.H., Ivnik, R.J., Smith, G.E., Tangalos, E.C., Petersen, R.C., & Boeve, B.F. (2002). Diagnostic utility of letter fluency, category fluency, and fluency difference scores in Alzheimer's disease. Clinical Peuropsychologist, 16, 35-42.
15. Cliff, N. & Caruso, J.C. (1998). Reliable component analysis through maximizing composite reliability. Psychological Methods, 3, 291-308.
16. Corey, D.M., Dunlap, W.P., & Burke, M.J. (1998). Averaging correlations: Expected values and bias in combining Pearson r's and Fisher's z transformation. Journal of General Psychology, 125, 245-262.
17. Dikmen, S.S., Heaton, R.K., Grant, I., & Temkin, N.R. (1999). Test-retest reliability and practice effects of expanded Halstead-Reitan Neuropsychological Test Battery. Journal of the International Neuropsychological Society, 5, 346-356.
18. Fabrigar, L.R., Wegener, D.T., MacCallum, R.C., & Strahan, E.J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4, 272-299.
19. Fastenau, P.S., Denburg, N.L., & Mauer, B.A. (1998). Parallel short forms for the Boston Naming Test: Psychometric properties and norms for older adults. Journal of Clinical and Experimental Neuropsychology, 20, 828-834.
20. Fennell, E.B. (1995). The role of neuropsychological assessment in learning disabilities. Journal of Child Neurology, 10, S36-S41.
21. Franzen, M.D., Haut, M.W., Rankin, E., & Keefover, R. (1995). Empirical comparison of alternate forms of the Boston Naming Test. Clinical Neuropsychologist, 9, 225-229.
22. Franzen, M.D., Paul, D., & Iverson, G.L. (1996). Reliability of alternate forms of the Trail Making Test. Clinical Neuropsychologist, 10, 125-129.
23. Graham, J.W. & Schafer, J.L. (1999). On the performance of multiple imputation for multivariate data with small sample size. In R.H. Hoyle (Ed.), Statistical strategies for small sample research (pp. 1-29). Thousand Oaks, CA: Sage Publications, Inc.
24. Halstead, W.C. (1947). Brain and intelligence. Chicago: University of Chicago Press.
25. Heaton, R.K., Chelune, G.J., Talley, J.L., Kay, G.G. & Curtiss, G. (1993). Wisconsin Card Sorting Test manual: Revised and expanded. Odessa, FL: Psychological Assessment Resources.
26. Ingram, F., Greve, K. W., Ingram, P. T. F., & Soukup, V. M. (1999). Temporal stability of the Wisconsin Card Sorting Test in an untreated patient sample. British Journal of Clinical Psychology, 38, 209-211.
27. Ivnik, R.J., Smith, G.E., Petersen, R.C., Boeve, B.F., Kokmen, E., & Tangalos, E.G. (2000). Diagnostic accuracy of four approaches to interpreting neuropsychological test data. Neuropsychology, 14, 163-177.
28. Jacobson, M.W., Delis, D.C., Bondi, M.W., & Salmon, D.P. (2002). Do neuropsychological tests detect preclinical Alzheimer's disease: Individual-test versus cognitive-discrepancy score analyses. Neuropsychology, 16, 132-139.
29. Johnstone, B. & Wilhelm, K.L. (1996). The longitudinal stability of the WRAT-R reading subtest: Is it an appropriate estimate of premorbid intelligence? Journal of the International Neuropsychological Society, 2, 282-285.
30. Kaplan, E., Fein, D., Morris, R., & Delis, D.C. (1991).The WAIS-R as a neuropsychological instrument. Manual. San Antonio, TX: Psychological Corporation.
31. Kaplan, E., Goodglass, H., & Weintraub, S. (1983). The Boston Naming Test. Philadelphia: Lea and Febiger.
32. Kaufman, A.S. (1994). Intelligent testing with the WISC-III. New York: Wiley.
33. Keilp, J.G., Gorlyn, M., Alexander, G.E., Stern, Y., & Prohovnik, I. (1999). Cerebral blood flow patterns underlying the differential impairment in category vs. letter fluency in Alzheimer's disease. Neuropsychologia, 37, 1251-1261.
34. Kelley, T.L. (1927). Interpretation of educational measurements. Yonkers, NY: World Books.
35. Langeluddecke, P.M. & Lucas, S.K. (2003). Quantitative measures of memory malingering on the Wechsler Memory Scale-Third Edition in mild head injury litigants. Archives of Clinical Neuropsychology, 18, 181-197.
36. Lees-Haley, P.R., Smith, H.H., Williams, C.W., & Dunn, J.T. (1996). Forensic neuropsychological test usage: An empirical survey. Archives of Clinical Neuropsychology, 11, 45-51.
37. Lezak, M.D. (1995). Neuropsychological assessment (3rd ed.). New York: Oxford University Press.
38. Lord: F.M. & Novick, M.R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
39. Macmann, G.M. & Barnett, D.W. (1997). Myth of the master detective: Reliability of interpretations for Kaufman's "intelligent testing" approach to the WISC-III. School Psychology Quarterly, 12, 197-234.
40. Matarazzo, J.D. (1972). Measurement and appraisal of adult intelligence. Baltimore: Williams & Wilkins.
41. McCrae, R.R., Zonderman, A.B., Costa, P.T., Bond, M.H., & Paunonen, S.V.(1996). Evaluating Replicability of factors in the revised NEO Personality Inventory: Confirmatory factor analysis versus procrustes rotation. Journal of Personality and Social Psychology, 70, 552-566.
42. McDermott, P.A., Fantuzzo, J.W., Glutting, JJ, Watkins, M.W., & Baggaley, A.R. (1992). Illusions of meaning in the ipsative assessment of children's ability. Journal of Special Education, 25, 504-526.
43. Millis, S.R., Ross, S.R., & Ricker, J.H. (1998). Detection of incomplete effort on the Wechsler Adult Intelligence Scale-Revised: A cross-validation. Journal of Clinical and Experimental Neuropsychology, 20, 167-173.
44. Mitrushina, M.N., Boone, K.B., & D'Elia, L.F. (1999). Handbook of normative data for neuropsychological assessment. New York: Oxford University Press.
45. Nunnally, J.C. & Bernstein, I.H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill, Inc.
46. O'Connor, B.P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer's MAP test. Behavior Research Methods Instruments and Computers, 32, 396-402.
47. Reitan, R.M. (1955). Investigation of the validity of Halstead's measures of biological intelligence. Archives of Neurology and Psychiatry, 73, 28-35.
48. Reitan, R.M. (1958). Validity of the trail making test as an indicator of organic brain damage. Perceptual and Motor Skills, 8, 271-276.
49. Russell, E.W. (2000). The cognitive-metric, fixed battery approach to neuropsychological assessment. In R.D. Vanderploeg (Ed.), Clinician's guide to neuropsychological assessment (pp. 449-482); Mahwah, NJ: Lawrence Erlbaum Associates.
50. Schafer, J.L. (2002). Multiple imputation with PAN. In L. Collins & G. Aline (Eds.), New methods for the analysis of change (pp. 357-377). Washington, DC: American Psychological Association.
51. Scott, J.G., Krull, K.R., Williamson, D.J.G., Adams, R.L., & Iverson, G.L. (1997). Oklahoma Premorbid Intelligence Estimation (OPIE): Utilization in clinical samples. Clinical Neuropsychologist, 11, 146-154.
52. Soper, H.V., Cicchetti, .D V., Satz, P., Light, R., & Orsini, D.L. (1988). Null hypothesis disrespect in neuropsychology: Dangers of alpha and beta errors. Journal of Clinical and Experimental Neuropsychology, 10, 255-270.
53. Spreen, O. & Benton, A.L. (1969). Neurosensory Center Comprehensive Examination for Aphasia: Manual of directions. Victoria, British Columbia, Canada: Neuropsychology Laboratory, University of Victoria.
54. Spreen, O. & Strauss, E. (1998). A compendium of neuropsychological tests: Administration, norms, and commentary (2nd ed.). New York: Oxford University-Press.
55. SPSS, Inc. (2002). SPSS Professional Statistical Package (Version 11.0). Chicago: Author.
56. Streiner, D.L. & Norman, G.R. (1995). Health measurement scales: A practical guide to their development and use (2nd ed.). New York: Oxford University Press.
57. Tarter, R.E. & Edwards, K.L. (1986). Neuropsychological assessment. In T. Incagnoli, G. Goldstein, & C.J. Golden (Eds.), Clinical application of neuropsychological test batteries (pp. 135-153). New York: Plenum Press.
58. Tate, R.L., Perdices, M., & Maggiotto, S. (1998). Stability of the Wisconsin Card Sorting Test and the determination of reliability of change in scores. Clinical Neuropsychologist, 12, 348-357.
59. Tucker, L.R. (1951). A method for synthesis of factor analysis studies (Personnel Research Section Report No. 984). Washington, DC: Department of the Army.
60. Wechsler, D. (1997a). Manual for the Wechsler Adult Intelligence Scale-Third Edition. San Antonio, TX: The Psychological Corporation.
61. Wechsler, D. (1997b). Manual for the Wechsler Memory Scale-Third Edition. San Antonio, TX: The Psychological Corporation.
62. Wechsler, D. (1997c). WAIS-III WMS-III technical manual. San Antonio, TX: The Psychological Corporation.
63. Widaman, K.F. (1993). Common factor analysis versus principal component analysis: Differential bias in representing model parameters? Multivariate Behavioral Research, 28, 263-311.
64. Wilde, N., Strauss, E., Chelune, G.J., Loring, D.W., Martin, R.C., Hermann, B.P., Sherman, E.M.S., & Hunter, M. (2001). WMS-III performance in patients with temporal lobe epilepsy: Group differences and individual classification. Journal of the International Neuropsychological Society, 7, 881-891.
65. Wilkinson, G.S. (1993). The Wide Range Achievement Test-3. San Antonio, TX: The Psychological Corporation.
66. Williams, V.S.L., Jones, L.V., & Tukey, J.W. (1999). Controlling error in multiple comparisons, with examples from state-to-state differences in educational achievement. Journal of Educational and Behavioral Statistics, 24, 42-69.
67. Zwick, W.R. & Velicer, W.F. (1986). Comparison of five rules for determining the number of components to retain. Psychological Bulletin, 99, 432-442.