Study samples are too small to produce sufficiently precise reliability coefficients

Journal of General Psychology

Volumn 130, Issue 2, 2003, Pages 117-129

  Charter, Richard A ^a  

^a VA LONG BEACH HEALTHCARE SYSTEM   (United States)

Author keywords

Precision; Reliability; Sample size

Indexed keywords

ARTICLE; CLINICAL TRIAL; HUMAN; META ANALYSIS; PSYCHOLOGY; REPRODUCIBILITY; RESEARCH; SAMPLE SIZE; STATISTICS;

HUMANS; PSYCHOLOGY; REPRODUCIBILITY OF RESULTS; RESEARCH; SAMPLE SIZE; STATISTICS;

EID: 0042009577     PISSN: 00221309     EISSN: 19400888     Source Type: Journal    
DOI: 10.1080/00221300309601280     Document Type: Article

References (36)

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2. Andrulis, R. S. (1977). Adult assessment: A source book of tests and measures of human behavior. Springfield, IL: Charles C. Thomas.
3. Charter, R. A. (1997). Confidence interval procedures for retest, alternate-forms, validity, and alpha coefficients. Perceptual and Motor Skills, 84, 1488-1490.
4. Charter, R. A. (1999). Sample size requirements for precise estimates of reliability, gen-eralizability, and validity coefficients. Journal of Clinical and Experimental Neuropsychology, 21, 559-566.
5. Charter, R. A. (2000). Confidence interval formulas for split-half reliability coefficients. Psychological Reports, 86, 1168-1170.
6. Charter, R. A. (2001a). –Damn the precision, full speed ahead with the clinical interpretation.— Journal of Clinical and Experimental Neuropsychology, 23, 692-694.
7. Charter, R. A. (2001b). It is time to bury the Spearman-Brown –prophecy— formula for some common applications. Educational and Psychological Measurement, 61, 690-696.
8. Charter, R. A. (in press). Breakdown of reliability coefficients by test type and reliability method. The Journal of General Psychology.
9. Charter, R. A., Adkins, T., Alekoumbides, A., & Seacat, G. (1987). Reliability of the WAIS, WMS, and Reitan Battery: Raw scores and standardized scores corrected for age and education. International Journal of Clinical Neuropsychology, 9, 28-32.
10. Charter, R. A., & Feldt, L. S. (2000). The relationship between two methods of evaluating an examinee's difference scores. Journal of Psychoeducational Assessment, 18, 125-142.
11. Charter, R. A., & Feldt, L. S. (2001a). Confidence intervals for true scores: Is there a correct approach? Journal of Psychoeducational Assessment, 19, 350-364.
12. Charter, R. A., & Feldt, L. S. (2001b). Meaning of reliability in terms of correct and incorrect clinical decisions: The art of decision making is still alive. Journal of Clinical and Experimental Neuropsychology, 23, 530-537.
13. Charter, R. A., & Feldt, L. S. (2002). The importance of reliability as it relates to true score confidence intervals. Measurement and Evaluation in Counseling and Development, 35, 104-112.
14. Charter, R. A., Walden, D. K., & Hoffman, C. (2000). Interscorer reliabilities for Memory and Location scores of the Tactual Performance Test. The Clinical Neuropsychologist, 12, 245-247.
15. Charter, R. A., & Webster, J. S. (1997). The psychometric structure of the Seashore Rhythm Test. The Clinical Neuropsychologist, 11, 167-173.
16. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.
17. Corcoran, K., & Fisher, J. (2000). Measures for clinical practice: A source book (3rd ed.). New York: Free Press.
18. Fan, X., & Thompson, B. (2001). Confidence intervals about score reliability coefficients, please: An EPM guidelines editorial. Educational and Psychological Measurement, 61, 517-531.
19. Feldt, L. S. (1965). The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty. Psychometrika, 30, 357-370.
20. Feldt, L. S., & Ankenmann, R. D. (1998). Appropriate sample size for comparing alpha reliabilities. Applied Psychological Measurement, 22, 170-178.
21. Feldt, L. S., & Ankenmann, R. D. (1999). Determining sample size for a test of the equality of alpha coefficients when the number of part-tests is small. Psychological Methods, 4, 366-377.
22. Feldt, L. S., Woodruff, D. J., & Salih, F. A. (1987). Statistical inference for coefficient alpha. Applied Psychological Measurement, 11, 93-103.
23. Folbrecht, J. R., Charter, R. A., Walden, D. K., & Dobbs, S. M. (1999). Psychometric properties of the Boston Qualitative Scoring System for the Rey-Osterrieth Complex Figure. The Clinical Neuropsychologist, 13, 442-49.
24. Hersen, M., & Bellack, A. S. (Eds.). (1988). Dictionary of behavioral assessment techniques. New York: Pergamon.
25. Hogan, T. P., Benjamin, A., & Brezinski, K. L. (2000). Reliability methods: A note on the frequency of use of various types. Educational and Psychological Measurement, 60, 523-531.
26. Johnson, O. G. (1976). Tests and measurements in child development: Handbook 11 (Vols. 1-2). San Francisco: Jossey-Bass.
27. Keyser, D. J., & Sweetland, R. C. (Eds.). (1984-1988). Test critiques (Vols. 1-7). Kansas City, MO: Test Corporation of America.
28. Keyser, D. J., & Sweetland, R. C. (Eds.). (1991-1992). Test critiques (Vols. 8-9). Austin, TX: Pro-Ed.
29. Kline, P. (1986). A handbook of test construction: Introduction to psychometric design. New York: Methuen.
30. Lopez, M. N., Charter, R. A., & Newman, J. R. (2000). Psychometric properties of the Halstead Category Test. The Clinical Neuropsychologist, 14, 157-161.
31. Lopez, M. N., Charter, R. A., & Schelling, B. (2000). Interrater reliability of the Career Ability Placement Survey. Journal of Applied Rehabilitation Counseling, 31, 33-35.
32. Nunnally, J. C. (1967). Psychometric theory. New York: McGraw-Hill.
33. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
34. Rapport, L. J., Charter, R. A., Dutra, R. L., Farchione, T. J., & Kingsley, J. (1997). Psychometric properties of the Rey-Osterrieth Complex Figure: Lezak-Osterrieth versus Denman Scoring Systems. The Clinical Neuropsychologist, 11, 46-53.
35. Segali, D. O. (1994). The reliability of linearly equated tests. Psychometrika, 59, 361-375.
36. Shaw, D. J. (1966). The reliability and validity of the Halstead Category Test. Journal of Clinical Psychology, 22, 176-180.