Enhancing accuracy in observational test scoring: The comprehensive system as a case example

Journal of Personality Assessment

Volumn 81, Issue 2, 2003, Pages 104-110

  McGrath, Robert E ^a  

^a FAIRLEIGH DICKINSON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACCURACY; BEHAVIOR; CLINICAL OBSERVATION; DIAGNOSTIC ERROR; DIAGNOSTIC PROCEDURE; ETHICS; EVALUATION; HUMAN; INTELLIGENCE TEST; NEUROPSYCHOLOGICAL TEST; PERSONALITY TEST; REVIEW; RISK; RORSCHACH COMPREHENSIVE SYSTEM; SCORING SYSTEM; SOCIAL PSYCHOLOGY; STANDARDIZATION; SYSTEM ANALYSIS; VALIDATION PROCESS;

EID: 0141732283     PISSN: 00223891     EISSN: None     Source Type: Journal    
DOI: 10.1207/S15327752JPA8102_02     Document Type: Review

References (39)

1. Allard, G., Butler, J., Faust, D., & Shea, M. T. (1995). Errors in hand scoring objective personality tests. Professional Psychology: Research & Practice, 26, 304-308.
2. Allard, G., & Faust, D. (2000). Errors in scoring objective personality tests. Assessment, 7, 119-129.
3. American Psychological Association. (2002). Ethical principles of psychologists and code of conduct. American Psychologist, 57, 1060-1073.
4. Beck, A. T., Steer, R. A., & Brown, G. K. (1996). The Beck Depression Inventory-Second edition. San Antonio, TX: Psychological Corporation.
5. Berry, K. J., & Mielke, P. W., Jr. (1997). Measuring the joint agreement between multiple raters and a standard. Educational and Psychological Measurement, 57, 527-530.
6. Butcher, J. N., Dahlstrom, W. G., Graham, J. R., Tellegen, A., & Kaemmer, B. (1989). MMPI-2 (Minnesota Multiphasic Personality Inventory-2): Manual for administration and scoring. Minneapolis: University of Minnesota Press.
7. Campbell, T. W. (2000). Sexual predator evaluations and phrenology: Considering issues of evidentiary reliability. Behavioral Sciences and the Law, 18, 111-130.
8. Clinical Laboratory Improvement Act, Pub. L. No. 90-174. (1967).
9. Clinical Laboratory Improvement Act-Revised, Pub. L. No. 100-578. (1988).
10. Exner, J. E., Jr. (1993). The Rorschach: A Comprehensive System: Vol. 1. Basic foundations (3rd ed.). New York: Wiley.
11. Exner, J. E., Jr. (2001). A Rorschach workbook for the Comprehensive System (5th ed.). Asheville, NC: Rorschach Workshops.
12. Garb, H. (1998). Studying the clinician: Judgment research and psychological assessment. Washington, DC: American Psychological Association.
13. Guamaccia, V., Dill, C. A., Sabatino, S., & Southwick, S. (2001). Scoring accuracy using the Comprehensive System for the Rorschach. Journal of Personality Assessment, 77, 464-474.
14. Hare, R. D. (1991). The Hare Psychopathy Checklist-Revised. Toronto, Ontario, Canada: Multi-Health Systems.
15. Hunsley, J., & Bailey, J. M. (1999). The clinical utility of the Rorschach: Unfulfilled promises and an uncertain future. Psychological Assessment, 11, 266-277.
16. Hunsley, J., & Bailey, J. M. (2001). Whither the Rorschach? An analysis of the evidence. Psychological Assessment, 13, 472-485.
17. Klassen, R. M., & Kishor, N. (1996). A comparative analysis of practitioners' errors on WISC-R and WISC-III. Canadian Journal of School Psychology, 12, 35-43.
18. Krishnamurthy, R. (2001, March). Practical issues and logistics of having a gold standard. In S. D. Hickman (Chair), Exploring the implications of a Rorschach coding "gold standard." Symposium conducted at the midwinter meeting of the Society for Personality Assessment, Philadelphia.
19. Lees-Haley, P. R. (1992). Psychodiagnostic test usage by forensic psychologists. American Journal of Forensic Psychology, 10, 25-30.
20. Light, R. J. (1971). Measures of response agreement for qualitative data: Some generalizations and alternatives. Psychological Bulletin, 76, 365-377.
21. Martin, M., Allan, A., & Allan, M. M. (2001). The use of psychological tests by Australian psychologists who do assessments for the courts. Australian Journal of Psychology, 53, 77-82.
22. McKinzey, R. K., & Campagna, V. (2002, April 27). The Rorschach, Exner's Comprehensive System, interscorer agreement, and death. Retrieved November 22, 2002, from http://home.earthlink.net/-rkmck/vault/ Rorinterscor/McKCam02.pdf
23. Meyer, G. J., Hilsenroth, M. J., Baxter, D., Exner, J. E., Jr., Fowler, J. C., Piers, C. C., et al. (2002). An examination of interrater reliability for scoring the Rorschach Comprehensive System in eight data sets. Journal of Personality Assessment, 78, 219-274.
24. Murray, H. A. (1943). Thematic Apperception Test: Manual. Cambridge, MA: Harvard University Press.
25. Peterson, J. L., & Markham, P. N. (1995). Crime laboratory proficiency testing results, 1978-1991. I: Identification and classification of physical evidence. Journal of Forensic Sciences, 40, 994-1008.
26. Pogge, D. L., McGrath, R. E., Stokes, J. M., Cragnolino, A., Zaccario, M., Hayman, J., et al. (2002). Comprehensive System scoring reliability in an adolescent inpatient sample. Manuscript submitted for publication.
27. Simons, R., Goddard, R., & Patton, W. (2002). Hand-scoring error rates in psychological testing. Assessment, 9, 292-300.
28. Slate, J. R., & Jones, C. H. (1990). Examiner errors on the WAIS-R: A source of concern. Journal of Psychology, 124, 343-345.
29. Slate, J. R., Jones, C. H., Coulter, C., & Covert, T. L. (1992). Practitioners' administration and scoring of the WISC-R: Evidence that we do err. Journal of School Psychology, 30, 77-82.
30. Slate, J. R., Jones, C. H., Murray, R. A., & Coulter, C. (1993). Evidence that practitioners err in administering and scoring the WAIS-R. Measurement and Evaluation in Counseling and Development, 25, 156-161.
31. Smith, G. A. (1986). Observer drift: A drifting definition. Behavior Analyst, 9, 127-128.
32. Speilberger, C. D. (1983). Manual for the State-Trait Anxiety Inventory (STAI). Palo Alto, CA: Consulting Psychologists Press.
33. Viglione, D. J. (2002). Rorschach coding solutions: A reference guide for the Comprehensive System. San Diego, CA: Author. Available at http://www.geocities.com/donaldviglione
34. Wakefield, H., & Underwager, R. (1993). Misuse of psychological tests in forensic settings: Some horrible examples. American Journal of Forensic Psychology, 11, 55-75.
35. Wechsler, D. (1981). Manual for the Wechsler Adult Intelligence Scale-Revised (WAIS-R). San Antonio, TX: Psychological Corporation.
36. Wechsler, D. (1997). WAIS-III administration and scoring manual. San Antonio, TX: Psychological Corporation.
37. Wennberg, J., & Gittelsohn, A. (1982, April). Variations in medical care among small areas. Scientific American, 246(4), 120-134.
38. Whitten, J., Slate, J. R., Jones, C. H., & Shine, A. E. (1994). Examiner errors in administering and scoring the WPPSI-R. Journal of Psychoeducational Assessment, 12, 49-54.
39. Wood, J. M., Nezworski, M. T., & Stejskal, W. J. (1996). The Comprehensive System for the Rorschach: A critical examination. Psychological Science, 7, 3-10.