The interclinician reliability of Rorschach interpretation in four data sets

Journal of Personality Assessment

Volumn 84, Issue 3, 2005, Pages 296-314

  Meyer, Gregory J ^a   Mihura, Joni L ^a   Smith, Brace L ^b  

^a UNIVERSITY OF TOLEDO   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; CLINICAL PRACTICE; DATA ANALYSIS; FEMALE; HUMAN; MAJOR CLINICAL STUDY; MALE; MENTAL PATIENT; PERSONALITY TEST; PSYCHOLOGIC ASSESSMENT; PSYCHOMETRY; RELIABILITY; REVIEW; RORSCHACH COMPREHENSIVE SYSTEM; SCORING SYSTEM;

EID: 19444366112     PISSN: 00223891     EISSN: None     Source Type: Journal    
DOI: 10.1207/s15327752jpa8403_09     Document Type: Review

References (55)

1. Acklin, M. W., McDowell, C. J., & Verschell, M. S. (2000). Interobserver agreement, intraobserver reliability, and the Rorschach Comprehensive System. Journal of Personality Assessment, 74, 15-47.
2. Allen, M. J., & Yen, W. M. (1979). Introduction to measurement theory. Monterey, CA: Brooks/Cole.
3. Blackman, M. C., & Funder, D. C. (1998). The effect of information on consensus and accuracy in personality judgment. Journal of Experimental Social Psychology, 34, 164-181.
4. Block, J. (1961). The Q-sort method in personality assessment and psychiatric research. Springfield, IL: Thomas.
5. Borum, R., Otto, R., & Golding, S. (1993). Improving clinical judgment and decision making in forensic evaluation. Journal of Psychiatry & Law, 21, 35-76.
6. Butcher, J. N., Dahlstrom, W. G., Graham, J. R., Tellegen, A., & Kaemmer, B. (1989). MMPI-2: Minnesota Multiphasic Personality Inventory-2. Manual for administration and scoring. Minneapolis: University of Minnesota Press.
7. Chapman, L. J., & Chapman, J. P. (1969). Illusory correlation as an obstacle to the use of valid psychodiagnostic signs. Journal of Abnormal Psychology, 74, 271-280.
8. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284-290.
9. Conway, J. M., Jako, R. A., & Goodman, D. F. (1995). A meta-analysis of interrater and internal consistency reliability of selection interviews. Journal of Applied Psychology, 80, 565-579.
10. Cooke, J. K. (1967). Clinician's decisions as a basis for deriving actuarial formulae. Journal of Clinical Psychology, 23, 232-233.
11. Elmore, J. G., & Feinstein, A. R. (1992). A bibliography of publications on observer variability (final installment). Journal of Clincial Epidemiology, 45, 567-580.
12. Exner, J. E., Jr. (2003). The Rorschach: A Comprehensive System: Volume 1. Basic foundations and principles of interpretation (4th Ed.). Hoboken, NJ: Wiley.
13. Fagot, R. F. (1991). Reliability of ratings for multiple judges: Intraclass correlation and metric scales. Applied Psychological Measurement, 15, 1-12.
14. Fleiss, J. L. (1981). Statistical methods for rates and proportions (2nd ed.). New York: Wiley.
15. Funder, D. C. (1995). On the accuracy of personality judgment: A realistic approach. Psychological Review, 102, 652-670.
16. Funder, D. C. (1999). Personality judgment: A realistic approach to person perception. San Diego, CA: Academic.
17. Garb, H. N. (1998). Studying the clinician: Judgment research and psychological assessment. Washington, DC: American Psychological Association.
18. Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12, 19-30.
19. Guadagnoli, E., & Velicer, W. F. (1988). Relation of sample size to the stability of component patterns. Psychological Bulletin, 103, 265-275.
20. Hammond, K. R. (1996). Human judgment and social policy: Irreducible uncertainty, inevitable error, unavoidable injustice. New York: Oxford University Press.
21. Hathaway, S. R., & McKinley, J. C. (1943). The Minnesota Multiphasic Personality Inventory. Minneapolis: University of Minnesota Press.
22. Jako, R. A., & Murphy, K. R. (1990). Distributional ratings, judgment decomposition, and their impact on interrater agreement and rating accuracy. Journal of Applied Psychology, 75, 500-505.
23. Kolar, D. W., Funder, D. C., & Colvin, C. R. (1996). Comparing the accuracy of personality judgments by the self and knowledgeable others. Journal of Personality, 64, 311-337.
24. Koran, L. M. (1975a). The reliability of clinical methods, data and judgments (Part 1). New England Journal of Medicine, 293, 642-646.
25. Koran, L. M. (1975b). The reliability of clinical methods, data and judgments (Part 2). New England Journal of Medicine, 293, 695-701.
26. Little, K. B., & Shneidman, E. S. (1959). Congruencies among interpretations of psychological test and anamnestic data. Psychological Monographs. 73(6, Whole No. 476).
27. McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30-46.
28. Meehl, P. E. (1996). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. Northvale, NJ: Aronson. (Original work published 1954)
29. Meyer, G. J. (1996). Construct validation of scales derived from the Rorschach method: A review of issues and introduction to the Rorschach Rating Scale. Journal of Personality Assessment, 67, 598-628.
30. Meyer, G. J. (1997). Assessing reliability: Critical corrections for a critical examination of the Rorschach Comprehensive System. Psychological Assessment, 9, 480-489.
31. Meyer, G. J. (1999). Simple procedures to estimate chance agreement and kappa for the interrater reliability of response segments using the Rorschach Comprehensive System. Journal of Personality Assessment, 72, 230-255.
32. Meyer, G. J. (2004). The reliability and validity of the Rorschach and TAT compared to other psychological and medical procedures: An analysis of systematically gathered evidence. In M. Hersen (Ed. in Chief) & M. Hilsenroth & D. Segal (Vol. Eds.), Comprehensive handbook of psychological assessment: Vol. 2. Personality assessment (pp. 315-342). Hoboken, NJ: Wiley.
33. Meyer, G. J., Bates, M., & Gacono, C. (1999). The Rorschach Rating Scale: Item adequacy, scale development, and relations with the Big Five model of personality. Journal of Personality Assessment, 73, 199-244.
34. Meyer, G. J., Baxter, D., Exner, J. E., Jr., Fowler, J. C., Hilsenroth, M. J., Piers, C. C., et al. (2002). An examination of interrater reliability for scoring the Rorschach Comprehensive System in eight data sets. Journal of Personality Assessment, 78, 219-274.
35. Meyer, G. J., Finn, S. E., Eyde, L., Kay, G. G., Moreland, K. L., Dies, R. R., et al. (2001). Psychological testing and psychological assessment: A review of evidence and issues. American Psychologist, 56, 128-165.
36. Mihura, J. L., Meyer, G. J., Bel-Bahar, T., & Gunderson, J. (2003). Correspondence among observer ratings of Rorschach, Big Five model, and DSM-IV personality disorder constructs. Journal of Personality Assessment, 81, 20-39.
37. Mihura, J. L., Meyer, G. J., & Wright, A. (2002, March). Review and meta-analysis of the interpretive reliability of psychological tests. Paper presented at the annual meeting of the Society for Personality Assessment, San Antonio, TX.
38. Millon, T. (1994). MCMI-III manual. Minneapolis: National Computer Systems.
39. Morey, L. C. (1991). The Personality Assessment Inventory: Professional manual. Odessa, FL: Psychological Assessment Resources.
40. Murphy, K. R., & DeShon, R. (2000). Interrater correlations do not estimate the reliability of job performance ratings. Personnel Psychology, 53, 873-900.
41. Murray, H. E. (1943). Thematic Apperception Test manual. Cambridge, MA: Harvard University Press.
42. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
43. Ozer, D. J. (1993). The Q-sort method and the study of personality development. In D. C. Funder, R. D. Parke, C. Tomlinson-Keasey, & K. Widaman (Eds.), Studying lives through time: Personality and development (pp. 147-168). Washington, DC: American Psychological Association.
44. Poythress, N. G., & Blaney, P. H. (1978). The validity of MMPI interpretations based on the Minimult and the FAM. Journal of Personality Assessment, 42, 143-147.
45. Schmidt, F. L., Viswesvaran, C., & Ones, D. S. (2000). Reliability is not validity and validity is not reliability. Personnel Psychology, 53, 901-912.
46. Sines, L. K., & Silver, R. J. (1963). An index of psychopathology (Ip) derived from clinicians' judgments of MMPI profiles. Journal of Clinical Psychology, 19, 324-326.
47. Spengler, P. M., Strohmer, D. C., Dixon, D. N., & Shivy, V. A. (1995). A scientist-practitioner model of psychological assessment: Implications for training, practice and research. Counseling Psychologist, 23, 506-534.
48. Streiner, D. L. (2003). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80, 99-103.
49. Viglione, D. J., Jr. (1999). A Review of recent research addressing the utility of the Rorschach. Psychological Assessment, 11, 251-265.
50. Viglione, D. J., Jr., & Hilsenroth, M. J. (2001). The Rorschach: Facts, fictions, and future. Psychological Assessment, 13, 452-471.
51. Viglione, D. J., & Taylor, N. (2003). Empirical support for interrater reliability of Rorschach Comprehensive System coding. Journal of Clinical Psychology, 59, 111-121.
52. Westen, D., & Shedler, J. (1999). Revising and assessing Axis II, Part I: Developing a clinically and empirically valid assessment method. American Journal of Psychiatry, 156, 258-272.
53. Westen, D., & Weinberger, J. (2004). When clinical description becomes statistical prediction. American Psychologist, 59, 595-613.
54. Wilkinson, L., & the American Psychological Association Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.
55. Zwick, W. R., & Velicer, W. F. (1986). Comparison of five rules for determining the number of components to retain. Psychological Bulletin, 99, 432-442.