Temporal stability in the Rorschach method: A meta-analytic review

Journal of Personality Assessment

Volumn 80, Issue 3, 2003, Pages 272-293

  Grønnerød, Cato ^a,b  

^a UNIVERSITY OF OSLO   (Norway)

^b Department of Psychology ^*  (Finland)

Author keywords

[No Author keywords available]

Indexed keywords

CORRELATION ANALYSIS; HUMAN; META ANALYSIS; METHODOLOGY; PERSONALITY TEST; PREDICTION; REVIEW; RORSCHACH METHOD; SCORING SYSTEM; VALIDATION PROCESS;

EID: 0038648478     PISSN: 00223891     EISSN: None     Source Type: Journal    
DOI: 10.1207/S15327752JPA8003_06     Document Type: Review

References (129)

1. Acklin, M. W. (1999). Behavioral science foundations of the Rorschach Test: Research and clinical applications. Assessment, 6, 319-326.
2. Acklin, M. W., McDowell, C. J., Verschell, M. S., & Chan, D. (2000). Interobserver agreement, intraobserver reliability, and the Rorschach Comprehensive System. Journal of Personality Assessment, 74, 15-47.
3. Adair, H. E., & Wagner, E. E. (1992). Stability of unusual verbalizations on the Rorschach for outpatients with schizophrenia. Journal of Clinical Psychology, 48, 250-256.
4. Alho, J. (2002). Combining skewness statistics. Unpublished manuscript, University of Joensuu, Finland.
5. Allen, R. M., Manne, S. H., & Stiff, M. (1952). The influence of color on the consistency of responses in the Rorschach test. Journal of Clinical Psychology, 8, 97-98.
6. *Ames, L. B. (1966). Changes in Rorschach response throughout the human life span. Genetic Psychology Monographs, 74, 89-125.
7. Beck, S. J. (1950). Rorschach's Test. Vol. 1. Basic processes (2nd ed.). New York: Grune & Stratton.
8. Bornstein, R. F. (1999). Criterion validity of objective and projective dependency tests: A meta-analytic assessment of behavioral prediction. Psychological Assessment, 11, 48-57.
9. *Bornstein, R. F., Rossner, S. C., & Hill, E. L. (1994). Retest reliability of scores on objective and projective measures of dependency: Relationship to life events and intertest interval. Journal of Personality Assessment, 62, 398-415.
10. Corel. (2000). Quattro Pro (Version 9.0 for Windows) [Computer software]. Ottawa, Ontario, Canada: Author.
11. *Daston, P. G., & McConnell, O. L. (1962). Stability of Rorschach penetration and barrier scores over time. Journal of Consulting Psychology, 26, 104.
12. *Dudek, S. Z. (1970). Effects of different types of therapy on the personality as a whole. Journal of Nervous and Mental Disease, 150, 329-345.
13. *Dudek, S. Z., & Kolivakis, T. (1983). Stability of intellect and personality in schizophrenia. Canadian Journal of Psychiatry, 28, 2-7.
14. Dunlap, W. P., Burke, M. J., & Greer, T. (1995). The effect of skew on the magnitude of product-moment correlations. Journal of General Psychology, 122, 365-377.
15. Dunlap, W. P., Chen, R., & Greet, T. (1994). Skew reduces test-retest reliability. Journal of Applied Psychology, 79, 310-313.
16. Endicott, N. A., & Endicott, J. (1963). Objective measures of somatic preoccupation. Journal of Nervous and Mental Disease, 137, 427-437.
17. Erginel, A. (1972). On the test-retest reliability of the Rorschach. Journal of Personality Assessment, 36, 203-212.
18. *Erstad, D. H. (1996). An investigation of older adults' less frequent human movement and color responses on the Rorschach. Unpublished doctoral dissertation, Marquette University, Milwaukee, Wisconsin.
19. Exner, J. E., Jr. (1973). The Self-Focus Sentence Completion: A study of egocentricity. Journal of Personality Assessment, 37, 437-455.
20. Exner, J. E., Jr. (1974). The Rorschach: A comprehensive system: Vol. 1. Basic foundations. New York: Wiley.
21. *Exner, J. E., Jr. (1978). The Rorschach: A comprehensive system, Vol. 2: Current research and advanced interpretation. New York: Wiley.
22. Exner, J. E., Jr. (1980). But it's only an inkblot. Journal of Personality Assessment, 44, 563-577.
23. Exner, J. E., Jr. (1983). Rorschach Assessment. In I. B. Weiner (Ed.), Clinical methods in psychology (pp. 58-99). New York: Wiley.
24. *Exner, J. E., Jr. (1986). The Rorschach: A comprehensive system: Vol I. Basic foundations (2nd ed.). New York: Wiley.
25. Exner, J. E., Jr. (1988). Problems with brief Rorschach protocols. Journal of Personality Assessment, 52, 640-647.
26. Exner, J. E., Jr. (1991). The Rorschach: A comprehensive system: Vol. 2. Interpretation (2nd ed.). New York: Wiley.
27. Exner, J. E., Jr. (1992). But it's only an inkblot. In C. D. S. Edwin I. Megargee (Ed.), Personality assessment in America: A retrospective on the occasion of the fiftieth anniversary of the Society for Personality Assessment (pp. 126-138): Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
28. Exner, J. E., Jr. (1993). The Rorschach: A comprehensive system: Vol. 1. Basic foundations (3rd ed.). New York: Wiley.
29. Exner, J. E., Jr. (1997). The future of Rorschach in personality assessment. Journal of Personality Assessment, 68, 37-46.
30. Exner, J. E., Jr. (1999). The Rorschach: Measurement concepts and issues of validity. In S. E. Embretson & S. L. Hershberger (Eds.), The new rules of measurement: What every psychologist and educator should know (pp. 159-183). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
31. Exner, J. E., Jr., & Andronikof-Sanglade, A. (1992). Rorschach changes following brief and short-term therapy. Journal of Personality Assessment, 59, 59-71.
32. *Exner, J. E., Jr., Armbruster, G. L., & Viglione, D. J. (1978). The temporal stability of some Rorschach features. Journal of Personality Assessment, 42, 474-482.
33. Exner, J. E., Jr., & Murillo, L. G. (1977). A long term followup of schizophrenics treated with regressive ECT. Diseases of the Nervous System, 38, 162-168.
34. Exner, J. E., Jr., Thomas, E. A., & Mason, B. J. (1985). Children's Rorschachs: Description and prediction. Journal of Personality Assessment, 49, 13-20.
35. Exner, J. E., Jr., & Weiner, I. B. (1982). The Rorschach: A comprehensive system: Vol. 3. Assessment of children and adolescents. New York: Wiley.
36. Exner, J. E., Jr., & Weiner, I. B. (1995). The Rorschach: A comprehensive system: Vol. 3. Assessment of children and adolescents (2nd ed.). New York: Wiley.
37. *Fabrikant, B. (1953). Perceptual control on the Rorschach test. Journal of Clinical Psychology, 9, 396-397.
38. Fabrikant, B. (1954). Rigidity and flexibility on the Rorschach. Journal of Clinical Psychology, 10, 255-258.
39. Fabrikant, B. (1955). Suggestibility and the Rorschach. Journal of Clinical Psychology, 11, 309-310.
40. Field, A. P. (2001). Meta-analysis of correlation coefficients: A Monte Carlo comparison of fixed- and random-effects methods. Psychological Methods, 6, 161-180.
41. *Fisher, R. (1958). The effect of a disturbing situation upon the stability of various projective tests. Psychological Monographs, 72(14, Whole No. 467).
42. Fisher, S., & Cleveland, S. E. (1958). Body image and personality. Princeton, NJ: Van Nostrand.
43. Fosberg, I. A. (1941). An experimental study of the reliability of the Rorschach psychodiagnostic technique. Rorschach Research Exchange, 5, 72-84.
44. Friedman, H. (1953). Perceptual regression in schizophrenia. An hypothesis suggested by the use of the Rorschach test. Journal of Projective Techniques, 17, 171-185.
45. Garb, H. N. (1999). Call for a moratorium on the use of the Rorschach Inkblot Test in clinical and forensic settings. Assessment, 6, 313-317.
46. Goldfried, M. R., Stricker, G., & Weiner, I. B. (1971). Rorschach handbook of clinical and research applications. Englewood Cliffs, NJ: Prentice-Hall.
47. Griffith, R. M. (1951). Test-retest similarities of the Rorschachs of patients without retention, Korsakoff. Journal of Projective Techniques, 15, 516-525.
48. Grønnerød, C. (1999). Rorschach interrater agreement estimates: An empirical evaluation. Scandinavian Journal of Psychology, 40, 115-120.
49. Grønnerød, C. (2002). Rorschach assessment of changes following psychotherapy: A meta-analytic review. Manuscript submitted for publication.
50. Grove, W. M., Andreassen, N. C., McDonald-Scott, P., Keller, M. B., & Shapiro, R. W. (1981). Reliability studies of psychiatric diagnosis. Archives of General Psychiatry, 38, 408-413.
51. *Haimovitz, N., & Haimovitz, M. (1952). Personality changes in client-centered psychotherapy. In W. Wolff & J. A. Precker (Eds.), Success in psychotherapy (pp. 63-93). New York: Grune & Stratton.
52. *Haller, N., & Exner, J. E., Jr. (1985). The reliability of Rorschach variables for inpatients presenting symptoms of depression and/or helplessness. Journal of Personality Assessment, 49, 516-521.
53. *Hamlin, R. M., & Albee, G. W. (1948). Muench's tests before and after nondirective therapy: A control group for his subjects. Journal of Consulting Psychology, 12, 412-416.
54. Hedges, L. V., & Vevea, J. L. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3, 486-504.
55. Hiller, J. B., Rosenthal, R., Bornstein, R. F., Berry, D. T. R., & Brunell-Neuleib, S. (1999). A comparative meta-analysis of Rorschach and MMPI validity. Psychological Assessment, 11, 278-296.
56. Hoaglin, D. C. (1983). Letter values: A set of selected ordered statistics. In D. C. Hoaglin, F. Mosteller, & J. W. Tuckey (Eds.), Understanding robust and exploratory data analysis (pp. 33-43). New York: Wiley.
57. Holt, R. R. (1977). A method for assessing primary process and their control in Rorschach responses. In M. Rickers-Ovsiankina (Ed.), Rorschach psychology. New York: Krieger.
58. *Holzberg, J. D., & Wexler, M. (1950). The predictability of schizophrenic performance on the Rorschach test. Journal of Consulting Psychology, 14, 395-399.
59. Hunsley, J., & Bailey, J. M. (1999). The clinical utility of the Rorschach: Unfulfilled promises and an uncertain future. Psychological Assessment, 11, 266-277.
60. Hunsley, J., & Di Giulio, G. (2001). Norms, norming, and clinical assessment. Clinical Psychology: Science and Practice, 8, 378-382.
61. Janson, H., & Erdberg, P. (1998). Recoding Rorschachs from a longitudinal study into the Comprehensive System: Intercoder agreement and temporal stability. Unpublished report, Stocholm University, Stockholm, Sweden.
62. Janson, H., & Stattin, H. (1994). Recurring objects in Rorschach records over a period of about 30 years: How often do people report the same thing again? In I. B. Weiner (Ed.), Rorschachiana 19: Yearbook of the International Rorschach Society (pp. 156-170). Göttingen, Germany: Hogrefe & Huber.
63. Jensen, A. R. (1959). The reliability of projective techniques: Review of the literature. Acta Psychologica, Amsterdam, 16, 108-136.
64. Kaplan, B., & Berger, S. (1956). Increments and consistency of performance in four repeated Rorschach administrations. Journal of Projective Techniques, 20, 304.
65. Kelley, D. M., Margulies, H., & Barrera, S. (1941). The stability of the Rorschach method as demonstrated in electric convulsive therapy cases. Rorschach Research Exchange, 5, 35-43.
66. Kikuchi, T., Kitamura, S., Sato, I., & Oyama, M. (1962). Rorschach performance in alcoholic intoxication. Part II. Tohoku Psychologica Folia, 21, 19-46.
67. Klopfer, B., Ainsworth, M., Klopfer, W., & Holt, R. (1954). Developments in the Rorschach Technique: Vol. I. Technique and theory. New York: World Book.
68. Lambert, M. J., & Hill, C. E. (1994). Assessing psychotherapy outcome and process. In A. E. Bergin & S. L. Garfield (Eds.), Handbook of psychotherapy and behavior change (4th ed.). New York: Wiley.
69. *Leavitt, F., & Labott, S. M. (1997). Criterion-related validity of Rorschach analogues of dissociation. Psychological Assessment, 9, 244-249.
70. Lerner, P. M. (1975). Handbook of Rorschach scales. New York: International Universities Press.
71. Lindberg, D. (1981). A controlled study of 5 years treatment with psychotherapy in combination with depot neuroleptics in schizophrenia. Acta Psychiatrica Scandinavica: Supplementum, 289, 55-66.
72. Loveland, N. T., & Singer, M. T. (1959). Projective test assessment of the effects of sleep deprivation. Journal of Projective Techniques, 23, 323-334.
73. Masling, J. M., Rabie, L., & Blondheim, S. H. (1967). Obesity, level of aspiration, and Rorschach and TAT measures of oral dependence. Journal of Consulting psychology, 31, 233-239.
74. McDowell, C., & Acklin, M. W. (1996). Standardizing procedures for calculating Rorschach interrater reliability: Conceptual and empirical foundations. Journal of Personality Assessment, 66, 308-320.
75. Meltzoff, J., & Kornreich, M. (1970). Research in psychotherapy. New York: Atherton.
76. Meyer, G. J. (1997a). Assessing reliability: Critical corrections for a critical examination of the Rorschach Comprehensive System. Psychological Assessment, 9, 480-489.
77. Meyer, G. J. (1997b). Thinking clearly about reliability. More critical corrections regarding the Rorschach Comprehensive System. Psychological Assessment, 9, 495-498.
78. Meyer, G. J. (1999). Simple procedures to estimate chance agreement and kappa for the interrater reliability of response segments using the Rorschach Comprehensive System. Journal of Personality Assessment, 72, 230-255.
79. Meyer, G. J. (2000). On the science of Rorschach research. Journal of Personality Assessment, 75, 46-81.
80. Meyer, G. J. (2001). Evidence to correct misperceptions about Rorschach norms. Clinical Psychology: Science and Practice, 8, 389-396.
81. Meyer, G. J., & Handler, L. (1997). The ability of the Rorschach to predict subsequent outcome: A meta-analysis of the Rorschach Prognostic Rating Scale. Journal of Personality Assessment, 69, 1-38.
82. Meyer, G. J., Hilsenroth, M. J., Baxter, D., Exner, J. E., Jr., Fowler, J. C., Piers, C. C., et al. (2002). An examination of interrater reliability for scoring the Rorschach Comprehensive System in eight data sets. Journal of Personality Assessment, 78, 219-274.
83. *Mills, E. S. (1955). Personality adjustment and the study of abnormal psychology. Journal of Applied Psychology, 39, 358-361.
84. Muench, G. A. (1947). An evaluation of non-directive psychotherapy. International Review of Applied Psychology, 13, 17-38.
85. Orlinsky, D. E., Grawe, K., & Parks, B. K. (1994). Process and outcome in psychotherapy: Noch einmal [once again]. In A. E. Bergin & S. L. Garfield (Eds.), Handbook of psychotherapy and behavior change (4th ed., pp. 270-376). New York: Wiley.
86. *Perry, W., McDougall, A., & Viglione, D. (1995). A five-year follow-up on the temporal stability of the Ego Impairment Index. Journal of Personality Assessment, 64, 112-118.
87. Piotrowski, Z. A. (1957). Perceptanalysis. New York: Macmillan.
88. Raines, G. N., & Broomhead, E. (1945). Rorschach studies on combat fatigue. Diseases of the Nervous System, 6, 250-256.
89. Rapaport, D. (1949). Diagnostic psychological testing. New York: International Universities Press.
90. Rickers-Ovsiankina, M. A. (Ed.). (1960). Rorschach psychology. New York: Wiley.
91. Roberts, B. W., & DelVecchio, W. F. (2000). The rank-order consistency of personality traits from childhood to old age: A quantitative review of longitudinal studies. Psychological Bulletin, 126, 3-25.
92. Rorschach, H. (1921/1942). Psychodiagnostics (P. Lemkau, Trans., 9th ed.). Bern, Germany: Verlag. (Original work published 1921)
93. Rosenthal, R. (1991a). Meta-analytic procedures in social research (Rev. ed.). Baltimore: Sage.
94. Rosenthal, R. (1991b). Quality-weighting of studies in meta-analytic research. Psychotherapy Research, 1, 25-28.
95. Schachtel, E. G. (1966). Experiential foundations of Rorschach's test. New York: Basic.
96. Schafer, R. (1954). Psychoanalytic interpretation in Rorschach testing. New York: Grune & Stratton.
97. Schimek, J. G. (1968a). Cognitive style and defenses: A longitudinal study of intellectualization and field independence. Journal of Abnormal Psychology, 73, 575-580.
98. Schimek, J. G. (1968b). A note on the long-range stability of selected Rorschach scores. Journal of Projective Techniques and Personality Assessment, 32, 63-65.
99. Schimek, J. G. (1974). Some developmental aspects of primary process manifestations in the Rorschach. Journal of Personality Assessment, 38, 226-229.
100. Seim, S. (1940). Intelligensfaktorene i Rorschachs formtydningsforsøk [Factors of intelligence in Rorschach's Form Apprehension test]. Unpublished master's thesis, University of Oslo, Norway.
101. *Seim, S. (1989). Teenagers become adult and elderly: intelligence and personality from 13 to 30 to 60 years (Tech. Rep. No.5). Oslo: Norwegian Institute of Gerontology.
102. *Seim, S. (1998). Teenagers become pensioners (Tech. Rep. No. 2). Oslo: NOVA-Norwegian Social Research.
103. Shadish, W. R., Navarro, A. M., Matt, G. E., & Phillips, G. (2000). The effects of psychological therapies under clinically representative conditions: A meta-analysis. Psychological Bulletin, 126, 512-529.
104. Sharma, V. M., Baskaran, A. S., & Malhotra, M. S. (1974). Emotional composure under altitude stress. Psychologia: An International Journal of Psychology in the Orient, 17, 103-107.
105. Singer, M. T., & Wynne, L. C. (1966). Principles for scoring communication defects and deviances in parents of schizophrenics. Psychiatry, 29, 260-288.
106. SPSS. (1997). Statistical Package for the Social Sciences (Version 8.0.0 for Windows) [Computer software]. Chicago: Author.
107. Steel, P. D., & Kammeyer-Mueller, J. D. (2002). Comparing meta-analytic moderator estimation techniques under realistic conditions. Journal of Applied Psychology, 87, 96-111.
108. Stricker, G., & Gold, J. R. (1999). The Rorschach: Toward a nomothetically based, idiographically applicable configurational model. Psychological Assessment, 11, 240-250.
109. Sudik, N. A. (1983). The reliability of Rorschach depressive indices in major depressive disorder. Unpublished doctoral dissertation, United States International University, San Diego, CA.
110. Sæther, L., & Berg, M. (1999). "Inkblots again?!" Stability of Rorschach scores: An artefact of memory? Unpublished master's thesis, University of Tromsø, Norway.
111. Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58, 6-20.
112. Viglione, D. J. (1995). Basic considerations regarding data analysis. In J. E. Exner, Jr. (Ed.), Issues and methods in Rorschach research (pp. 195-226). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
113. Viglione, D. J. (1999). A review of recent research addressing the utility of the Rorschach. Psychological Assessment, 11, 251-265.
114. Wagner, E. E., & Daubney, J. H. (1976). Test-retest reliability of the Rorschach on neurologically impaired subjects. Journal of Personality Assessment, 40, 579-581.
115. *Wahlberg, K. E., Wynne, L. C., Keskitalo, P., Nieminen, P., Moring, J., Laeksy, K., et al. (2001). Long-term stability of communication deviance. Journal of Abnormal Psychology, 110, 443-448.
116. *Warren, V. M. (1974). Retest reliability of the Rorschach. Unpublished doctoral dissertation, University of California, Los Angeles.
117. Weiner, I. B. (1996). Some Observations on the Validity of the Rorschach inkblot method. Psychological Assessment, 8, 206-213.
118. Weiner, I. B. (1999a). Contemporary perspectives on Rorschach assessment. European Journal of Psychological Assessment, 15, 78-86.
119. Weiner, I. B. (1999b). What the Rorschach can do for you: Incremental validity in clinical applications. Assessment, 6, 327-339.
120. Weiner, I. B. (2000). Using the Rorschach properly in practice and research. Journal of Clinical Psychology, 56, 435-438.
121. Weiner, I. B., & Exner, J. E., Jr. (1991). Rorschach changes in long-term and short-term psychotherapy. Journal of Personality Assessment, 56, 453-465.
122. Wimmers, D. (1971). The stability of projective interpretations. Dissertation Abstracts International, 32(3-B), 1865-1866.
123. Wittenborn, J. R., & Mettler, F. A. (1952). A Lack of Perceptual Control Score for the Rorschach test. Journal of Clinical Psychology, 7, 331-334.
124. Wood, J. M., & Lilienfeld, S. O. (1999). The Rorschach Inkblot Test: A case of overstatement? Assessment, 6, 341-351.
125. Wood, J. M., Lilienfeld, S. O., Garb, H. N., & Nezworski, M. T. (2000). The Rorschach test in clinical diagnosis: A critical review, with a backward look at Garfield (1947). Journal of Clinical Psychology, 56, 395-430.
126. Wood, J. M., Nezworski, M. T., Garb, H. N., & Lilienfeld, S. O. (2001a). The misperception of psychopathology: Problems with norms of the Comprehensive System for the Rorschach. Clinical Psychology: Science and Practice, 8, 350-373.
127. Wood, J. M., Nezworski, M. T., Garb, H. N., & Lilienfeld, S. O. (2001b). Problems with the norms of the Comprehensive System for the Rorschach: Methodological and conceptual considerations. Clinical Psychology: Science and Practice, 8, 397-402.
128. Wood, J. M., Nezworski, M. T., & Stejskal, W. J. (1996). The Comprehensive System for the Rorschach: A critical examination. Psychological Science, 7, 3-10.
129. Zulliger, H. (1941). Behn-Rorschach test. Bern, Germany: Verlag.