Using mixed-effects models in reliability generalization studies

Educational and Psychological Measurement

Volumn 63, Issue 1, 2003, Pages 75-95

  Beretvas, S Natasha ^a   Pastor, Dena A ^b  

^a UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^b NONE

Author keywords

Beck Depression Inventory; Mixed effects models; Reliability generalization studies

Indexed keywords

EID: 0037277310     PISSN: 00131644     EISSN: None     Source Type: Journal    
DOI: 10.1177/0013164402239318     Document Type: Article

References (110)

1. Andersen, S. M., & Schwartz, A. H. (1992). Intolerance of ambiguity and depression: A cognitive vulnerability factor linked to hopelessness. Social Cognition, 10, 271-298.
2. Anderson, O., Melin, L., Lindberg, P., & Scott, B. (1995). Dispositional optimism, dyspheria, health and coping with hearing impairment in elderly adults. Audiology, 34, 76-84.
3. Barrera, M., Jr., & Oliver, J. M. (1981). Sex-ratio and gender differences in depression in an unelected adult population. Journal of Clinical Psychology, 37, 570-574.
4. Beck, A. T. (1967). Depression: Causes and treatment. Philadelphia: University of Pennsylvania Press.
5. Beck, A. T., Guth, D., Steer, R. A., & Ball, R. (1997). Screening for major depression disorders in medical inpatients with the Beck Depression Inventory for primary care. Behavior Research and Therapy, 35, 785-791.
6. Beck, A. T., & Steer, R. A. (1984). Internal consistencies of the original and revised Beck Depression Inventory. Journal of Clinical Psychology, 40, 1365-1367.
7. Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Manual for the Beck Depression Inventory-II. San Antonio, TX: The Psychological Corporation.
8. Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4, 53-63.
9. Beck, J. T., & Strong, S. R. (1982). Stimulating therapeutic change with interpretations: A comparison of positive and negative connotations. Journal of Counseling Psychology, 29, 551-559.
10. Bosscher, R. J., Koning, H., & Van Meurs, R. (1986). Reliability and validity of the Beck Depression Inventory in a Dutch college population. Psychological Reports, 58, 696-798.
11. Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models. Newbury Park, CA: Sage.
12. Bryson, S. E., & Pilon, D. J. (1984). Sex differences in depression and the method of administering the Beck Depression Inventory. Journal of Clinical Psychology, 40, 529-534.
13. Byerly, F. G., & Carlson, W. A. (1982). Comparison among inpatients, outpatients, and normals on three self-report depression inventories. Journal of Clinical Psychology, 30, 797-804.
14. Campbell, I. M., Burgess, P. M., & Finch, S. J. (1984). A factorial analysis of BDI scores. Journal of Clinical Psychology, 40, 992-996.
15. Capraro, M. M., Capraro, R. M., & Henson, R. K. (2001). Measurement error of scores on the mathematics anxiety rating scale across studies. Educational and Psychological Measurement, 61, 373-386.
16. Carr, J. G., Gilroy, F. D., & Sherman, M. F. (1996). Silencing the self and depression among women. Psychology of Women Quarterly, 20, 375-392.
17. Caruso, J. C. (2000). Reliability generalization of the NEO personality scales. Educational and Psychological Measurement, 60, 236-254.
18. Caruso, J. C., Witkiewitz, K., Belcourt-Dittloff, A., & Gottlieb, J. D. (2001). Reliability of scores from the Eysenck personality questionnaire: A reliability generalization study. Educational and Psychological Measurement, 61, 675-689.
19. Cascardi, M., & O'Leary, L. D. (1992). Depressive symptomatology, self-esteem, and self-blame in battered women. Journal of Family Violence, 7, 249-259.
20. Catanzaro, S. J. (1993). Mood regulation expectancies, anxiety sensitivity, and emotional distress. Journal of Abnormal Psychology, 102, 327-330.
21. Chan, D. W. (1991). The Beck Depression Inventory: What difference does the Chinese version make? Psychological Assessment, 3, 616-622.
22. Cook, M. L., & Peterson, C. (1986). Depressive irrationality. Cognitive Therapy and Research, 10, 293-298.
23. Cox, B. J., Swinson, R. P., Kuch, K., & Reichman, J. T. (1993). Self-reported differentiation of anxiety and depression in an anxiety disorders sample. Psychological Assessment, 5, 484-486.
24. Dobson, K. S., & Breiter, H. J. (1983). Cognitive assessment of depression: Reliability and validity of three measures. Journal of Abnormal Psychology, 92, 107-109.
25. Elevin, B. E., Llabre, M. M., & Weiner, W. J. (1988). Parkinson's disease and depression: Psychometric properties of the Beck Depression Inventory. Journal of Neurology, Neurosurgery, and Psychiatry, 51, 1401-1404.
26. Ellis, T., Rudd, M. D., Rajab, M. H., & Wehrly, T. (1996). Cluster analysis of MCMT scores of suicidal psychiatric patients: Four personality profiles. Journal of Clinical Psychology, 52, 411-422.
27. Feather, N. T. (1983). Some correlates of attributional style: Depressive symptoms, self-esteem and protestant ethic values. Personality and Social Psychology Bulletin, 9, 125-135.
28. Feather, N. T. (1987). The rosy glow of self-esteem: Depression, masculinity, and causal attributions. Australian Journal of Psychology, 39, 25-41.
29. Feather, N. T., & Bond, M. J. (1983). Time structure and purposeful activity among employed and unemployed university graduates. Journal of Occupational Psychology, 56, 241-254.
30. Field, A. P. (2001). Meta-analysis of correlation coefficients: A Monte Carlo comparison of fixed- and random-effects methods. Psychological Methods, 6, 161-180.
31. Fisher, R. A. (1928). Statistical methods for research workers (2nd ed.). London: Oliver & Boyd.
32. Foelker, G. A., Jr., Shewchuk, R. M., & Niederehe, G. (1987). Confirmatory factor analysis of the short form of the Beck Depression Inventory in elderly community samples. Journal of Clinical Psychology. 43, 111-118.
33. Gallagher, D., Nies, G., & Thompson, L. W. ( 1982). Reliability of the Beck Depression Inventory with older adults. Journal of Consulting and Clinical Psychology, 50, 152-153.
34. Gatewood-Colwell, G., Kaczmarek, M., & Ames, M. H. (1989). Reliability of the Beck Depression Inventory for a White and Mexican-American gerontic population. Psychological Reports. 65, 1163-1166.
35. Giambra, L. M. (1977). Independent dimensions of depression: A factor analysis of three self-report measures. Journal of Clinical Psychology, 33, 928-935.
36. Golin, S., & Hartz, M. A. (1979). A factor analysis of the Beck Depression Inventory in a mildly depressed population. Journal of Clinical Psychology, 35, 322-325.
37. Gorenstein, C., & Pompeia, S. (1995). Scores of Brazilian University students on the Beck Depression and the State-Trait Anxiety Inventories. Psychological Reports, 77, 635-641.
38. Gotlib, I. H. (1984). Depression and general psychopathology in university students. Journal of Abnormal Psychology, 93, 19-30.
39. Gould, J. (1982). A psychometric investigation of the standard and short form Beck Depression Inventory. Psychological Reports, 51, 1167-1170.
40. Hankin, B. L., Roberts, J., & Gotlib, I. H. (1997). Elevated self-standards and emotional distress during adolescence: Emotional specificity and gender differences. Cognitive Therapy and Research, 21, 663-679.
41. Harwell, M. (1997) An empirical study of Hedges' homogeneity test. Psychological Methods, 2, 219-231.
42. Hatzenbuehler, L. C., Parpal, M., & Matthews, L. (1983). Classifying college students as depressed or nondepressed using the Beck Depression Inventory: An empirical analysis. Journal of Consulting and Clinical Psychology, 51, 360-366.
43. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press.
44. Hedges, L. V., & Vevea, J. L. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3, 486-504.
45. Henson, R. K., Kogan, L. R., & Vacha-Haase, T. (2001). A reliability generalization study of the teacher efficacy scale and related instruments. Educational and Psychological Measurement, 61, 404-420.
46. Hjelle, L. A., & Bernard, M. (1994). Private self-consciousness and the retest reliability of self-reports. Journal of Research in Personality, 28, 52-67.
47. Hojat, M., & Schapurian, R. (1986). Psychometric properties of a Persian version of the short form of the Beck Depression Inventory for Iranian college students. Psychological Reports, 59, 331-338.
48. Joe, G. W., Knezek, L., Watson, D., & Simpson, D. D. (1991). Depression and decision making among intravenous drug users. Psychological Reports, 68, 339-347.
49. Johnson, J. G., Crofton, A., & Feinstein, S. B. (1996). Enhancing attributional style and positive life events increased hopefulness among depressed psychiatric inpatients. Motivation and Emotion, 20, 285-297.
50. Kalaian, H. A., & Raudenbush, S. W. (1996). A multivariate mixed linear model for meta-analysis. Psychological Methods, 1, 227-235.
51. Kirkham, M. A., Schinke, S. P., Schilling, R. F. II, Meltzer, N. J., & Norelius, K. L. (1986). Cognitive, behavioral skills, social supports, and child abuse potential among mothers of handicapped children. Journal of Family Violence, 1, 235-245.
52. Knight, R. G. (1984). Some general population norms for the short form Beck Depression Inventory. Journal of Clinical Psychology, 40, 751-753.
53. Kreft, I., & de Leeuw, J. (1998). Introducing multilevel modeling. London: Sage.
54. Kwon, S. M., & Dei, T.P.S. (1992). Differential causal roles of dysfunctional attitudes and automatic thought in depression. Cognitive Therapy and Research, 16, 309-328.
55. Leahy, J. M. (1992). Validity and reliability of the Beck Depression Inventory-Short Form in a group of adult bereaved females. Journal of Clinical Psychology, 48, 64-68.
56. Leahy, J. M. (1993). A comparison of depression in women bereaved of a spouse, child or a parent. Omega, 26, 207-217.
57. Lee, A. M., & Lee, S. (1996). Disordered eating and its psychosocial correlates among Chinese adolescent females in Hong Kong. International Journal of Eating Disorders, 20, 177-183.
58. Leigh, I. W., Robins, C. J., & Welkowitz, J. (1988). Modification of the Beck Depression Inventory for use with a deaf population. Journal of Clinical Psychology, 44, 728-732.
59. Marcovitz, R. J., & Smith, J. E. (1983). Patients' perceptions of curative factors in short-term group psychotherapy. International Journal of Group Psychotherapy, 33, 21-39.
60. Matsumi, J., & Kameoka, V. A. (1986). Reliabilities and concurrent validities of popular self-report measures of depression, anxiety, and social desirability. Journal of Consulting and Clinical Psychology, 54, 328-333.
61. McCabe, K. M. (1997). Sex differences in the long term effects of divorce on children: Depression and heterosexual relationship difficulties in the young adult years. Journal of Divorce and Remarriage, 27, 123-135.
62. Michelson, L., & Mavissakalian, M. (1983). Temporal stability of self-report measures in agoraphobia research. Behavior Research and Therapy, 21, 695-698.
63. Moilanen, D. L. (1995). Validity of Beck's cognitive therapy of depression with nonreferred adolescents. Journal of Counseling and Development, 73, 438-442.
64. Oliver, J. M., & Baumgart, E. P. (1995). The dysfunctional attitude scale: Psychometric properties and relations to depression in an unselected adult population. Cognitive Therapy and Research, 9, 161-167.
65. Oliver, J. M., & Burkham, R. (1979). Depression in university students: Duration, relation to calendar time, prevalence, and demographic correlates. Journal of Abnormal Psychology, 88, 667-670.
66. Oliver, J. M., & Simmons, M. E. (1985). Affective disorders and depression as measures by the diagnostic interview schedule and the Beck Depression Inventory in an unselected adult population. Journal of Clinical Psychology, 41, 469-477.
67. Palinkas, L. A., Wingard, P. L., & Barrett-Connor, E. (1996). Depressive symptoms in over-weight and obese older adults: A test of the "jolly fat" hypothesis. Journal of Psychosomatic Research, 40, 59-66.
68. Peterson, C. (1979). Uncontrollability and self-blame in depression: Investigation of the paradox in a college population. Journal of Abnormal Psychology, 88, 620-624.
69. Peterson, C., & Avila, M.E.D. (1995). Optimistic explanation style and the perception of health problems. Journal of Clinical Psychology, 51, 128-132.
70. Peterson, C., & Barrett, L. (1987). Explanatory style and academic performance among university freshman. Journal of Personality and Social Psychology, 53, 603-607.
71. Peterson, C., Rosenbaum, A. C., & Conn, M. K. (1985). Depression mood reactions to breaking up: Testing the learned helplessness model of depression. Journal of Social and Clinical Psychology, 3, 161-169.
72. Pulos, S. (1996). Validity of the Beck Depression Inventory with eating disorder patients. Educational and Psychological Measurement, 56, 139-141.
73. Radnitz, C. L., et al. (1997). Use of the Beck Depression Inventory in veterans with spinal cord injury. Rehabilitation Psychology, 42, 93-101.
74. Rapp, S. R., Walsh, D. A., Parisi, S., & Wallace, C. (1988). Detecting depression in elderly medical inpatients. Journal of Consulting and Clinical Psychology, 56, 509-513.
75. Raudenbush, S. W. (1988). Educational applications of hierarchical linear models: A review. Journal of Educational Statistics, 13, 85-116.
76. Raudenbush, S. W., & Bryk, A. S. (1985). Empirical Bayes meta-analysis. Journal of Educational Statistics, 10, 75-98.
77. Rees, A., Hardy, G. E., & Barkham, M. (1997). Covariance in the measurement of depression/anxiety and three cluster C personality disorders (avoidant, dependent, obsessive-compulsive).Journal of Affective Disorders, 45, 143-153.
78. Reynolds, W. M., & Gould, J. W. (1981). A psychometric investigation of the standard and short form of the Beck Depression Inventory. Journal of Consulting and Clinical Psychology, 49, 306-307.
79. Rosenthal, R. (1979). The "file drawer problem" and tolerance for null results. Psychological Bulletin, 86, 1165-1168.
80. Rosenthal, R. (1985). Parametric measures of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 231-244). San Diego, CA: Academic Press.
81. Schaefer, A., et al. (1985). Comparison of the validities of the Beck, Zung, and MMPI Depression Scales. Journal of Consulting and Clinical Psychology, 53, 415-418.
82. Schmidt, F. L., & Hunter, J. E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62, 529-540.
83. Schotte, C.K.W., Maes, M., Cluydts, R., DeDoncker, D., & Cosyns, P. (1997). Construct validity of the Beck Depression Inventory in a depressive population. Journal of Affective Disorders, 46, 115-125.
84. Scogin, F., Beutler, L., Corbishley, A., & Hamblin, D. (1988). Reliability and validity of the short form Beck Depression Inventory with older adults. Journal of Clinical Psychology, 44, 853-857.
85. Shek, D.T.L. (1990). Reliability and factorial structure of the Chinese version of the Beck Depression Inventory. Journal of Clinical Psychology, 46, 35-43.
86. Somoza, E., Steer, R. A., Beck, A. T., & Clark, D. A. (1994). Differentiating major depression and panic disorders by self-report and clinical rating scales: ROC analysis and information theory. Behavior Research and Therapy, 32, 771-782.
87. Steer, R. A., Beck, A. T., Riskind, J. H., & Brown, G. (1987). Relationships between the Beck Depression Inventory and the Hamilton Psychiatric Rating Scale for depression in depressed outpatients. Journal of Psychopathology and Behavioral Assessment, 9, 327-339.
88. Steer, R. A., & Clark, D. A. (1997). Psychometric characteristics of the Beck Depression Inventory-II with college students. Measurement and Evaluation in Counseling and Development, 30, 128-136.
89. Steer, R. A., McElroy, M. G., & Beck, A. T. (1982). Structure of depression in alcoholic men: A partial replication. Psychological Reports, 50, 723-728.
90. Steer, R. A., Shaw, B. F., Beck, A. T., & Fine, E. W. (1977). Structure of depression in Black alcoholic men. Psychological Reports, 41, 1235-1241.
91. Strober, M., Green, J., & Carlson, G. (1981). Utility of the Beck Depression Inventory with psychiatrically hospitalized adolescents. Journal of Consulting and Clinical Psychology, 49, 482-483.
92. Subkoviak, M., et al. (1995). Measuring interpersonal forgiveness in late adolescence and middle adulthood. Journal of Adolescence, 18, 641-655.
93. Tanaka, J. S., & Huba, G. J. (1987). Assessing the stability of depression in college students. Multivariate Behavioral Research, 22, 5-19.
94. Tangney, J. P., Wagner, P., & Granizow, R. (1992). Proneness to shame, proneness to guilt, and psychopathology. Journal of Abnormal Psychology, 101, 469-478.
95. Tashakkori, A., Barefoot, J., & Mehryar, A. H. (1989). What does the Beck Depression Inventory measure in college students? Evidence from a non-Western culture. Journal of Clinical Psychology, 45, 595-602.
96. Teri, L. (1982). The use of the Beck Depression Inventory with adolescents. Journal of Abnormal Child Psychology, 10, 277-284.
97. Thompson, B., & Vacha-Haase, T. (2000). Psychometrics is datametrics: The test is not reliable. Educational and Psychological Measurement, 60, 174-195.
98. Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58, 6-20.
99. Vacha-Haase, T., Kogan, L. R., Tani, C. R., & Woodall, R. A. (2001). Reliability generalization: Exploring variation of reliability coefficients of MMPI clinical scale scores. Educational and Psychological Measurement, 61, 45-59.
100. Viswesvaran, C., & Ones, D. S. (2000). Measurement error in "big five factors" personality assessment: Reliability generalization across studies and measures. Educational and Psychological Measurement, 60, 224-235.
101. Vrendenburg, K., Krames, L., & Flelt, G. L. (1985). Reexamining the Beck Depression Inventory: The long and short form of it. Psychological Reports, 56, 767-778.
102. Webster-Stratton, C. (1985). Predictor of treatment outcome in parent training for conduct-disordered children. Behavior Therapy, 16, 223-243.
103. Welch, G., Hall, A., & Walkey, F. (1990). The replicable dimensions of the Beck Depression Inventory. Journal of Clinical Psychology, 46, 817-827.
104. Westaway, M., & Wolmarans, L. (1992). Depression and self-esteem: Rapid screening for depression in Blacks, low literacy and hospitalized tuberculosis patients. Social Science and Medicine, 35, 1311-1315.
105. Whishman, M. A., & Kwon, P. (1992). Parental representations, cognitive distortions, and mild depression. Cognitive Therapy and Research, 16, 557-568.
106. Yin, P., & Fan, X. (2000). Assessing the reliability of Beck depression inventory scores: Reliability generalization across studies. Educational and Psychological Measurement, 60, 201-223.
107. Zea, M. C., Belgrave, F. Z., Townsend, T. G., Jarama, S. L., & Banks, S. R. (1996). The influence of social support and active coping on depression among African Americans and Latinos with disabilities. Rehabilitation Psychology, 41, 225-242.
108. Zemore,R. (1983). Development of a self-report measure of depression-proneness. Psychological Reports, 52, 211-216.
109. Zemore, R., & Bretell, D. (1983). Depression-proneness, low self-esteem, unhappy outlook, and narcissistic vulnerability. Psychological Reports, 52, 223-230.
110. Zimmerman, M. (1986). The stability of the revised Beck Depression Inventory in college students: Relationship with life events. Cognitive Therapy and Research, 10, 37-43.