How to meta-analyze coefficient-of-stability estimates: Some recommendations based on Monte Carlo studies

Educational and Psychological Measurement

Volumn 67, Issue 5, 2007, Pages 765-783

  Mason, Corinne ^a   Allam, Reynald ^b   Brannick, Michael T ^a  

^a UNIVERSITY OF SOUTH FLORIDA   (United States)

^b Morton Plant Hospital   (United States)

Author keywords

Meta analysis; Monte Carlo; Reliability generalization

Indexed keywords

EID: 34548459327     PISSN: 00131644     EISSN: 15523888     Source Type: Journal    
DOI: 10.1177/0013164407301532     Document Type: Article

References (37)

1. Beretvas, S.N., Meyers, J.L., & Leite, W.L. (2002). A reliability generalization study of the Marlowe-Crowne Social Desirability Scale. Educational and Psychological Measurement, 62, 570-589.
2. Beretvas, S.N., & Pastor, D.A. (2003). Using mixed-effects models in reliability generalization studies. Educational and Psychological Measurement, 63, 75-95.
3. Bonett, D.G. (2003). Sample size requirements for comparing two alpha coefficients. Applied Psychological Measurement, 27, 72-74.
4. Caruso, J.C. (2000). Reliability generalization of the NEO personality scales. Educational and Psychological Measurement, 60, 236-254.
5. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
6. Cooper, H., & Hedges, L. V. (Eds.). (1994). The handbook of research synthesis. New York: Russell Sage.
7. Deditius-Island, H.K., & Caruso, J.C. (2002). An examination of the reliability of scores from Zuckerman's Sensation Seeking Scales, Form V. Educational and Psychological Measurement, 62, 728-734.
8. Dierdorff, E.C., & Wilson, M.A. (2003). A meta-analysis of job analysis reliability. Journal of Applied Psychology, 88, 635-646.
9. Field, A.P. (2003). The problems in using fixed-effects models of meta-analysis on real-world data. Understanding Statistics, 2, 105-124.
10. Hall, S.M., & Brannick, M.T. (2002). Comparison of two random-effects methods of meta-analysis. Journal of Applied Psychology, 87, 377-389.
11. Hedges, L.V. (1983). A random effects model for effect sizes. Psychological Bulletin, 93, 388-395.
12. Hedges, L.V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
13. Hedges, L.V., & Vevea, J.L. (1998). Fixed and random-effects models in meta-analysis. Psychological Methods, 3, 486-504.
14. Henson, R.K., & Hwang, D. (2002). Variability and prediction of measurement error in Kolb's Learning Style Inventory scores: A reliability generalization study. Educational and Psychological Measurement, 62, 712-727.
15. Hunter, J.E., & Schmidt, F.L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Thousand Oaks, CA: Sage.
16. Hunter, J.E., & Schmidt, F.L. (2000). Fixed effects vs. random effects meta-analysis models: Implications for cumulative research knowledge. International Journal of Selection and Assessment, 8, 275-292.
17. Hunter, J.E., & Schmidt, F.L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2 nd ed). Thousand Oaks, CA: Sage.
18. Kaufman, L.S., & Kaufman, N. (1998). The Kaufman Test of Educational Achievement-Normative Update. Circle Pines, MN: AGS.
19. Kieffer, K.M., & Reese, R.J. (2002). A reliability generalization study of the Geriatric Depression Scale. Educational and Psychological Measurement, 62, 969-994.
20. Lipsey, M.W., & Wilson, D.B. (2001). Practical meta-analysis. Newbury Park, CA: Sage.
21. Nunnally, J.C., & Bernstein, I.H. (1994). Psychometric theory (3 rd ed). New York: McGraw-Hill.
22. O'Rourke, N. (2004). Reliability generalization of responses by care providers to the Center for Epidemiologic Studies-Depression Scale. Educational and Psychological Measurement, 64, 973-990.
23. Pedhazur, E.J. (1997). Multiple regression in behavioral research: Explanation and prediction (3 rd ed.). Fort Worth, TX: Harcourt Brace.
24. Rosenthal, R. (1991). Meta-analytic procedures for social research (Rev. ed.). Newbury Park, CA: Sage.
25. Sawilowsky, S.S. (2000). Psychometrics versus datametrics: Comment on Vacha-Haase's "reliability generalization" method and some EPM editorial policies. Educational and Psychological Measurement, 60, 157-173.
26. Schmidt, F.L., & Hunter, J.E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62, 529-549.
27. Schmidt, F.L., Hunter, J.E., & Raju, N.S. (1988). Validity generalization and situational specificity: A second look at the 75% rule and Fisher's r to z transformation. Journal of Applied Psychology, 73, 665-672.
28. Shields, A.L., & Caruso, J.C. (2004). A reliability induction and reliability generalization study of the Cage Questionnaire. Educational and Psychological Measurement, 62, 254-270.
29. Silver, N., & Dunlop, W. (1987). Averaging coefficients: Should Fisher's z-transformation be used? Journal of Applied Psychology, 72, 3-9.
30. Steel, P.D., & Kammeyer-Mueller, J.D. (2002). Comparing meta-analytic moderator estimation techniques under realistic conditions. Journal of Applied Psychology, 87, 96-111.
31. Thompson, B., & Cook, C. (2002). Stability of the reliability of LibQUAL+TM scores: A reliability generalization meta-analysis study. Educational and Psychological Measurement, 62, 735-743.
32. Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58, 6-20.
33. Viswesvaran, C., & Ones, D.S. (2000). Measurement error in "Big Five Factors" personality assessment: Reliability generalization across studies and measures. Educational and Psychological Measurement, 60, 224-235.
34. Viswesvaran, C., Ones, D.S., & Schmidt, F.L. (1996). Comparative analysis of the reliability of job performance ratings. Journal of Applied Psychology, 81, 557-574.
35. Voskuijl, O.F., & van Sliedregt, T. (2002). Determinants of interrater reliability of job analysis: A meta-analysis. European Journal of Psychological Assessment, 18, 52-62.
36. Wallace, K.A., & Wheeler, A.J. (2002). Reliability generalization of the Life Satisfaction Index. Educational and Psychological Measurement, 62, 674-684.
37. Yin, P., & Fan, X. (2000). Assessing the reliability of Beck Depression Inventory scores: Reliability generalization across studies. Educational and Psychological Measurement, 60, 201-223.