The file drawer problem in reliability generalization: A strategy to compute a fail-safe N with reliability coefficients

Educational and Psychological Measurement

Volumn 68, Issue 1, 2008, Pages 120-128

  Howell, Ryan T ^a   Shields, Alan L ^b  

^a SAN FRANCISCO STATE UNIVERSITY   (United States)

^b EAST TENNESSEE STATE UNIVERSITY   (United States)

Author keywords

Fail Safe N; Meta analysis; Reliability; Reliability generalization

Indexed keywords

EID: 37649020143     PISSN: 00131644     EISSN: 15523888     Source Type: Journal    
DOI: 10.1177/0013164407301528     Document Type: Article

References (18)

1. Charter, R.A. (2003). Combining reliability coefficients: Possible application to meta-analysis and reliability generalization. Psychological Reports, 93, 643-647.
2. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2 nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
3. Henson, R.K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurement and Evaluation in Counseling and Development, 34, 177-189.
4. Henson, R.K. (2004). Expanding reliability generalization: Confidence intervals and charter's combined reliability coefficient. Perceptual and Motor Skills, 99, 818-820.
5. Henson, R.K., Kogan, L.R., & Vacha-Haase, T. (2001). A reliability generalization study of the teacher efficacy scale and related instruments. Educational and Psychological Measurement, 61, 404-420.
6. Henson, R.K., & Thompson, B. (2002). Characterizing measurement error in scores across studies: Some recommendations for conducting ''reliability generalization'' studies. Measurement and Evaluation in Counseling and Development, 35, 113-126.
7. Lane, G.G., White, A.E., & Henson, R.K. (2002). Expanding reliability generalization methods with KR-21 estimates: An RG study of the Coopersmith Self-Esteem Inventory. Educational and Psychological Measurement, 62, 685-711.
8. Nunnally, J.C., & Bernstein, I.H. (1994). Psychometric theory (3 rd ed.). New York: McGraw-Hill.
9. Rosenthal, R. (1979). The ''file drawer problem'' and tolerance for null results. Psychological Bulletin, 86, 638-641.
10. Rosenthal, R. (1991). Meta-analytic procedures for social research (2 nd ed.). Newbury Park, CA: Sage.
11. Schmidt, F., & Hunter, J.E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62, 529-540.
12. Shields, A.L., Howell, R.T., Potter, J.S., & Weiss, R.D. (in press). The Michigan Alcoholism Screening Test and its shortened forms: A meta-analytic inquiry into score reliability. Substance Use and Misuse.
13. Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58, 6-20.
14. Vacha-Haase, T., Henson, R., & Caruso, J.C. (2002). Reliability generalization: Moving toward improved understanding and use of score reliability. Educational and Psychological Measurement, 62, 562-569.
15. Vacha-Haase, T., Kogan, L.R., & Thompson, B. (2000). Sample compositions and variabilities in published studies versus those in test manuals: Validity of score reliability inductions. Educational and Psychological Measurement, 60, 509-522.
16. Viswesvaran, C., & Ones, D.S. (2000). Measurement error in ''Big Five Factors'' personality assessment: Reliability generalization across studies and measures. Educational and Psychological Measurement, 60, 224-235.
17. Wang, J. (2002). Reliability generalization: An HLM approach. Instructional Psychology, 29, 213-218.
18. Yin, P., & Fan, X. (2000). Assessing the reliability of Beck Depression Inventory scores: Reliability generalization across studies. Educational and Psychological Measurement, 60, 201-223.