Reliability and statistical power: How measurement fallibility affects power and required sample sizes for several parametric and nonparametric statistics

Journal of Modern Applied Statistical Methods

Volumn 6, Issue 1, 2007, Pages 81-90

  Kanyongo, Gibbs Y ^a,c   Brook, Gordon P ^b   Kyei Blankson, Lydia ^b,d   Gocmen, Gulsah ^b,d  

^a DUQUESNE UNIVERSITY   (United States)

^b OHIO UNIVERSITY   (United States)

^c Department of Education ^*

^d Department of Measurement and Statistics

Author keywords

Effect size; Monte Carlo simulations; Pseudorandom generation

Indexed keywords

EID: 48249135129     PISSN: 15389472     EISSN: 15389472     Source Type: Journal    
DOI: 10.22237/jmasm/1177992480     Document Type: Article

References (28)

1. Aron, A., & Aron, E. N. (1997). Statistics for the behavioral and social sciences: A brief course. Upper Saddle River, NJ: Prentice Hall.
2. Brooks, G. P. (2002). MC2G: Monte Carlo Analyses for 1 or 2 Groups (Version 3.0.7) [Computer software]. Retrieved from http://oak.cats.ohiou.edu/ ~brooksg/mc2g.htm
3. Brooks, G. P. (2002). MC3G: Monte Carlo Analyses for 3 Groups. (Version 1.1.1) [Computer software]. Retrieved from http://oak.cats.ohiou.edu/ ~brooksg/mc3g.htm
4. Brooks, G. P., Barcikowski, R. S., & Robey, R. R. (1999, April). Monte Carlo simulation for perusal and practice. Paper presented at the meeting of the American Educational Research Association, Montreal, Quebec, Canada.
5. Cleary, T. A., & Linn, R. L. (1969). Error of measurement and the power of a statistical test. British Journal of Mathematical and Statistical Psychology, 22, 49-55.
6. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
7. Crocker, L. & Algina, J. (1986). Introduction to classical and modern test theory. Fort Worth, TX: Holt, Rinehart, & Winston.
8. Feldt, L. S., & Brennen, R. L. (1993). Reliability. In R. L. Linn (Ed.). Educational measurement (pp. 105-146). Phoenix, AZ: Oryx.
9. Fink, A., & Kosecoff, J. (1998). How to conduct surveys: A step-by-step guide (2nd. ed.). Thousand Oaks, CA: Sage.
10. Glass, G. V., & Hopkins, K. D. (1996). Statistical methods in education and psychology (3rd ed.). Boston: Allyn & Bacon.
11. Hopkins, K. D., & Hopkins, B. R. (1979). The effect of the reliability of the dependent variable on power. Journal of Special Education, 13, 463-466.
12. Humphreys, L. G. (1993). Further comments on reliability and power of significance tests. Applied Psychological Measurement, 17, 11-14.
13. L'Ecuyer, P. (1988). Efficient and portable combined random number generators. Communications of the ACM, 31, 742-749, 774.
14. Levin, J. R., & Subkoviak, M. J. (1977). Planning an experiment in the company of measurement error. Applied Psychological Measurement, 1, 331-338.
15. Light, R. J., Singer, J. D., & Willett, J. B. (1990). By design: Planning research on higher education. Cambridge, MA: Harvard University.
16. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
17. McMillan, J. H., & Schumacher, S. (2001). Research in education: A conceptual introduction (5th ed.). New York: Longman.
18. Nicewander, W. A. & Price J. M. (1983). Reliability of measurement and the power of statistical tests: Some new results. Psychological Bulletin, 94, 524-533.
19. Park, S. K., & Miller, K. W. (1988). Random number generators: Good ones are hard to find. Communications of the ACM, 31, 1192-1201.
20. Press, W. H., Flannery, B. P., Teukolsky, S. A., & Vetterling, W. T. (1989). Numerical recipes in Pascal: The art of scientific computing. New York: Cambridge University.
21. Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1992). Numerical recipes in FORTRAN: The art of scientific computing (2nd ed.). New York: Cambridge University.
22. Robey, R. R., & Barcikowski, R. S. (1992). Type I error and the number of iterations in Monte Carlo studies of robustness. British Journal of Mathematical and Statistical Psychology, 45, 283-288.
23. Stevens, J. (2002). Applied multivariate statistics for the social sciences (4th ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
24. Subkoviak, M. J., & Levin, J. R. (1977). Fallibility of measurement and the power of a statistical test. Journal of Educational Measurement, 14, 47-52.
25. Sutcliffe, J. P. (1958). Error of measurement and the sensitivity of a test of significance. Psychometrika, 23, 9-17.
26. Williams, R. H., & Zimmerman, D. W. (1989). Statistical power analysis and reliability of measurement. Journal of General Psychology, 116, 359-369.
27. Zimmerman, D. W., & Williams, R. H. (1986). Note on the reliability of experimental measures and the poser of significance tests. Psychological Bulletin, 100, 123-124.
28. Zimmerman, D. W., Williams, R. H., & Zumbo, B. D. (1993). Reliability of measurement and power of significance tests based on differences. Applied Psychological Measurement, 17, 1-9.