Estimating the standardized mean difference with minimum risk: Maximizing accuracy and minimizing cost with sequential estimation

Psychological Methods

Volumn 22, Issue 1, 2017, Pages 94-113

  Chattopadhyay, Bhargab ^a   Kelley, Ken ^b  

^a UNIVERSITY OF TEXAS AT DALLAS   (United States)

^b UNIVERSITY OF NOTRE DAME   (United States)

Author keywords

Accuracy; Effect size; Research design; Sample size planning; Sequential procedure

Indexed keywords

EFFECT SIZE; PSYCINFO; SAMPLE SIZE; ALGORITHM; HUMAN; METHODOLOGY;

ALGORITHMS; HUMANS; RESEARCH DESIGN; SAMPLE SIZE;

EID: 84986005398     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/met0000089     Document Type: Article

References (57)

1. American Educational Research Association. (2006). Standards for reporting on empirical social science research in AERA publications: American Educational Research Association. Educational Researcher, 35, 33-40. http://dx.doi.org/10.3102/0013189X035006033.
2. American Psychological Association. (2001). Publication manual of the American Psychological Association (5th ed.). Washington, DC: Author.
3. American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author.
4. American Statistical Association. (2016). American statistical association releases statement on statistical significance and p-values: Provides principles to improve the conduct and interpretation of quantitative science. Retrieved from http://www.amstat.org/newsroom/pressreleases/ P-ValueStatement.pdf
5. Armitage, P. (1969). Sequential analysis in therapeutic trials. Annual Review of Medicine, 20, 425-430. http://dx.doi.org/10.1146/annurev.me .20.020169.002233.
6. Association for Psychological Science. (2014). Submission guidelines. Retrieved from http://www.psychologicalscience.org/index.php/publications/ journals/psychological_science/ps-submissions
7. Brannath, W., Gutjahr, G., & Bauer, P. (2012). Probabilistic foundation of confirmatory adaptive designs. Journal of the American Statistical Association, 107, 824-832. http://dx.doi.org/10.1080/01621459.2012 .682540.
8. Cedilnik, A., Kosmelj, K., & Blejec, A. (2006). Ratio of two random variables: A note on the existence of its moments. Metodoloski Zvezki, 3, 1-7.
9. Chattopadhyay, B., & Kelley, K. (in press). Estimation of the coefficient of variation with minimum risk: A sequential method for minimizing sampling error and cost with a stopping rule. Multivariate Behavioral Research.
10. Chattopadhyay, B., & Mukhopadhyay, N. (2013). Two-stage fixed-width confidence intervals for a normal mean in the presence of suspect outliers. Sequential Analysis, 32, 134-157. http://dx.doi.org/10.1080/ 07474946.2013.774609.
11. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.) Hillsdale, NJ: Erlbaum.
12. Dantzig, G. B. (1940). On the non-existence of tests of "student's" hypothesis having power functions independent of-. The Annals of Mathematical Statistics, 11, 186-192. Retrieved from http://www.jstor.org/ stable/2235875.
13. Davey, A., & Savla, J. (2010). Statistical power analysis with missing data: A structural equation modeling approach. New York, NY: Routledge.
14. De, S. K., & Chattopadhyay, B. (2015). Minimum risk point estimation of Gini Index. Retrieved from http://arxiv.org/abs/1503.08148.
15. Donaire, A., Falcón, C., Carreno, M., Bargallo, N., Rumia`, J., Setoain, J., . Fernadnez, S. (2009). Sequential analysis of fmri images: A new approach to study human epileptic networks. Epilepsia, 50, 2526-2537. http://dx.doi.org/10.1111/j.1528-1167.2009.02152.x
16. Eich, E. (2014). Business not as usual. Psychological Science, 25, 3-6. http://dx.doi.org/10.1177/0956797613512465.
17. Funder, D. C., Levine, J. M., Mackie, D. M., Morf, C. C., Sansone, C., Vazire, S., & West, S. G. (2014). Improving the dependability of research in personality and social psychology: Recommendations for research and educational practice. Personality and Social Psychology Review, 18, 3-12. http://dx.doi.org/10.1177/1088868313507536.
18. Ghosh, B. K., & Sen, P. K. (1991). Handbook of sequential analysis. New York, NY: CRC Press.
19. Graf, A. C., & Bauer, P. (2011). Maximum inflation of the type 1. error rate when sample size and allocation rate are adapted in a pre-planned interim look. Statistics in medicine, 30, 1637-1647.
20. Gut, A. (2009). Stopped random walks: Limit theorems and applications. Cambridge, UK: Springer.
21. Hedges, L. V. (1981). Distribution theory for glass's estimator of effect size and related estimators. Journal of Educational and Behavioral Statistics, 6, 107-128. http://dx.doi.org/10.3102/10769986006002107.
22. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
23. Jennison, C., & Turnbull, B. W. (2010). Group sequential methods with applications to clinical trials. Boca Raton, FL: CRC Press.
24. Kelley, K. (2007a). Confidence intervals for standardized effect sizes: Theory, application, and implementation. Journal of Statistical Software, 20, 1-24. http://dx.doi.org/10.18637/jss.v020.i08.
25. Kelley, K. (2007b). Methods for the behavioral, educational, and educational sciences: An R package. Behavior Research Methods, 39, 979-984. http://dx.doi.org/10.3758/BF03192993.
26. Kelley, K. (2016). MBESS 4.0.0 (or greater). [Computer software and manual]. Retrieved from http://www.cran.r-project.org Kelley, K., Maxwell, S. E., & Scott, E. (2003). Sample size for multiple regression: Obtaining regression coefficients that are accurate, not simply significant. Psychological Methods, 8, 305-321. http://dx.doi.org/10 .1037/1082-989X.8.3.305.
27. Kelley, K., & Preacher, K. J. (2012). On effect size. Psychological Methods, 17, 137-152. http://dx.doi.org/10.1037/a0028086.
28. Kelley, K., & Rausch, J. R. (2006). Sample size planning for the standardized mean difference: Accuracy in parameter estimation via narrow confidence intervals. Psychological Methods, 11, 363-385. http://dx.doi .org/10.1037/1082-989X.11.4.363.
29. Kothari, B. (2008). Let a billion readers bloom: Same language subtitling (sls) on television for mass literacy. International review of education, 54, 773-780.
30. Kothari, B., & Bandyopadhyay, T. (2014). Same language subtitling of Bollywood film songs on TV: Effects on literacy. Information Technologies & International Development, 10, 31-47. Retrieved from http:// itidjournal.org/index.php/itid/article/view/1307.
31. Kothari, B., Pandey, A., & Chudgar, A. R. (2004). Reading out of the idiot box: Same-language subtitling on television in India. Information Technologies and International Development, 2, 23-44. http://dx.doi.org/10 .1162/1544752043971170.
32. Kothari, B., Takeda, J., Joshi, A., & Pandey, A. (2002). Same language subtitling: A butterfly for literacy? International Journal of Lifelong Education, 21, 55-66. http://dx.doi.org/10.1080/02601370110099515.
33. Kraemer, H. C., & Thiemann, S. (1987). How many subjects? Beverly Hills, CA: Sage.
34. Leroux, B. G., Mancl, L. A., & DeRouen, T. A. (2005). Group sequential testing in dental clinical trials with longitudinal data on multiple outcome variables. Statistical Methods in Medical Research, 14, 591-602. http://dx.doi.org/10.1191/0962280205sm421oa.
35. Lim, I. S., & Leek, E. C. (2012). Curvature and the visual perception of shape: Theory on information along object boundaries and the minima rule revisited. Psychological Review, 119, 668. http://dx.doi.org/10 .1037/a0025962.
36. Lipsey, M. W. (1990). Design sensitivity: Statistical power for experimental research. Newbury Park, CA: Sage.
37. Lord, F. M. (1953). On the statistical treatment of football numbers. American Psychologist. http://dx.doi.org/10.1037/h0063675.
38. Mahalanobis, P. C. (1940). A sample survey of the acreage under jute in Bengal, with discussion on planning of experiments. Snakhya, 4, 511-531. Retrieved from http://www.jstor.org/stable/40383954.
39. Maxwell, S. E., & Kelley, K. (2011). Ethics and sample size planning. In A. T. Panter & S. K. Sterba (Eds.), Handbook of ethics in quantitative methodology (pp. 159-184). New York, NY: Taylor & Francis.
40. Miladinovic, B., Mhaskar, R., Hozo, I., Kumar, A., Mahony, H., & Djulbegovic, B. (2013). Optimal information size in trial sequential analysis of time-to-event outcomes reveals potentially inconclusive results because of the risk of random error. Journal of clinical epidemiology, 66, 654-659. http://dx.doi.org/10.1016/j.jclinepi.2012.11.007.
41. Mukhopadhyay, N., & Chattopadhyay, B. (2012). A tribute to Frank Anscombe and random central limit theorem from 1952. Sequential Analysis, 31, 265-277. http://dx.doi.org/10.1080/07474946.2012 .694344.
42. Mukhopadhyay, N., & Chattopadhyay, B. (2013). On a new interpretation of the sample variance. Statistical Papers, 54, 827-837. http://dx.doi .org/10.1007/s00362-012-0465-y
43. Mukhopadhyay, N., & Chattopadhyay, B. (2014). A note on the construction of a sample variance. Sri Lankan Journal of Applied Statistics, 15, 71-80. http://dx.doi.org/10.4038/sljastats.v15i1.6795.
44. Mukhopadhyay, N., & De Silva, B. M. (2009). Sequential methods and their applications. Boca Raton, FL: CRC Press.
45. Murphy, K. R., & Myors, B. (2004). Statistical power analysis: A simple and general model for traditional and modern hypothesis tests (2nd ed.). Mahwah, NJ: Erlbaum.
46. Petrie, A., Bulman, J., & Osborn, J. (2002). Further statistics in dentistry Pt. 4: Clinical trials 2. British Dental Journal, 193, 557-561. Retrieved from http://discovery.ucl.ac.uk/id/eprint/121963.
47. Pornprasertmanit, S., & Schneider, W. J. (2014). Accuracy in parameter estimation in cluster randomized designs. Psychological Methods, 19, 356-379. http://dx.doi.org/10.1037/a0037036.
48. Rao, C. R. (1973). Linear statistical inference and its applications. New York, NY: Wiley.
49. Raudenbush, S. W. (1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2, 173. http://dx.doi .org/10.1037/1082-989X.2.2.173.
50. Rozeboom, W. W. (1966). Foundations of the theory of prediction. Homewood, IL: Dorsey Press.
51. Sen, P. K. (1981). Sequential nonparametrics: Invariance principles and statistical inference. New York, NY: Wiley.
52. Sen, P. K., & Ghosh, M. (1981). Sequential point estimation of estimable parameters based on u-statistics. The Indian Journal of Statistics, Series A, 331-344. Retrieved from http://www.jstor.org/stable/25050282.
53. Statistical Research Group. (1945). Sequential analysis in inspection and experimentation: Introduction (Vols. SRG Report 255, Section 1). New York, NY: Columbia University Press.
54. Timmesfeld, N., Schäfer, H., & Müller, H.-H. (2007). Increasing the sample size during clinical trials with t-distributed test statistics without inflating the type I error rate. Statistics in Medicine, 26, 2449-2464. http://dx.doi.org/10.1002/sim.2725.
55. Todd, S., Whitehead, A., Stallard, N., & Whitehead, J. (2001). Interim analyses and sequential designs in phase iii studies. British Journal of Clinical Pharmacology, 51, 394-399. http://dx.doi.org/10.1046/j.1365-2125.2001.01382.x
56. Wald, A. (1945). Sequential tests of statistical hypotheses. Annals of Mathematical Statistics, 16, 117-186. Retrieved from http://www.jstor .org/stable/2235829.
57. Wasserstein, R. L., & Lazar, N. A. (2016). The ASA's statement on p-values: Context, process, and purpose. The American Statistician, 70, 129-133. http://dx.doi.org/10.1080/00031305.2016.1154108