Using the student's t-test with extremely small sample sizes

Practical Assessment, Research and Evaluation

Volumn 18, Issue 10, 2013, Pages 1-12

  de Winter, J C F ^a  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84886302893     PISSN: None     EISSN: 15317714     Source Type: Journal    
DOI: None     Document Type: Article

References (51)

1. Bem, D. J., Utts, J., & Johnson, W. O. (2011). Must psychologists change the way they analyze their data? Journal of Personality and Social Psychology, 101, 716-719.
2. Blair, R. C., Higgins, J. J., & Smitley, W. D. (1980). On the relative power of the U and t tests. British Journal of Mathematical and Statistical Psychology, 33, 114-120.
3. Box, J. F. (1987). Guinness, Gosset, Fisher, and small samples. Statistical Science, 2, 45-52.
4. Bridge, P. D., & Sawilowsky, S. S. (1999). Increasing physicians' awareness of the impact of statistics on research outcomes: comparative power of the t-test and Wilcoxon rank-sum test in small samples applied research. Journal of Clinical Epidemiology, 52, 229-235.
5. Campbell, M. J., Julious, S. A., & Altman, D. G. (1995). Estimating sample sizes for binary, ordered categorical, and continuous outcomes in two group comparisons. BMJ: British Medical Journal, 311, 1145-1148.
6. Cohen, J. (1970). Approximate power and sample size determination for common one-sample and twosample hypothesis tests. Educational and Psychological Measurement, 30, 811-831.
7. Conover, W. J., & Iman, R. L. (1981). Rank transformations as a bridge between parametric and nonparametric statistics. The American Statistician, 35, 124-129.
8. De Winter, J. C. F., & Dodou, D. (2010). Five-point Likert items: t test versus Mann-Whitney-Wilcoxon. Practical Assessment, Research & Evaluation, 15, 11.
9. Elliott, A. C., & Woodward, W. A. (2007). Comparing one or two means using the t-test. Statistical Analysis Quick Reference Guidebook. SAGE.
10. Fay, M. P., & Proschan, M. A. (2010). Wilcoxon-Mann- Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. Statistics Surveys, 4, 1-39.
11. Fitts, D. A. (2010). Improved stopping rules for the design of efficient small-sample experiments in biomedical and biobehavioral research. Behavior Research Methods, 42, 3-22.
12. Fritz, C. O., Morris, P. E., & Richler, J. J. (2012). Effect size estimates: Current use, calculations, and interpretation. Journal of Experimental Psychology: General, 141, 2-18.
13. Harlow, H. F., Dodsworth, R. O., & Harlow, M. K. (1965). Total social isolation in monkeys. Proceedings of the National Academy of Sciences of the United States of America, 54, 90-97.
14. Hesterberg, T., Moore, D. S., Monaghan, S., Clipson, A., & Epstein, R. (2005). Bootstrap methods and permutation tests. Introduction to the Practice of Statistics, 5, 1-70.
15. Ingre, M. (in press). Why small low-powered studies are worse than large high-powered studies and how to protect against "trivial" findings in research: Comment on Friston (2012). NeuroImage.
16. Ioannidis, J. P. (2005). Why most published research findings are false. PLOS Medicine, 2, e124.
17. Ioannidis, J. P. (2008). Why most discovered true associations are inflated. Epidemiology, 19, 640-648.
18. Ioannidis, J. P. (2013). Mega-trials for blockbusters. JAMA, 309, 239-240.
19. Janušonis, S. (2009). Comparing two small samples with an unstable, treatment-independent baseline. Journal of Neuroscience Methods, 179, 173-178.
20. Lehmann, E. L. (2012). "Student" and small-sample theory. In Selected Works of E.L. Lehmann (pp. 1001-1008). Springer US.
21. Ludbrook, J., & Dudley, H. (1998). Why permutation tests are superior to t and F tests in biomedical research. The American Statistician, 52, 127-132.
22. Mathes, E.W., King, C.A., Miller, J.K. and Reed, R.M. (2002). An evolutionary perspective on the interaction of age and sex differences in short-term sexual strategies. Psychological Reports, 90, 949-956.
23. Mesholam, R. I., Moberg, P. J., Mahr, R. N., & Doty, R. L. (1998). Olfaction in neurodegenerative disease: a metaanalysis of olfactory functioning in Alzheimer's and Parkinson's diseases. Archives of Neurology, 55, 84-90.
24. Mudge, J. F., Baker, L. F., Edge, C. B., & Houlahan, J. E. (2012). Setting an optimal a that minimizes errors in null hypothesis significance tests. PLOS One, 7, e32734.
25. OECD (2010), PISA 2009 Results: What students know and can do - Student performance in reading, mathematics and science (volume I) http://dx.doi.org/10.1787/9789264091450-en
26. Pashler, H., & Harris, C. R. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7, 531-536.
27. Posten, H. O. (1982). Two-sample Wilcoxon power over the Pearson system and comparison with the t-test. Journal of Statistical Computation and Simulation, 16, 1-18.
28. Ramsey, P. H. (1980). Exact Type 1 error rates for robustness of Student's t test with unequal variances. Journal of Educational and Behavioral Statistics, 5, 337-349.
29. Richard, F. D., Bond, C. F., Jr., & Stokes-Zoota, J. J. (2003). One hundred years of social psychology quantitatively described. Review of General Psychology, 7, 331-363.
30. Rossi, J. S. (1990). Statistical power of psychological research: What have we gained in 20 years? Journal of Consulting and Clinical Psychology, 58, 646-656.
31. Rost, D. H. (1991). Effect strength vs. statistical significance: A warning against the danger of small samples: A comment on Gefferth and Herskovits's article "Leisure activities as predictors of giftedness". European Journal for High Ability, 2, 236-243.
32. Royle, N. A., Booth, T., Valdés Hernàndez, M. C., Penke, L., Murray, C., Gow, A. J.,. & Wardlaw, J. M. (in press). Estimated maximal and current brain volume predict cognitive ability in old age. Neurobiology of Aging.
33. Ruthsatz, J., & Urbach, J. B. (2012). Child prodigy: A novel cognitive profile places elevated general intelligence, exceptional working memory and attention to detail at the root of prodigiousness. Intelligence, 40, 419-426.
34. Ruxton, G. D. (2006). The unequal variance t-test is an underused alternative to Student's t-test and the Mann- Whitney U test. Behavioral Ecology, 17, 688-690.
35. Satterthwaite, F. E. (1946). An approximate distribution of estimates of variance components. Biometrics Bulletin, 2, 110-114.
36. Sawilowsky, S. S., & Blair, R. C. (1992). A more realistic look at the robustness and Type II error properties of the t test to departures from population normality. Psychological Bulletin, 111, 352-360.
37. Shea, C. H., & Wulf, G. (1999). Enhancing motor learning through external-focus instructions and feedback. Human Movement Science, 18, 553-571.
38. Sheppard, C. R. (1999). How large should my sample be? Some quick guides to sample size and the power of tests. Marine Pollution Bulletin, 38, 439-447.
39. Siegel, S. (1956). Nonparametric statistics for the behavioral sciences. New York: McGraw-Hill.
40. Siegel, S. (1957). Nonparametric statistics. The American Statistician, 11, 13-19.
41. Siegel, S., & Castellan, N. J. (1988). Nonparametric statistics for the behavioral sciences. Second Edition. New York: McGraw- Hill.
42. Sloane, N. J. (2003). The on-line encyclopedia of integer sequences. http://oeis.org.
43. Stevens, S. S. (1961). To honor Fechner and repeal his law. Science, 133, 80-86.
44. Student. (1908). The probable error of a mean. Biometrika, 6, 1-25.
45. Tucker-Drob, E. M. (2013). How many pathways underlie socioeconomic differences in the development of cognition and achievement? Learning and Individual Differences, 25, 12-20.
46. Voracek, M., Fisher, M. L., Hofhansl, A., Vivien Rekkas, P., & Ritthammer, N. (2006). "I find you to be very attractive." Biases in compliance estimates to sexual offers. Psicothema, 18, 384-391.
47. Welch, B. L. (1958), "'Student' and Small Sample Theory," Journal of the American Statistical Association, 53, 777-788.
48. Wikipedia (2013). http://en.wikipedia.org/wiki/Paired_difference_test Accessed 21 July 2013.
49. Zabell, S. L. (2008). On Student's 1908 article "The Probable Error of a Mean". Journal of the American Statistical Association, 103, 1-7.
50. Zimmerman, D. W., & Zumbo, B. D. (1989). A note on rank transformations and comparative power of the student t-test and Wilcoxon-Mann-Whitney Test. Perceptual and Motor Skills, 68, 1139-1146.
51. Zimmerman, D. W., & Zumbo, B. D. (1993). Rank transformations and the power of the Student t-test and Welch t' test for non-normal populations with unequal variances. Canadian Journal of Experimental Psychology, 47, 523-539.