Bootstrapping in Applied Linguistics: Assessing its Potential Using Shared Data

Applied Linguistics

Volumn 36, Issue 5, 2015, Pages 591-610

  Plonsky, Luke ^a   Egbert, Jesse ^a   Laflair, Geoffrey T ^a  

^a NORTHERN ARIZONA UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84894096224     PISSN: 01426001     EISSN: 1477450X     Source Type: Journal    
DOI: 10.1093/applin/amu001     Document Type: Article

References (67)

1. Abbuhl, R. 2012. Practical methods for teaching replication to applied linguistics students in G. Porte (ed.): Replication Research in Applied Linguistics. Cambridge, pp. 135-50.
2. APA. 2010. Ethical principles of psychologists and code of conduct. Washington, DC: Retrieved 18 April 2013 from http://www. apa.org/ethics/code/index.aspx.
3. Bachman, L. F. 2004. Statistical Analyses for Language Assessment. Cambridge University Press.
4. Beasley, W. H., L. DeShea, L. E. Toothaker, J. L. Mendoza, D. E. Bard, and J. L. Rodgers. 2007. Bootstrapping to test for nonzero population correlation coefficients using univariate sampling, Psychological Methods 12: 414-33.
5. Beasley, W. H. and J. L. Rogers. 2009. Resampling Methods in R. E. Millsap and A. Maydeu-Olivares (eds): The SAGE Handbook of Quantitative Methods in Psychology. Sage.
6. Brown, J. D. 2011. Quantitative research in second language studies in E. Hinkel (ed.): Handbook of Research on Second Language Teaching and Learning. vol. 2. Routledge.
7. Canty, A. and B. Ripley. 2012. boot: Bootstrap R (S-Plus) Functions. R package version 1. 3-4.
8. Chernick, M. R. 2008. Bootstrap Methods: A Guide for Practitioners and Researchers, 2nd edn. John Wiley & Sons.
9. Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences, 2nd edn. Erlbaum.
10. Crookes, G. 1991. Power, effect size, and second language research. Another researcher comments, TESOL Quarterly 25: 762-5.
11. Davies, M. 2008. The Corpus of Contemporary American English: 450 million words, 1990-present, available at http://corpus.byu.edu/coca/.
12. Davison, A. C. and D. V. Hinkley. 1997. Bootstrap Methods and Their Applications. Cambridge University Press.
13. Di Nocera, F. and F. Ferlazzo. 2000. Resampling approach to statistical inference: Bootstrapping from event-related potentials, Behavior Research Methods, Instruments, & Computers 32: 111-19.
14. Efron, B. 1979. Bootstrap methods: Another look at the jackknife, Annals of Statistics 7: 1-26.
15. Efron, B. and R. J. Tibshirani. 1993. An Introduction to the Bootstrap. Chapman & Hall.
16. Ellis, N. C. 2000. Editorial statement, Language Learning 50: xi-xiii.
17. Field, A. 2005. Discovering Statistics Using SPSS, 2nd edn. Sage.
18. Firebaugh, G. 2007. Replication data sets and favored-hypothesis bias: Comment on Jeremy Freese (2007) and Gary King (2007), Sociological Methods & Research 36: 200-9.
19. Gass, S. 2009. A survey of SLA research in W. Ritchie and T. Bhatia (eds): Handbook of Second Language Acquisition. Emerald, pp. 3-28.
20. Gelman, A. and D. Weakliem. 2009. Of beauty, sex and power: Too little attention has been paid to the statistical challenges in estimating small effects, American Scientist 97: 310-16.
21. Ioannidis, J. P. A. 2008. Why most discovered true associations are inflated, Epidemiology 19: 640-8.
22. IRIS. A digital repository of data collection instruments for research in second language learning and teaching available at http://www.iris-database.org/iris/app/home/index.
23. Keselman, H. J., J. Algina, L. M. Lix, R. R. Wilcox, and K. N. Deering. 2008. A generally robust approach for testing hypotheses and setting confidence intervals for effect sizes, Psychological Methods 13: 110-29.
24. LaFlair, G. T., J. Egbert, and L. Plonsky. forthcoming. Bootstrapping in L. Plonsky (ed.): Advancing Quantitative Methods in Second Language Research. Routledge, pp. 135-50.
25. Lansing, L. 1999. Bootstrapping versus the Students t: The problems of type I error and power. Unpublished masters thesis, Lehigh University.
26. Lansing, L. 2004. A self-report and experimental study of situational factors involved in academic dishonesty using standard and bootstrapping analyses. Unpublished doctoral dissertation, Lehigh University.
27. Larson-Hall, J. 2010. A Guide to Doing Statistics in Second Language Research Using SPSS. Routledge.
28. Larson-Hall, J. 2012. Our statistical intuitions may be misleading us: Why we need robust statistics, Language Teaching 45: 460-74.
29. Larson-Hall, J. and R. Herrington. 2010. Improving data analysis in second language acquisition by utilizing modern developments in applied statistics, Applied Linguistics 31: 368-90.
30. Lazaraton, A. 1991. Power, effect size, and second language research. A researcher comments, TESOL Quarterly 25: 759-62.
31. Lazaraton, A. 2005. Quantitative research methods in E. Hinkel (ed.): Handbook of Research in Second Language Teaching and Learning. Erlbaum, pp. 109-224.
32. Lee, W.-C. and J. L. Rogers. 1998. Bootstrapping correlation coefficients using univariate and bivariate sampling, Psychological Methods 3: 91-103.
33. Lee, J., J. Jang, and L Plonsky. 2014. The effectiveness of second language pronunciation instruction: A meta-Analysis, Applied Linguistics. doi:10.093/applin/amu040.
34. Light, R. J., J. D. Singer, and J. B. Willet. 1990. By Design: Planning Research on Higher Education. Harvard University Press.
35. Loewen, S., E. Lavolette, L. A. Spino, M. Papi, J. Schmidtke, S. Sterling, and D. Wolff. 2014. Statistical literacy among applied linguists and second language acquisition researchers, TESOL Quarterly 48: 360-88.
36. Lucas, R. E. and M. B. Donnellan. 2013. Improving the replicability and reproducibility of research published in the Journal of Research in Personality, Journal of Research in Personality 47: 453-4.
37. Magnan, S. S. 1994. From the editor: The MLJ tradition and the challenges ahead, The Modern Language Journal 78: 7-9.
38. Meier, K. J. 1995. Replication: A view from the streets, PS: Political Science and Politics 28: 456-8.
39. Merton, R. K. 1973. The Sociology of Science: Theoretical and Empirical Investigations. University of Chicago Press.
40. Mooney, C. Z. and R. D. Duval. 1993. Bootstrapping: A Nonparametric Approach to Statistical Inference. Sage.
41. Norris, J. M. and L. Ortega. 2000. Effectiveness of L2 instruction: A research synthesis and quantitative meta-Analysis, Language Learning 50: 417-528.
42. Norris, J. M. and L. Ortega. 2006. The value and practice of research synthesis for language learning and teaching in J. M. Norris and L. Ortega (eds): Synthesizing Research on Language Learning and Teaching. Benjamins, pp. 3-50.
43. Oswald, F. L. and L. Plonsky. 2010. Meta-Analysis in second language research: Choices and challenges, Annual Review of Applied Linguistics 30: 85-110.
44. Phakiti, A. 2010. Analysing quantitative data in B. Paltridge and A. Phakiti (eds): Continuum Companion to Research Methods in Applied Linguistics. Continuum, pp. 39-49.
45. Plonsky, L. 2009. Nix the null: Why statistical significance is overrated, Paper presented at the Second Language Research Forum (SLRF), East Lansing, MI.
46. Plonsky, L. 2011. The effectiveness of second language strategy instruction: A meta-Analysis, Language Learning 61: 993-1038.
47. Plonsky, L. 2012. Replication, meta-Analysis, and generalizability in G. Porte (ed.): Replication Research in Applied Linguistics. Cambridge University Press, pp. 116-32.
48. Plonsky, L. 2013. Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research, Studies in Second Language Acquisition 35: 655-87.
49. Plonsky, L. 2014. Study quality in quantitative L2 research (1990-2010): A methodological synthesis and call for reform, Modern Language Journal 98: 450-70.
50. Plonsky, L. and S. Gass. 2011. Quantitative research methods, study quality, and outcomes: The case of interaction research, Language Learning 61: 325-66.
51. Plonsky, L. and F. L. Oswald (2012). How to do a meta-Analysis in A. Mackey and S. M. Gass (eds): Research methods in second language acquisition: A practical guide. Basil Blackwell, pp. 275-95.
52. Plonsky, L. and F. L. Oswald. under review. How big is big? Interpreting effect sizes in L2 research, Manuscript under review.
53. Porte, G. (ed.) 2012. Replication Research in Applied Linguistics. Cambridge University Press.
54. Puschmann, C. 2010. Blog, available at http://blog.ynada.com/184. Accessed 18 April, 2013.
55. R Core Team. 2012. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, available at http://www. R-project.org/.
56. Reder, S., K. Harris, and K. Setzler. 2003. The multimedia adult ESL learner corpus, TESOL Quarterly 37: 546-57.
57. Reidpath, D. D. and P. A. Allotey. 2001. Data sharing in medical research: An empirical investigation, Bioethics 15: 125-34.
58. Spanish Learner Language Oral Corpora 2 (SPLLOC2) Available at http://www.splloc. soton.ac.uk/splloc2/index.html.
59. Tongbai, R. R., F. Yu, and K. M. Miller. 2010. Multivariate nonparametric techniques for astigmatism analysis, Journal of Cataract & Refractive Surgery 36: 594-602.
60. Tukey, J. W. 1960. A survey of sampling from contaminated distributions in I. Olkin, S. G. Ghwyne, W. Hoeffding, W. G. Madow, and H. B. Mann (eds): Contributions to Probability and Statistics: Essays in Honour of Harold Hotelling. Stanford University Press, pp. 448-85.
61. Tukey, J. W. 1991. The philosophy of multiple comparisons, Statistical Science 6: 100-16.
62. Welch, W. W., D. Huffman, and F. Lawrenz. 1998. The precision of data obtained in largescale science assessments: An investigation of bootstrapping and half-sample replication methods, Journal of Research in Science Teaching 35: 697-704.
63. Wicherts, J. M., M. Bakker, and D. Molenaar. 2011. Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results, PLoS One 6: 1-7.
64. Wicherts, J. M., D. Borsboom, J. Kats, and D. Molenaar. 2006. The poor availability of psychological research data for reanalysis, American Psychologist 61: 726-8.
65. Wilcox, R. 2001. Fundamentals of Modern Statistical Methods: Substantially Improving Power and Accuracy. Springer.
66. Wilcox, R. 2003. Applying Contemporary Statistical Techniques. Elsevier Science.
67. Wolfe, E. W. and M. T. McGill. 2011. Comparison of asymptotic and bootstrap item fit indices in identifying misfit to the Rasch model, Paper presented at the National Conference on Measurement in Education, New Orleans, LA.