Meta-analyzing second language research

Advancing Quantitative Methods in Second Language Research

Volumn , Issue , 2015, Pages 106-128

  Plonsky, Luke ^a   Oswald, Frederick L ^b  

^a NORTHERN ARIZONA UNIVERSITY   (United States)

^b RICE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85111289948     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4324/9781315870908-6     Document Type: Chapter

References (72)

1. APA Publications and Communications Board Working Group on Journal Article Reporting Standards. (2008). Reporting standards for research in psychology: Why do we need them? What might they be? American Psychologist, 63, 839-851.
2. Aytug, Z. G., Rothstein, H. R., Zhou, W., & Kern, M. C. (2012). Revealed or concealed? Transparency of procedures, decisions, and judgment calls in meta-analyses. Organizational Research Methods, 15, 103-133.
3. Borenstein, M., Hedges, L. V., Higgins, J.P.T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester, UK: Wiley.
4. Cheung, S. F., & Chan, D. K-S. (2004). Dependent effect sizes in meta-analysis: Incorporating the degree of interdependence. Journal of Applied Psychology, 89(5), 780-791.
5. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
6. Cohn, L. D., & Becker, B. J. (2003). How meta-analysis increases statistical power. Psychological Methods, 8(3), 243-253.
7. Cooper, H. (2010). Research synthesis and meta-analysis: A step-by-step approach (4th ed). Thousand Oaks, CA: Sage.
8. Cooper, H., Hedges, L. V., & Valentine, J. C. (Eds.). (2009). The handbook of research synthesis and meta-analysis (2nd ed.). New York: Russell Sage Foundation.
9. Cooper, H. M., & Rosenthal, R. (1980). Statistical versus traditional procedures for summarizing research findings. Psychological Bulletin, 87(3), 442-449.
10. Dalton, D. R., & Dalton, C. M. (2008). Meta-analyses: Some very good steps toward a bit longer journey. Organizational Research Methods, 11(1), 127-147.
11. Fern, E. F., & Monroe, K. B. (1996). Effect-size estimates: Issues and problems in interpretation. Journal of Consumer Research, 23(2), 89-105.
12. Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3-8.
13. Gleser, L. J., & Olkin, I. (2009). Stochastically dependent effect sizes. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (second ed., pp. 357-376). New York: Sage.
14. Harzing, A. W. (2007). Publish or perish, available from http://www.harzing.com/pop.htm.
15. Hattie, J. A., Biggs, J., & Purdie, N. (1996). Effects of learning skills interventions on student learning: A meta-analysis. Review of Educational Research, 66(2), 99-136.
16. Hedges, L. V. (2008). What are effect sizes and why do we need them? Child Development Perspectives, 2(3), 167-171.
17. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
18. Hedges, L. V., & Pigott, T. D. (2001). The power of statistical tests in meta-analysis. Psychological Methods, 6(3), 203-217.
19. Higgins, J.P.T., White, I. R., & Wood, A. M. (2008). Imputation methods for missing outcome data in meta-analysis of clinical trials. Clinical Trials, 5(3), 225-239.
20. Hunter, F. L., & Schmidt, J. E. (2014). Methods of meta-analysis: Correcting error and bias in research findings (third ed.). Thousand Oaks, CA: Sage.
21. Inʼnami, Y., & Koizumi, R. (2009). A meta-analysis of test format effects on reading and listening test performance: Focus on multiple-choice and open-ended formats. Language Testing, 26(2), 219-244.
22. Inʼnami, Y., & Koizumi, R. (2010). Database selection guidelines for meta-analysis in applied linguistics. TESOL Quarterly, 44(1), 169-184.
23. Inʼnami, Y., & Koizumi, R. (Eds.) (2014). Research synthesis and meta-analysis in second language learning and testing. Special issue of English Teaching and Learning.
24. Jeon, E. H., & Yamashita, J. (2014). L2 reading comprehension and its correlates: A meta-analysis. Language Learning, 64, 160-212.
25. Kepes, S., McDaniel, M. S., Brannick, M. T., & Banks, G. C. (2013). Meta-analytic reviews in the organizational sciences: Two meta-analytic schools on the way to MARS (the meta-analytic reporting standards). Journal of Business Psychology, 28, 123-143.
26. Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56(5), 746-759.
27. Kline, R. B. (2013). Beyond significance testing: Statistics reform in the behavioral sciences (2nd ed.). Washington, DC: American Psychological Association.
28. Lee, S-K., & Huang, H-T. (2008). Visual input enhancement and grammar learning: A meta-analytic review. Studies in Second Language Acquisition, 30(3), 307-331.
29. Lee, J., Jang, J., & Plonsky, L. (in press). The effectiveness of second language pronunciation instruction: A meta-analysis. Applied Linguistics.
30. Li, S. (2010). The effectiveness of corrective feedback in SLA: A meta-analysis. Language Learning, 60(2), 309-365.
31. Li, S., Shintani, N., & Ellis, R. (Eds.) (forthcoming). The complementary contribution of meta-analysis and narrative review in second language acquisition research. Applied Linguistics, special issue.
32. Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.
33. Lyster, R., & Saito, K. (2010). Oral feedback in classroom SLA: A meta-analysis. Studies in Second Language Acquisition, 32(2), 265-302.
34. Mackey, A., & Goo, J. (2007). Interaction research in SLA: A meta-analysis and research synthesis. In A. Mackey (Ed.), Conversational interaction in second language acquisition: A collection of empirical studies (pp. 407-451). New York: Oxford University Press.
35. McManus, R. J., Wilson, S., Delaney, B. C., Fitzmaurice, D. A., Hyde, C. J., Tobias, R. S., Jowett, S., & Hobbs, F.D.R. (1998). Review of the usefulness of contacting other experts when conducting a literature search for systematic reviews. British Medical Journal, 317, 1562-1563.
36. Morris, S. B. (2008). Estimating effect sizes from pretest-posttest-control group designs. Organizational Research Methods, 11(2), 364-386.
37. Norris, J. M., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning, 50(3), 417-528.
38. Norris, J. M., & Ortega, L. (2006a). Synthesizing research on language learning and teaching. Philadelphia, PA: John Benjamins.
39. Norris, J. M., & Ortega, L. (2006b). The value and practice of research synthesis for language learning and teaching. In J. M. Norris & L. Ortega (Eds.), Synthesizing research on language learning and teaching (pp. 3-50). Philadelphia, PA: John Benjamins.
40. Norris, J. M., & Ortega, L. (2007). The future of research synthesis in applied linguistics: Beyond art or science. TESOL Quarterly, 41(4), 805-815.
41. Norris, J. M., & Ortega, L. (2010). Research Timeline: Research synthesis. Language Teaching, 43, 61-79.
42. Ortega, L. (in press). Research synthesis. In B. Paltridge & A. Phakiti (Eds.), Companion to research methods in applied linguistics. London: Continuum.
43. Orwin, R. G. (1994). Evaluating coding decisions. In H. Cooper & L. V. Hedges (Eds.), Handbook of research synthesis (pp. 139-162). New York: Russell Sage Foundation.
44. Orwin, R. G., & Cordray, D. S. (1985). Effects of deficient reporting on meta-analysis: A conceptual framework and reanalysis. Psychological Bulletin, 97(1), 134-147.
45. Oswald, F. L., & Johnson, J. W. (1998). On the robustness, bias, and stability of statistics from meta-analysis of correlation coefficients: Some initial Monte Carlo findings. Journal of Applied Psychology, 83(2), 164-178.
46. Oswald, F. L., & McCloy, R. A. (2003). Meta-analysis and the art of the average. In K. R. Murphy (Ed.), Validity generalization: A critical review (pp. 311-338). Mahwah, NJ: Lawrence Erlbaum.
47. Oswald, F. L., & Plonsky, L. (2010). Meta-analysis in second language research: Choices and challenges. Annual Review of Applied Linguistics, 30, 85-110.
48. Pearson, K. (1904). Report on certain enteric fever inoculation statistics. British Medical Journal, 3, 1243-1246.
49. Plonsky, L. (2011). The effectiveness of second language strategy instruction: A metaanalysis. Language Learning, 61, 993-1038.
50. Plonsky, L. (2012). Replication, meta-analysis, and generalizability. In G. Porte (Ed.), Replication research in applied linguistics (pp. 116-132). New York: Cambridge University Press.
51. Plonsky, L. (2013). Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35, 655-687.
52. Plonsky, L., & Brown, D. (2015). Domain definition and search techniques in meta-analyses of L2 research (or why 18 meta-analyses of feedback have different results). Second Language Research, 31, 267-276.
53. Plonsky, L., Egbert, J., & LaFlair, G. T. (in press). Bootstrapping in applied linguistics: Assessing its potential using shared data. Applied Linguistics.
54. Plonsky, L., & Gass, S. M. (2011). Quantitative research methods, study quality, and outcomes: The case of interaction research. Language Learning, 61, 325-366.
55. Plonsky, L., & Oswald, F. L. (2014). How big is ‘big’? Interpreting effect sizes in L2 research. Language Learning, 64, 878-912.
56. Poltavtchenko, E., & Johnson, M. D. (2009, March). Feedback and second language writing: A meta-analysis. Poster session presented at the annual meeting of TESOL, Denver, CO.
57. Rosenthal, R. (1978). Combining results of independent studies. Psychological Bulletin, 85(1), 185-193.
58. Ross, S. (1998). Self-assessment in second language testing: A meta-analysis and analysis of experiential factors. Language Testing, 15(1), 1-20.
59. Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.). (2005). Publication bias in meta-analysis: Prevention, assessment and adjustments. Chichester, England: Wiley.
60. Russell, J., & Spada, N. (2006). The effectiveness of corrective feedback for the acquisition of L2 grammar: A meta-analysis of the research. In J. M. Norris & L. Ortega (Eds.), Synthesizing research on language learning and teaching (pp. 133-164). Philadelphia: John Benjamins.
61. Schmidt, F. L., & Hunter, J. E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62(5), 529-540.
62. Schmidt, F. L., Le, H., & Oh, I-S. (2009). Correcting for the distorting effects of study artifacts in meta-analysis. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (second ed., pp. 317-333). New York: Russell Sage Foundation.
63. Schmidt, F. L., Oh, I-S., & Hayes, T. (2009). Fixed versus random effects models in meta-analysis: Model properties and an empirical comparison of differences in results. British Journal of Mathematical and Statistical Psychology, 62(1), 97-128.
64. Spada, N., & Tomita, Y. (2010). Interactions between type of instruction and type of language feature: A meta-analysis. Language Learning, 60(2), 263-308.
65. Stukas, A. A., & Cumming, G. (in press). Interpreting effect sizes: Towards a quantitative cumulative social psychology. European Journal of Social Psychology.
66. Sutton, A. J., & Higgins, J. P. T. (2008). Recent development in meta-analysis. Statistics in Medicine, 27(5), 625-650.
67. Taylor, A., Stevens, J. R., & Asher, J. W. (2006). The effects of explicit reading strategy training on L2 reading comprehension: A meta-analysis. In J. M. Norris & L. Ortega (Eds.), Synthesizing research on language learning and teaching (pp. 213-244). Philadelphia, PA: John Benjamins.
68. Truscott, J. (2007). The effect of error correction on learners’ ability to write accurately. Journal of Second Language Writing, 16(4), 255-272.
69. Valentine, J. C., Pigott, T. D., & Rothstein, H. R. (2010). How many studies do you need?: A primer on statistical power for meta-analysis. Journal of Educational and Behavioral Statistics, 35(2), 215-247.
70. White, H. D. (2009). Scientific communication and literature retrieval. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis (second ed., pp. 51-71). New York: Russell Sage Foundation.
71. Wilkinson, L., & Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54(8), 594-604.
72. Wilson, D. B. (2009). Systematic coding. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis (second ed., pp. 159-176). New York: Russell Sage Foundation.