Synthesizing Evidence in Public Policy Contexts: The Challenge of Synthesis When There Are Only a Few Studies

Evaluation Review

Volumn 41, Issue 1, 2017, Pages 3-26

  Valentine, Jeffrey C ^a   Wilson, Sandra Jo ^b   Rindskopf, David ^c   Lau, Timothy S ^a   Tanner Smith, Emily E ^b   Yeide, Martha ^d   LaSota, Robin ^d   Foster, Lisa ^d  

^a UNIVERSITY OF LOUISVILLE   (United States)

^b VANDERBILT UNIVERSITY   (United States)

^c CITY UNIVERSITY OF NEW YORK   (United States)

^d Development Services Group Inc   (United States)

Author keywords

Bayesian statistics; content area; education; meta analysis; methodological development; research synthesis; systematic review

Indexed keywords

ARTICLE; EDUCATION; HUMAN; HUMAN EXPERIMENT; META ANALYSIS; PUBLIC POLICY; SYNTHESIS;

EID: 85011990354     PISSN: 0193841X     EISSN: 15523926     Source Type: Journal    
DOI: 10.1177/0193841X16674421     Document Type: Article

References (16)

1. Cohen J., (1962). The statistical power of abnormal-social psychological research: A review. Journal of Abnormal and Social Psychology, 65, 145–153.
2. Cooper H., Hedges L. V., (2009). Research synthesis as a scientific process. In Cooper H., Hedges L. V., Valentine J. C., (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 3–18). New York, NY: Russell Sage Foundation.
3. Dwan K., Altman D. G., Arnaiz J. A., Bloom J., Chan A. W., Cronin E.,, …Williamson P. R., (2008). Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One, 3, e3081. doi:10.1371/journal.pone.0003081
4. Hedges L. V., Olkin I., (1985). Statistical methods for meta-analysis. Orlando: FL: Academic Press.
5. Hedges L. V., Vevea J. L., (1998). Fixed-and random-effects models in meta-analysis. Psychological Methods, 3, 486–504.
6. Higgins J. P. T., Green S., (Eds.). (2011). Cochrane handbook for systematic reviews of interventions (Version 5.1.0). Retrieved from www.cochrane-handbook.org
7. Pigott T. D., Valentine J. C., Polanin J. R., Williams R. T., (2013). Outcome-reporting bias in education research. Educational Researcher, 42, 424–432. doi:0013189X13507104
8. Press S. J., (2003). Subjective and objective Bayesian statistics: Principles, models, and applications (2nd ed.). Hoboken, NJ: John Wiley and Sons.
9. Rothstein H. R., Hopewell S., (2009). Grey literature. In Cooper H., Hedges L. V., Valentine J. C., (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 103–128). New York, NY: Russell Sage Foundation.
10. Sedlmeier P., Gigerenzer P., (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105, 309–316.
11. U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2013, 6). Early childhood education intervention report: Doors to Discovery™. Retrieved from http://whatworks.ed.gov
12. U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2014a, 5). Students with learning disabilities intervention report: Repeated reading. Retrieved from http://whatworks.ed.gov
13. U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2014b, 11). Developmental students in postsecondary education intervention report: Linked learning communities. Retrieved from http://whatworks.ed.gov
14. Valentine J. C., Biglan A., Boruch R. F., Castro F. G., Collins L. M., Flay B. R.,, …Schinke S. P., (2011). Replication in prevention science. Prevention Science, 12, 103–117.
15. Valentine J. C., Pigott T. D., Rothstein H. R., (2010). How many studies do you need? A primer on statistical power in meta-analysis. Journal of Educational and Behavioral Statistics, 35, 215–247.
16. Viechtbauer W., (2005). Bias and efficiency of meta-analytic variance estimators in the random-effects model. Journal of Educational and Behavioral Statistics, 30, 261–293.