The role of method in treatment effectiveness research: Evidence from meta-analysis

Psychological Methods

Volumn 6, Issue 3, 2001, Pages 413-429

  Wilson, David B ^a   Lipsey, Mark W ^b  

^a GEORGE MASON UNIVERSITY   (United States)

^b NONE

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; BEHAVIOR THERAPY; CLINICAL TRIAL; EPIDEMIOLOGY; HUMAN; META ANALYSIS; PSYCHOLOGY; PSYCHOMETRY; PSYCHOTHERAPY; REPRODUCIBILITY; RESEARCH; STATISTICAL ANALYSIS; STATISTICS; TREATMENT OUTCOME;

BEHAVIOR THERAPY; BIAS (EPIDEMIOLOGY); CLINICAL TRIALS; DATA INTERPRETATION, STATISTICAL; HUMANS; OUTCOME AND PROCESS ASSESSMENT (HEALTH CARE); PSYCHOLOGY, EDUCATIONAL; PSYCHOMETRICS; PSYCHOTHERAPY; REPRODUCIBILITY OF RESULTS; RESEARCH;

EID: 0035748271     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/1082-989x.6.4.413     Document Type: Review

References (69)

1. Angert, J. F., & Clark, F. E. (1982, May). Finding the rose among the thorns: Some thoughts on integrating media research. Paper presented at the meeting of the Association for Educational Communications and Technology, Dallas, TX. (ERIC Document Reproduction Service No. 223 192)
2. Begg, C. B. (1994). Publication bias. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 399-409). New York: Russell Sage Foundation.
3. Boruch, R. F. (1997). Randomized experiments for planning and evaluation: A practical guide. Thousand Oaks, CA: Sage.
4. Chidester, T. R., & Grigsby, W. C. (1984). A meta-analysis of the goal setting-performance literature. In Academy of Management Proceedings (pp. 202-206). Briarcliff Manor, NY: Academy of Management.
5. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
6. Cohen, P.A. (1980). Effectiveness of student-rating feed-back for improving college instruction: A meta-analysis of findings. Reseach in Higher Education, 13, 321-341.
7. Conrad, K. J. (Ed.). (1994). Critically evaluating the role of experiments. New Directions for Program Evaluation, 63.
8. Cook, T. D. (1993). A theory of the generalization of causal relationships. New Directions for Program Evaluation, 57, 39-82.
9. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin.
10. Cook, T. D., Cooper, H., Cordray, D. S., Hartmann, H., Hedges, L. V., Light, R. J., Louis, T. A., & Mosteller, F. (Eds.). (1992). Meta-analysis for explanation: A case book. New York: Russell Sage Foundation.
11. Cook, T. D., & Leviton, L. C. (1980). Reviewing the literature: A comparison of traditional methods with meta-analysis. Journal of Personality, 48, 449-472.
12. Cook, T. D., & Shadish, W. R. (1994). Social experiments: Some developments over the past fifteen years. Annual Review of Psychology, 45, 545-580.
13. Cooper, H., & Hedges, L. V. (Eds.). (1994). The handbook of research synthesis. New York: Russell Sage Foundation.
14. Dennis, M. L. (1990). Assessing the validity of randomized field experiments: An example from drug abuse treatment research. Evaluation Review, 14, 347-373.
15. Efron, B. (1982). The jackknife, the bootstrap, and other resampling plans. Philadelphia, PA: Society for Industrial and Applied Mathematics.
16. El-Nemr, M. A. (1980). A meta-analysis of the outcomes of teaching biology as inquiry (Doctoral dissertation, University of Colorado, 1979). Dissertation Abstracts International, 40, 5813A. (University Microfilms No. 80-11274)
17. Ferguson, G. A. (1966). Statistical analysis in psychology and education (2nd ed.). New York: McGraw-Hill.
18. Fraker, T., & Maynard, R. (1987). The adequacy of comparison group designs for evaluations of employment-related programs. Journal of Human Resources, 22, 194-227.
19. Francis, T., Jr., Korns, R., Voight, R., Boisen, M., Memphill, F., Napier, J., & Tolchinski, A. (1955). An evaluation of the 1954 poliomyclitis vaccine trials: Summary report. American Journal of Public Health, 45 (Suppl.), 1-50.
20. Glass, G. V. (1976). Primary, secondary and meta-analysis of research. Educational Researcher, 5, 3-8.
21. Hays, W. L. (1988). Statistics (4th ed.). Fort Worth, TX: Holt, Rinehart & Winston.
22. Hedges, L. V. (1981). Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6, 107-128.
23. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
24. Heinsman, D. T., & Shadish, W. R. (1996). Assignment methods in experimentation: When do nonrandomized experiments approximate the answers from randomized experiments? Psychological Methods, 1, 154-169.
25. Hembree, R. (1985). Model for meta-analysis of research in education with a demonstration in mathematics education. Effects of hand held calculators (Doctoral dissertation, University of Tennessee, 1984). Dissertation Abstracts International, 45(10), 3087A. (University Microfilms No. 84-29597)
26. Henk, W. A., & Stahl, N. A. (1984, November). A meta-analysis of the effect of notetaking on learning from lecture. Paper presented at the meeting of the National Reading Conference, St. Petersburg Beach, FL. (ERIC Document Reproduction Service No. 258 533)
27. Horak, W. J. (1985, April). A meta-analysis of learning science concepts from textual materials. Paper presented at the meeting of the National Association for Research in Science Teaching, French Lick Springs, IN. (ERIC Document Reproduction Service No. 256 629)
28. Horon, P. F., & Lynn, D. D. (1980). Learning hierarchies research. Evaluation in Education, 4, 82-83.
29. Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
30. Kaufman, P. (1985). Meta-analysis of juvenile delinquency prevention programs. Unpublished master's thesis, Claremont Graduate School, Claremont, CA.
31. Kazdin, A. E. (Ed.). (1992). Methodological issues and strategies in clinical research. Washington, DC: American Psychological Association.
32. Klauer, K. J. (1984). Intentional and incidental learning with instructional texts: A meta-analysis for 1970-1980. American Educational Research Journal, 21, 323-339.
33. Kraemer, H. C., Gardner, C., Brooks, J. O., III., & Yesavage, J. A. (1998). Advantages of excluding underpowered studies in meta-analysis: Inclusionist versus exclusionist viewpoints. Psychological Methods, 3, 23-31.
34. LaLonde, R., & Maynard, R. (1987). How precise are evaluations of employment and training programs: Evidence from a field experiment. Evaluation Review, 11, 428-451.
35. Lipsey, M. W. (1990). Design sensitivity: Statistical power for experimental research. Newbury Park, CA: Sage.
36. Lipsey, M. W. (1992). Juvenile delinquency treatment: A meta-analytic inquiry into the variability of effects. In T. D. Cook, H. Cooper, D. S. Cordray, H. Hartmann, L. V. Hedges, R. J. Light, T. A. Louis, & F. Mosteller (Eds.), Meta-analysis for explanation: A casebook (pp. 83-127). New York: Russell Sage Foundation.
37. Lipsey, M. W. (1997). What can you build with thousands of bricks? Musings on the cumulation of knowledge in program evaluation. New Directions for Evaluation, 76, 7-24.
38. Lipsey, M. W., & Cordray, D. S. (2000). Evaluation methods for social intervention. Annual Review of Psychology, 51, 345-375.
39. Lipsey, M. W., & Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48, 1181-1209.
40. Lipsey, M. W., & Wilson, D. B. (1998). Effective intervention for serious juvenile offenders: A synthesis of research. In R. Loeber & D. Farrington (Eds.), Serious and violent juvenile offenders: Risk factors and successful interventions (pp. 313-395). Thousand Oaks, CA: Sage.
41. Luiten, J., Ames, W., & Ackerson, G. (1980). A meta-analysis of the effects of advance organizers on learning and retention. American Educational Research Journal, 4, 211-218.
42. Luiten, J. W. (1980). Advance organizers in learning. Evaluation in Education, 4, 49-50.
43. Lunneborg, C. E. (1985). Estimating the correlation coefficient: The bootstrap approach. Psychological Bulletin, 98, 209-215.
44. Lyday, N. L. (1984). A meta-analysis of the adjunct question literature (Doctoral dissertation, Pennsylvania State University, 1983). Dissertaton Abstracts International, 45(1), 129A. (University Microfilms No. 84-09065)
45. McNemar, Q. (1966). Psychological statistics (3rd ed.). New York: Wiley.
46. Meldrum, M. (1998). "A calculated risk": The Salk polio vaccine fields trials of 1954. British Medical Journal, 317, 1233-1236.
47. Mento, A. J., Steel, R. P., & Karren, R. J. (1987). A meta-analytic study of the effects of goal setting on task performance: 1966-1984. Organizational Behavior and Human Decision Processes, 39, 52-83.
48. Mooney, C. Z., & Duval, R. D. (1993). Bootstrapping: A nonparametric approach to statistical inference. Newbury Park, CA: Sage.
49. Moore, D. W., & Readence, J. H. (1984). A quantitative and qualitative review of graphic organizer research. Journal of Educational Research, 78, 11-17.
50. Ogles, B. M., Lambert, M. J., Weight, D. G., & Payne, I. R. (1990). Agoraphobia outcome measurement: A review and meta-analysis. Psychological Assessment, 2, 317-325.
51. Parham, J. L. (1983). A meta-anlaysis of the use of manipulative materials and student achievement in elementary school mathematics (Doctoral dissertation, Auburn University, 1983). Dissertation Abstracts International, 44(1), 96A. (University Microfilms No. 83-12477)
52. Powell, G. (1980, December). A meta-analysis of the effects of "imposed" and "induced" imagery upon word recall. Paper presented at the meeting of the National Reading Conference, San Diego, CA. (ERIC Document Reproduction Service No. 199 644)
53. Readence, J., & Moore, D.W. (1981). A meta-analytic review of the effect of adjunct pictures on reading comprehension. Psychology in the Schools, 18, 218-224.
54. *Redfield, D. L., & Rousseau, E. W. (1981). A meta-analysis of experimental research on teacher questioning behavior. Review of Educational Research, 51, 237-245.
55. "Roberts, A. R., & Camasso, M. J. (1991). The effect of juvenile offender treatment programs on recidivism: A meta-analysis. Notre Dame Journal of Law, Ethics & Public Policy, 5, 421-441.
56. Rosenthal, R. (1991). Meta-analytic procedures for social research (2nd ed.). Newbury Park, CA: Sage.
57. Samson, G. E., Borger, J. B., Weinstein, T., & Walberg, H. J. (1984). Pre-teaching experiences and attitudes: A quantitative synthesis. Journal of Research and Development in Education, 17, 52-56.
58. Schmidt, F. L. (1992). What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology. American Psychologist, 47, 1173-1181.
59. Schmidt, F. L. (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers. Psychological Method, 1, 115-129.
60. Shadish, W. R., Jr. (1992). Do family and martial psychotherapies change what people do? A meta-analysis of behavioral outcomes. In T. D. Cook, H. Cooper, D. S. Cordray, H. Hartmann, L. V. Hedges, R. J. Light, T. A. Louis, & F. Mosteller (Eds.), Meta-analysis for explanation: A casebook (pp. 129-208). New York: Russell Sage Foundation.
61. Shadish, W. R., & Ragsdale, K. (1996). Random versus nonrandom assignment in psychotherapy experiments: Do you get the same answer? Journal of Consulting and Clinical Psychology, 64, 1290-1305.
62. Smith, M. L., & Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 32, 752-760.
63. Stine, R. (1990). An introduction to bootstrap methods: Examples and ideas. In J. Fox & J. S. Long (Eds.), Modern methods of data analysis (pp. 325-373). Newbury Park, CA: Sage.
64. Stone, C. L. (1983). A meta-analysis of advance organizer studies. Journal of Experimental Education, 51, 194-199.
65. Sweitzer, G. L., & Anderson, R. D. (1983). A meta-analysis of research on science teacher education practices associated with inquiry strategy. Journal of Research in Science Teaching, 20, 453-466.
66. Tubbs, M. E. (1986). Goal setting: A meta-analytic examination of the empirical evidence. Journal of Applied Psychology, 71, 474-483.
67. Winer, B. J., Brown, D. R., & Michels, K. M. (1991). Statistical principles in experimental design (3rd ed.). New York: McGraw-Hill.
68. Wood, R. E., Mento, A. J., & Locke, E. A. (1987). Task complexity as a moderator of goal effects: A meta-analysis. Journal of Applied Psychology, 72, 416-425.
69. Yeaton, W. H., & Sechrest, L. (1981). Critical dimensions in the choice and maintenance of successful treatments: Strength, integrity, and effectiveness. Journal of Consulting and Clinical Psychology, 49, 156-167.