Indiscriminate data aggregations in meta-analysis: A cause for concern among policy makers and social scientists

Evaluation Review

Volumn 26, Issue 5, 2002, Pages 520-544

  Lopez Lee, David ^a  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EPIDEMIOLOGY; EVIDENCE BASED MEDICINE; HUMAN; MANAGEMENT; META ANALYSIS; METHODOLOGY; REPRODUCIBILITY; REVIEW; STANDARD; STATISTICAL ANALYSIS;

BIAS (EPIDEMIOLOGY); DATA INTERPRETATION, STATISTICAL; EVIDENCE-BASED MEDICINE; HUMANS; META-ANALYSIS; POLICY MAKING; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN;

EID: 0036780737     PISSN: 0193841X     EISSN: None     Source Type: Journal    
DOI: 10.1177/019384102236522     Document Type: Article

References (29)

1. Baltes, B. B., T. E. Briggs, J. W. Huff, J. A. Wright, and G. A. Neuman. 1999. Flexible and compressed workweek schedules: A meta-analysis. Journal of Applied Psychology 84 (4): 496-513.
2. Berk, R. A., and D. A. Freedman. 2002. Statistical assumptions as empirical commitments. In Law, punishment, and social control: Essays in honor of Sheldon Messinger (2d ed.), edited by T. Blomberg and S. Cohen, 1-24. Hawthorne, NY: Aldine de Gruyter.
3. Boruch, R. 1997. Randomized experiments for planning and evaluation. Thousand Oaks, CA: Sage.
4. Cameron, J., K. M. Banko, and W. D. Pierce. 1994. Pervasive negative effects of rewards on intrinsic motivation: The myth continues. Behavior Analyst 24 (1): 1-44.
5. Campbell, D. T. and J. C. Stanley. 1966. Experimental and quasi-experimental designs for research. Chicago: Rand McNally.
6. Chalmers, T. C., H. Smith, B. Blackburn, B. Silverman, B. Schroeder, D. Reitman, and A. Ambroz. 1981. A method for assessing the quality of a randomized control trial. Controlled Clinical Trials 2:31-49.
7. Clark, M., and A. D. Oxman. (Eds.). 2001. Cochrane Reviewers' Handbook 4.1.1 [Updated October 2001]. In: The Cochrane Library, Issue 4, 2001.
8. Cook, T. D., and D. T. Campbell. 1979. Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin.
9. Cooper, H., and L. V. Hedges. 1994. Potentials and limitations of research synthesis. In The handbook of research synthesis, edited by H. Cooper and L. V. Hedges, 521-29. New York: Russell Sage Foundation.
10. General Accounting Office. 1991. Designing evaluations. Program Evaluation and Methodology Division, May. United States General Accounting Office, PEMD-10.1.4.
11. Glass, G. V. 1976. Primary, secondary, and meta-analysis. Educational Researcher 5:3-8.
12. _. 1978. In defense of generalization. The Behavioral and Brain Sciences 3:394-95.
13. Glass, G. V., B. McGaw, and M. L. Smith. 1981. Meta-analysis in social research. Beverly Hills, CA: Sage.
14. Hedges, L. V., and I. Olkin. 1985. Statistical methods for meta-analysis. Orlando, FL: Academic Press.
15. Hunter, J. E., and F. L. Schmidt. 1990. Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
16. Hunter, J. E., F. L. Schmidt, and G. B. Jackson. 1982. Meta-analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage.
17. Jadad, A. R., R. A. Moore, D. Carroll, C. Jenkinson, D. J. M. Reynolds, D. J. Gavaghan, and H. J. McQuay. 1996. Assessing the quality of randomized clinical trials: Is blinding necessary? Controlled Clinical Trials 17:1-12.
18. Jenkins, G.D., Jr., A. Mitra, N. Gupta, and J. D. Shaw. 1998. Are financial incentives related to performance? A meta-analytic review of empirical research. Journal of Applied Psychology 83 (5): 777-87.
19. Lee, D. M. 1969. Does Color Enhance Short Term Memory? Ph.D. diss., University of California, Los Angeles.
20. Lepper, M., J. Henderlong, and I. Gingras. 1999. Understanding the effects of extrinsic rewards on intrinsic motivation - uses and abuses of meta-analysis: Comment on Deci, Koestner, and Ryan (1999). Psychological Bulletin 125 (6): 669-76.
21. Matt, G. E., and T. D. Cook. 1994. Threats to the validity of research synthesis. In The handbook of research synthesis, edited by H. Cooper and L. V. Hedges, 503-20. New York: Russell Sage Foundation.
22. Orr, L. L. 1999. Social experiments: Evaluating public programs with experimental methods. Thousand Oaks, CA: Sage.
23. Ostroff, C., and D. A. Harrison. 1999. Meta-analysis, level of analysis, and best estimates of population correlations: Cautions for interpreting meta-analytic results in organizational behavior. Journal of Applied Psychology 84 (2): 260-70.
24. Porter, A. L., T. Connolly, R. G. Heikes, and C. Y. Park. 1981. Misleading indicators: The limitations of multiple linear regression in formulation of policy recommendations. Policy Sciences 13:397-418.
25. Robertson, P., and S. J. Seneviratne. 1996. Outcomes of planned organizational change in the public sector: A meta-analytic comparison to the private sector. Public Administration Review 55 (6): 547-58.
26. Rosenthal, R. 1984. Meta-analytic procedures for social research. Beverly Hills, CA: Sage.
27. Schmidt, F. L. & Hunter, J. E. 1984. A within setting test of the situational specificity hypothesis in personnel selection. Personnel Psychology 37: 317-26.
28. Wortman, P. M. 1994. Judging research quality. In The handbook of research synthesis, edited by H. Cooper and L. V. Hedges, 97-109. New York: Russell Sage Foundation.
29. Wortman, P. M., and W. H. Yeaton. 1983. Synthesis of results in controlled trials of coronary artery bypass graft surgery. Evaluation Studies Review Annual 8:536-51.