Incorporating quality scores in meta-analysis

Journal of Educational and Behavioral Statistics

Volumn 36, Issue 5, 2011, Pages 555-585

  Ahn, Soyeon ^a   Becker, Betsy Jane ^b  

^a UNIVERSITY OF MIAMI   (United States)

^b FLORIDA STATE UNIVERSITY   (United States)

Author keywords

deviated effect size; effect size (ES); meta analysis; quality score weights; standardized mean difference (d effect size); study quality

Indexed keywords

EID: 80054005730     PISSN: 10769986     EISSN: None     Source Type: Journal    
DOI: 10.3102/1076998610393968     Document Type: Article

References (72)

1. Abele-Horn M.,Dauber A.,Bauernfeind A.,Russwurm W.,Seyfarth-Metzger I.,Gleich P.,Ruckdeschel G.Decrease in nosocomial pneumonia in ventilated patients by selective oropharyngeal decontamination.Intensive Care Medicine. 1996;23:187-195.
2. Balk E.M.,Bonis P.A. L.,Moskowitz H.,Schmid C.H.,Ioannidis J.P. A.,Wang C.,Lau J.Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials.Journal of the American Medical Association. 2002;287:2973-2982.
3. Bangert-Drowns R.L.,Kulik C.-L. C.,Kulik J.A.,Morgan M.T.The instructional effect of feedback in test-like events.Review of Educational Research. 1991;61:213-238.
4. Bérard A.,Andreu N.,Tetrault J.-P.,Niyonsenga T.,Myhal D.Reliability of Chalmers' scale to assess quality in meta-analyses on pharmacological treatments for osteoporosis.Annals of Epidemiology. 2000;10:498-503.
5. Bérard A.,Bravo G.Combining studies using effect sizes and quality scores: Application to bone loss in postmenopausal woman.Journal of Clinical Epidemiology. 1998;51:801-807.
6. Bérard A.,Bravo G.,Gauthier P.Meta-analysis of the effectiveness of physical activity for the prevention of bone loss in postmenopausal women.Osteoporosis International. 1997;7:331-337.
7. Blair P.,Rowlands B.J.,Lowry K.,Webb H.,Armstrong P.,Smilie J.Selective decontamination of the digestive tract: A stratified, randomized, prospective study in a mixed intensive care unit.Surgery. 1991;110:303-310.
8. Bloch D.A.,Moses L.E.Nonoptimally weighted least squares.The American Statistician. 1988;42:50-53.
9. Bollini P.,Garcia Rodriguez L.A.,Perez Cutthann S.The impact of research quality and study design on epidemiologic estimates of the effect of nonsteroidal antiu{ogonek}inflammatory drugs on upper gastrointestinal tract disease.Archives of Internal Mediu{ogonek}cine. 1992;152:1289-1295.
10. Brun-Buisson C.,Legrand P.,Rauss A.,Richard C.,Montravers F.,Besbes M.Intestinal decontamination for control of nosocomial multiresistant Gram-negative bacilli: Study of an outbreak in an intensive care unit.Annals of Internal Medicine. 1989;110:873-881.
11. Cerra F.B.,Maddaus M.A.,Dunn D.L.,Wells C.L.,Konstantinides N.N.,Lehmann S.L.,Mann H.J.Selective gut decontamination reduces nosocomial infections and length of stay but not mortality or organ failure in surgical intensive care unit patients.Archives of Surgery. 1994;127:163-169.
12. Cockerill F.R. III.,Muller S.R.,Anhalt J.P.,Marsh H.,Farnell M.,Mucha P.,Thompson R.Prevention of infection in critically ill patients by selective decontamination of the digestive tract.Annals of Internal Medicine. 1992;117:545-553.
13. Colditz G.A.,Miller J.N.,Mosteller F.How study design affects outcomes in comparisons of therapy.Statistics in Medicine. 1989;8:441-454.
14. Detsky A.S.,Naylor C.D.,O'Rourke K.,McGeer A.J.,L'Abbé K. A.Incorporating variations in the quality of individual randomized trials into metaanalysis.Journal of Clinical Epidemiology. 1992;45:255-265.
15. DuMouchelW. (1994). Hierarchical Bayes linear models for meta-analysis. Technical Report 27. Research Triangle Park, NC: National Institute of Statistical Sciences. Downloaded from http://www.niss.org/sites/default/files/pdfs/technicalreports/tr27.pdf.
16. Eddy D.M.,Hasselblad V.,Shachter R.Meta-analysis by the confidence profile method: The statistical synthesis of evidence. San Diego, CA: Academic Press; 1992:.
17. Emerson J.D.,Burdick E.,Hoaglin D.C.,Mosteller F.,Chalmers T.C.An empirical study of the possible relation of treatment differences to quality scores in controlled randomized clinical trials.Controlled Clinical Trials. 1990;11:329-352.
18. Falchikov N.,Goldfinch J.Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks.Review of Educational Research. 2000;70:287-322.
19. Ferrer M.,Torres A.,Gonzalez J.,Puig D.,el-Ebiary M.,Roca M.,Rodriguez-Roisin R.Utility of selective digestive decontamination in mechanically ventilated patients.Annals of Internal Medicine. 1994;120:389-395.
20. Fleiss J.L.,Gross A.J.Meta-analysis in epidemiology, with special reference to studies of the association between exposure to environmental tobacco smoke and lung cancer: A critique.Journal of Clinical Epidemiology. 1991;44:127-139.
21. Gastinne H.,Wolff M.,Delatour F.,Faurisson F.,Chevret S.A controlled trial in intensive care units of selective decontamination of the digestive tract with nonu{ogonek}absorbable antibiotics.New England Journal of Medicine. 1992;326:594-599.
22. Gaussorgues P.,Salord F.,Sirodot M.,Tigaud S.,Cagnin S.,Gerard M.,Robert D.Efficacitéde la décontamination digestive sur la survenue des bacté rié mies nosocomiales chez les patients sous ventilation méanique et recevant des betamiméu{ogonek}tiques.Réanimation, Soins Intensifs, Médecine d'ugence. 1991;7:169-174.
23. Godard J.,Guillaume C.,Reverdy M.E.,Bachmann P.,Bui-Xuan B.,Nageotte A.,Motin J.Intestinal decontamination in a polyvalent ICU: a doubleblind study.Intensive Care Medicine. 1990;16:307-311.
24. Greenland S.,O'Rourke K.On the bias produced by quality scores in meta-analysis, and a hierarchical view of proposed solutions.Biostatistics. 2001;2:463-471.
25. Hammond J.M. J.,Potgieter P.D.,Saunders G.L.,Forder A.A.Double-blind study of selective decontamination of the digestive tract in intensive care.Lancet. 1992;340:5-9.
26. Hartenauer U.,Thülig B.,Diemer W.,Lawin P.,Fegeler W.,Kehrel R.,Ritzerfeld W.Effect of selective flora suppression on colonization, infection, and mortalu{ogonek}ity in critically ill patients: A one-year, prospective consecutive study.Critical Care Medicine. 1991;19:463-473.
27. Hedges L.V.Estimation of effect size from a series of independent experiments.Psychological Bulletin. 1982;92:490-499.
28. Hedges L.V.,Olkin I.Orlando. FL: Academic Press; 1985:.
29. Herbison P.,Hay-Smith J.,Gillespie W.J.Adjustment of meta-analyses on the basis of quality scores should be abandoned.Journal of Clinical Epidemiology. 2006;59:1249-1256.
30. Hunter J.E.,Schmidt F.L.Methods of meta-analysis: Correcting error and bias in research findings. 2004). Methods of meta-analysis: Correcting error and bias in research findings(2nd ed.). Thousands Oaks, CA: Sage. Thousands Oaks, CA: Sage; 2004:.
31. Jacobs S.,Foweraker J.E.,Roberts S.E.Effectiveness of selective decontau{ogonek}mination of the digestive tract in an ICU with a policy encouraging a low gastric pH.Clinical Intensive Care. 1992;3:52-58.
32. Jüni P.,Altman D.G.,Egger M.Assessing the quality of controlled clinical trials.British Medical Journal. 2001;323:42-46.
33. Jüni P.,Witschi A.,Bloch R.,Egger M.The hazards of scoring the quality of clinical trials for meta-analysis.Journal of the American Medical Association. 1999;282:1054-1060.
34. Kerver A.,Rommes J.H.,Mevissen-Verhage E.,Hulstaert P.F.,Vos A.,Verhoef J.,Wittebol P.Prevention of colonization and infection in critically ill patients: A prospective randomized study.Critical Care Medicine. 1988;16:1087-1093.
35. Korinek A.M.,Laisne M.J.,Nicolas M.H.,Raskine L.,Deroin V.,Sanson-Lepors M.J.Selective decontamination of the digestive tract in neurosurgical intensive care unit patients: A double-blind, randomized, placebo-controlled study.Critical Care Medicine. 1993;21:1466-1473.
36. Kunz R.,Oxman A.The unpredictability paradox: Review of empirical comparisons of randomized and non-randomized clinical trials.British Medical Journal. 1998;317:1185-1190.
37. Laggner A.N.,Tryba M.,Georgopoulos A.,Lenz K.,Grimm G.,Graninger W.,Druml W.Oropharyngeal decontamination with gentamicin for long-term ventilated patients on stress ulcer prophylaxis with sueralfate.Wien Klin Wochenschr. 1994;106:15-19.
38. Ledingham I.,Eastaway A.,McKay I.C.,Alcock S.R.,McDonalds J.C.,Ramsay G.Triple regimens of selective decontamination of the digestive tract, systemic cefotaxime, and microbiological surveillance for prevention of acquired infection in intensive care.The Lancet. 1988;331:785-790.
39. Linde K.,Scholz M.,Ramierz G.,Clausius N.,Melchart D.,Jonas W.B.Impact of study quality on outcome in placebo-controlled trials of homeopathy.Journal of Clinical Epidemiology. 1999;52:631-636.
40. Lingnau W.,Berger J.,Javorsky F.,Lejeune P.,Mutz N.,Benzer H.Selective intestinal decontamination in multiple trauma patients: Prospective, controlled trial.The Journal of Trauma. 1997;42:687-694.
41. McClelland P.,Murray A.,Williams P.,Van Saene H.,Gilbertson A.,Monstafa S.,Bone J.Reducing sepsis in severe combined acute renal and respiratory failure by selective decontamination of the digestive tract.Critical Care Medicine. 1990;18:935-939.
42. Moher D.,Cook D.J.,Jadad A.R.,Tugwell P.,Moher M.,Jones A.,Klassen T.P.Assessing the quality of reports of randomized trials: Implications for the conu{ogonek}duct of meta-analysis. Southampton, United Kingdom: University of Southampton, The National Coordinating Center for Health Technology Assessment; 1999:.
43. Olivo S.A.,Macedo L.G.,Gadotti I.C.,Fuentes J.,Stanton T.,Magee D.J.Scales to assess the quality of randomized controlled trials: A systematic review.Physical Therapy. 2008;88:156-175.
44. Ortiz Z.,Shea B.,Suarez A.M. E.,Moher D.,Wells G.A.,Tugwell P.The efficacy of folic acid and folinic acid in reducing methotrexate gastrointestinal toxicity in rheumatoid arthritis: A meta-analysis of randomized controlled trials.The Journal of Rheumatology. 1998;25:36-43.
45. Palomar M.,Alvarez-Lerma F.,Jorda R.,Bermejo B.Prevention of nosocou{ogonek}mial infection in mechanically ventilated patients: Selective digestive decontamination versus sueralfate.Clinical Intensive Care. 1997;8:228-235.
46. Pugin J.,Auckenthaler R.,Lew D.P.,Suter P.M.Oropharyngeal decontamiu{ogonek}nation decreases incidence of ventilator-associated pneumonia: A randomized, placebo-controlled, double-blind clinical trial.JAMA. 1991;265:2704-2710.
47. Quinio B.,Albanese J.,Bues-Charbit M.,Viviand X.,Martin C.Selective decontamination of the digestive tract in multiple trauma patients: A prospective double-blind, randomized, placebo-controlled study.Chest. 1996;109:765-772.
48. Rocha L.A.,Martin M.J.,Pita S.,Paz J.,Seco C.,Margusino L.,Durán M.Prevention of nosocomial infection in critically ill patients by selective decontamination of the digestive tract: A randomized, double blind, placebo-controlled study.lntensive Care Medicine. 1992;18:398-404.
49. Rodriguez-Roldán J.,Alma-Cuesta A.,López A.,Carrillo A.,Garcia J.,León J.,Martínez-Pellus A.Prevention of nosocomial lung infection in ventilated patients: Use of an antimicrobial pharyngeal nonabsorbable paste.Critical Care Medicine. 1990;18:1239-1242.
50. Rubin D.B.Meta-analysis: Literature synthesis or effect-size surface estimation?.Journal of Educational and Behavioral Statistics. 1992;17:363-374.
51. Sanchez Garcia M.,Cambronero Galache J.A.,Lopez Diaz J.,Cerda Cerda E.,Rubio Blasco J.,Gomez Aguinaga M.,Sacristan del Castillo J.Effectiveness and cost of selective decontamination of the digestive tract in critically ill intubated patients. A randomized, double-blind, placebo-controlled, multicenter trial.American Journal of Respiratory and Critical Care Medicine. 1998;158:908-916.
52. Schulz K.F.,Chalmers I.,Hayes R.J.,Altman D.G.Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials.Journal of the American Medical Association. 1995;273:408-412.
53. Shadish W.R.,Haddock C.K.The handbook of research synthesis. Cooper H.Hedges L.V., ed. New York, NY: Russell Sage Foundation; 1994:261-281.
54. Singer G.H. S.Meta-analysis of comparative studies of depression in mothers of children with and without developmental disabilities.American Journal on Mental Retardation. 2006;111:155-169.
55. Slavin R.E.What works? Issues in synthesizing educational program evaluau{ogonek}tions.Educational Researcher. 2008;37:5-14.
56. Swanson H.L.,Hoskyn M.Experimental intervention research on students with learning disabilities: A meta-analysis of treatment outcomes.Review of Educau{ogonek}tional Research. 1998;68:277-231.
57. Stoutenbeek C.,van Saene H.,Miranda D.,Zandstra D.,Binnedij K.The effect of selective decontamination of the digestive tract on colonization and infection rate in multiple trauma patients.Intensive Care Medicine. 1984;10:185-192.
58. Treadwell J.R.,Tregear S.J.,Reston J.T.,Turkelson C.M.A system for rating the stability and strength of medical evidence.BMC Medical Research Methodology. 2006;6:1-20.
59. Tritchler D.Modeling study quality in meta-analysis.Statistics in Medicine. 1999;10:2135-2145.
60. Ulrich Harinck-de Weerd J.E.,Bakker N.C.,Jacz K.,Doornbos L.,de Ridder V.A.Selective decontamination of the digestive tract with norfloxacin in the prevention of ICU-acquired infections: A prospective randomized study.Intensive Care Medicine. 1989;15:424-431.
61. Unertl K.,Ruckdeschel G.,Selbmann H.K.,Jensen U.,Forst H.,Lenhart F.P.,Peter K.Prevention of colonization and respiratory infections in long-term ventilated patients by local antimicrobial prophylaxis.Intensive Care Medicine. 1987;13:106-113.
62. Valentine J.C.,Cooper H.A systematic and transparent approach for assessing the methodological quality of intervention effectiveness research: The Study Design and Implementation Assessment Device (Study DIAD).Psychological Methods. 2008;13:330-349.
63. van den Noortgate W.,Onghena P.Estimating the mean effect size in meta-analysis: Bias, precision, and mean squared error of different weighted methods.Behavior Research Methods, Instruments, & Concepts. 2003;35:504-511.
64. van Nieuwenhoven C.A.,Buskens E.,van Tiel F.H.,Bonten M.J. M.Relationship between methodological trial quality and the effects of selective digestive decontamination on pneumonia and mortality in critically ill patients.Journal of American Medical Association. 2001;286:335-340.
65. Verwaest C.,Verhaegen J.,Ferdinande P.,Schetz M.,Van den Berghe G.,Verbist L.,Lauwers P.Randomized, controlled trial of selective digestive decontaminau{ogonek}tion in 600 mechanically ventilated patients in a multidisciplinary intensive care unit.Critical Care Medicine. 1997;25:63-71.
66. Wells K.,Littell J.H.Study quality assessment in systematic reviews of research on intervention effects.Research on Social Work Practice. 2009;19:52-62.
67. White K.,Taylor M.,Moss V.Does research support claims about the benefits of involving parents in early intervention programs?.Review of Educational Research. 1992;62:91-125.
68. Whiteside M.F.,Becker B.J.Parental factors and the young child's postdivorce adjustment: A meta-analysis with implications for parenting arrangements.Journal of Family Psychology. 2000;14:5-26.
69. Whiting P.,Harbord R.,Kleijnen J.No role for quality scores in systematic reviews of diagnostic accuracy studies.BMC Medical Research Methodology. 2005;5:1-9.
70. Whiting P.,Rutjes A.W. S.,Dinnes J.,Reitsma J.B.,Bossuyt P.M. M.,Kleijnen J.A systematic review finds that diagnostic reviews fail to incorporate quality despite available tools.Journal of Clinical Epidemiology. 2005;58:1-12.
71. Wiener J.,Itokazu G.,Nathan C.,Kabins S.A.,Weinstein P.A.A randomized, double-blind, placebo-controlled trial of selective decontamination in a medical-surgical intensive care unit.Clinical Infectious Diseases. 1995;20:861-867.
72. Winter R.,Humphreys H.,Pick A.,MacGowan A.P.,Willatts S.M.,Speller D.C. E.A controlled trial of selective decontamination of the digestive tract in intensive care and its effect on nosocomial infection.The Journal of Antimicrobial Chemotherapy. 1992;30:73-87.