Combining randomized and nonrandomized evidence in clinical research: A review of methods and applications

Research Synthesis Methods

Volumn 6, Issue 1, 2015, Pages 45-62

  Verde, Pablo E ^a   Ohmann, Christian ^a  

^a HEINRICH HEINE UNIVERSITY   (Germany)

Author keywords

Bias modeling; Cross design synthesis; Generalized evidence synthesis; Hierarchical Bayesian models; Network meta analysis; Observational studies; Randomized control trials

Indexed keywords

BAYES THEOREM; BIOSTATISTICS; EVIDENCE BASED MEDICINE; HUMAN; META ANALYSIS (TOPIC); PRACTICE GUIDELINE; PROCEDURES; RANDOMIZED CONTROLLED TRIAL (TOPIC); REGRESSION ANALYSIS; STATISTICAL BIAS; STATISTICAL MODEL; STATISTICS AND NUMERICAL DATA;

BAYES THEOREM; BIAS (EPIDEMIOLOGY); BIOSTATISTICS; EVIDENCE-BASED MEDICINE; HUMANS; LIKELIHOOD FUNCTIONS; META-ANALYSIS AS TOPIC; MODELS, STATISTICAL; PRACTICE GUIDELINES AS TOPIC; RANDOMIZED CONTROLLED TRIALS AS TOPIC; REGRESSION ANALYSIS;

EID: 84938693553     PISSN: None     EISSN: 17592887     Source Type: Journal    
DOI: 10.1002/jrsm.1122     Document Type: Article

References (124)

1. Abdelhamid A, Loke Y, Parekh-Bhurke S, Chen Y, Sutton A, Eastwood A, Holland R, Song F. 2012. Use of indirect comparison methods in systematic reviews: a survey of Cochrane review authors. Research Synthesis Methods 3: 71-79.
2. Adar R, Critchfield G, Eddy D. 1989. A confidence profile analysis of the results of femoropopliteal percutaneous transluminal angioplasty in the treatment of lower-extremity ischemia. Journal of Vascular Surgery 10: 57-67.
3. Ades A. 2003. A chain of evidence with mixed comparisons: models for multi-parameter synthesis and consistency of evidence. Statistics in Medicine. 22: 2995-3016.
4. Ades A, Sutton A. 2006. Multiparameter evidence synthesis in epidemiology and medical decision-making: current approaches. Journal of the Royal Statistical Society: Series A (Statistics in Society) 169: 5-35.
5. Agostino R. 1998. Tutorial in biostatistics propensity score methods for bias reduction in the comparison of a treatment to a non-randomised control group. Statistics in Medicine 2281: 2265-2281.
6. Ahern J, Hubbard A, Galea S. 2009. Estimating the effects of potential public health interventions on population disease burden: a step-by-step illustration of causal inference methods. American Journal of Epidemiology 169: 1140-1147.
7. Anonymous. 1992. Cross design synthesis: a new strategy for studying medical outcomes? The Lancet 944-946.
8. Austenfeld M, Thompson I, Middleton R. 1994. Meta-analysis of the literature: guideline development for prostate cancer treatment. The Journal of Urology 152: 1866-1869.
9. Begg C. 1992. Book review: cross design synthesis: a new strategy for medical effectiveness research. Statistics in Medicine 11: 1627-1630.
10. Begg C, Pilote L. 1991. A model for incorporating historical controls into a meta-analysis. Biometrics 47: 899-906.
11. Benson K, Hartz A. 2000. A comparison of observational studies and randomized, controlled trials. The New England Journal of Medicine 342: 1878-1886.
12. Bittl JA, He Y, Jacobs AK, Yancy CW, Normand S-LT. 2013. Bayesian methods affirm the use of percutaneous coronary intervention to improve survival in patients with unprotected left main coronary artery disease. Circulation 127: 2177-2185.
13. Block P, Peterson E, Krone R, Kesler K, Hannan E, O'Connor G, Detre K. 1998. Identification of variables needed to risk adjust outcomes of coronary interventions: evidence-based guidelines for efficient data collection. JACC 32: 275-82.
14. Brumback BA, Holmes LB, Ryan LM. 1999. Adverse effects of chorionic villus sampling: a meta-analysis. Statistics in Medicine 18, 2163-2175.
15. Chelimsky E, Silberman G, Droitcour J. 1993. Cross design synthesis. The Lancet 341, 498.
16. Clowse M, Behera M, Anders C, Copland S, Coffmann C, Leppert P, Bastian L. 2009. Ovarian preservation by GnRH agonists during chemotherapy: a meta-analysis. Journal of Women's Health 18(3): 311-319.
17. Cole S, Stuart, E. 2010. Generalizing evidence from randomized clinical trials to target populations: the ACTG 320 trial. American Journal of Epidemiology 172, 107-115.
18. Copas J. 2013. A likelihood-based sensitivity analysis for publication bias in meta-analysis. Journal of the Royal Statistical Society: Series C (Applied Statistics) 62: 47-66.
19. Craxi A, Camma C, Giunta M. 1999. Definition of response to antiviral therapy in chronic hepatitis C. Journal of Hepatology 31: 160-167.
20. Craxi A, DiBona D, Camma C. 2003. Interferon-alpha for HBeAg-positive chronic hepatitis B. Journal of Hepatology 39, 99-105.
21. Deeks JJ, Dinnes J, D'Amico R, Sowden AJ, Sakrovitch C, Song F, Petticrew M, Altman DG. 2003. Evaluating nonrandomised intervention studies. Health Technology Assessment NHS R&D HTA Programme 7(27): 1-173.
22. Dias S, Welton N, Caldwell J, Ades A. 2010. Checking consistency in mixed treatment comparison meta-analysis. Statistics in Medicine 29: 932-944.
23. Dias S, Sutton A, Ades A, Welton N. 2013. Evidence synthesis for decision making 2: a generalized linear modeling framework for pairwise and network meta-analysis of randomized controlled trials. Medical Decision Making 33: 607-617.
24. Dominici F, Parmigiani G, Wolpert R, Hasselblad V. 1999. Meta-analysis of migraine headache treatments: combining information from heterogeneous designs. Journal of the American Statistical Association 94: 16-28.
25. Donegan S, Williamson P, D'Alessandro U, Smith CT. 2012. Assessing the consistency assumption by exploring treatment by covariate interaction in mixed treatment comparison meta-analysis: individual patient level covariate versus aggregate trial level covariates. Statistics in Medicine 31: 3840-3857.
26. Donegan S, Williamson P, D'Alessandro U, Garner P, Smith CT. 2013. Combining individual patient data and aggregate data in mixed treatment comparison meta-analysis: individual patient data may be beneficial if only for a subset of trials. Statistics in Medicine 32: 914-930.
27. Droitcour J, Silberman G, Chelimsky E. 1993. A new form of meta-analysis for combining results from randomized clinical trials and medical-practice databases. International Journal of Technology Assessment in Health Care 9: 440-9.
28. Eddy DM. 1989. The confidence profile method - a Bayesian method for assessing health technologies." Operations Research 37: 210-28.
29. Eddy DM. 2013. "Top 10 projects. The Confidence Profile Method." Available from: http://www.davidmeddy.com/ Top10projects.htm.
30. Eddy DM, Hasselblad V, Shachter R. 1990a. A Bayesian method for synthesizing evidence. The confidence profile method." International Journal of Technology Assessment in Health Care 6: 31-55.
31. Eddy DM, Hasselblad V, Shachter R. 1990b. An introduction to a Bayesian method for meta-analysis: the confidence profile method. Medical Decision Making 10: 15-23.
32. Eddy DM, Hasselblad V, Shachter R. 1992. Meta-analysis by the confidence profile method: the statistical synthesis of evidence. Academic Press, San Diego, CA.
33. Egger M, Ebrahim S, Davey Smith G. 2002. Where now for meta-analysis? International Journal of Epidemiology 31: 1-5.
34. Evans J, Green J, Carlin P, Barrett L. 1995. Meta-analysis of antibiotic in tube thoracostomy. The American Surgeon 61: 215-9.
35. Fortin M, Dionne J, Pinho G, Gignac J, Lapointe L. 2006. Randomized controlled trials: do they have external validity for patients with multiple comorbidities? Annals Of Family Medicine 4: 104-108.
36. Frangakis C. 2009. The calibration of treatment effects from clinical trials to target populations. Clinical Trials 6: 136-140.
37. Greenhouse J, Kaizar E, Kelleher K, Seltman H, Gardner W. 2008. Generalizing from clinical trial data: a case study. The risk of suicidality among pediatric antidepressant users. Statistics in Medicine 27: 1801-1813.
38. Greenland S. 2005. Multiple-bias modelling for analysis of observational data. Journal of the Royal Statistical Society: Series A (Statistics in Society) 168: 267-306.
39. Grines C, Nelson T, Safian R, Hanzel G, Goldstein J, Dixon S. 2008. A Bayesian meta-analysis comparing AngioJet® thrombectomy to percutaneous coronary intervention alone in acute myocardial infarction. Journal of Interventional Cardiology 21: 459-482.
40. Hall M, Chang S, Dalbagni G, Pruthi R, Seigne J, Skinner E, Wolf J, Schellhammer P. 2007. Guideline for the management of nonmuscle invasive bladder cancer (stages Ta, T1 and Tis): 2007 update. The Journal of Urology 178: 2314-2330.
41. Hasselblad V. 1998. Meta-analysis of multi-treatment studies. Medical Decision Making 18: 37-43.
42. Higgins J, Whitehead A. 1996. Borrowing strength from external trials in a meta-analyses. Statistics in Medicine 15: 2733-2749.
43. Higgins J, Thompson S, Spiegelhalter D. 2009. A re-evaluation of random-effects meta-analysis. Journal of Royal Statistical Society: Series A 172: 137-159.
44. Higgins JPT, Jackson D, Barrett JK, Lu G, Ades AE, White IR. 2012. Consistency and inconsistency in network metaanalysis: concepts and models for multi-arm studies. Research Synthesis Methods 3: 98-110.
45. Higgins J, Ramsay C, Reeves B, Deeks J, Shea B, Valentine J, Tugwell P, Wells G. 2013. Issues relating to study design and risk of bias when including non-randomized studies in systematic reviews on the effects of interventions. Research Synthesis Methods 4: 12-25.
46. Hinkelmann K, Kempthorne O. 1994. Design and analysis of experiments. Volume I: introduction to experimental design. John Wiley & Sons, New York.
47. Huang J, van Gelder J. 2002. The probability of sudden death from rupture of intracranial aneurysms: a metaanalysis. Neurosurgery 51: 1001-1007.
48. Hurwitz E, Ake P, Adams A, Meeker W, Shekelle P. 1996. Manipulation and mobilization of the cervical spine. A systematic review of the literature. Spine (Phila Pa 1976) 21(15): 1746-59.
49. Ibrahim J, Chen M. 2000. Power prior distributions for regression models. Statistica science 15: 46-60.
50. Imai K, King G, Stuart E 2008. Misunderstandings between experimentalists and observationalists about causal inference. Journal of the Royal Statistical Society: Series A (Statistics in Society) 171: 481-502.
51. Ioannidis J. 2010. Meta-research: the art of getting it wrong. Research Synthesis Methods 1: 169-184.
52. Jackson C, Best N, Richardson S. 2006. Improving ecological inference using individual-level data. Statistics in Medicine 25: 2136-2159.
53. Jackson C, Best N, Richardson S. 2008. Hierarchical related regression for combining aggregate and individual data in studies of socio-economic disease risk factors. Journal of the Royal Statistical Society: Series A (Statistics in Society) 171: 159-178.
54. Jackson C, Best N, Richardson S. 2009. Bayesian graphical models for regression on multiple data sets with different variables. Biostatistics 10: 335-351.
55. Jones W, Schmit K, Vemulapalli S, Subherwal S, Patel M, Hasselblad V, Heidenfelder B, Chobot M, Posey R, Wing L, Sanders G, Dolor R. 2013 May. Treatment strategies for patients with peripheral artery disease. Rockville (MD): Agency for Healthcare Research and Quality (US) Report No.: 13-EHC090-EF.
56. Kaizar E. 2011. Estimating treatment effect via simple cross design synthesis. Statistics in Medicine 30: 2986-3009.
57. Klotzbuecher C, Ross P, Landsman P, Abbott T, Berger M. 2000. Patients with prior fractures have an increased risk of future fractures: a summary of the literature and statistical synthesis. Journal of Bone and Mineral Research 15(4): 721-39.
58. Larose D, Dey D. 1997. Grouped random effects models for Bayesian meta-analysis. Statistics in Medicine 16: 1817-1829.
59. Leach GE, Dmochowski RR, Appell RA. 1997. Female stress urinary incontinence clinical guidelines panel summary report on surgical management of female stress urinary incontinence. Journal of Urology 158: 875-880.
60. Lefevre F, Aronson N. 2000. Ketogenic diet for the treatment of refractory epilepsy in children: a systematic review of efficacy. Pediatrics 105(4): E46.
61. Lehmann H, Hinton R, Morello P, Santoli J. 2000. Developmental dysplasia of the hip practice guideline: technical report. Pediatrics 105: 896-905.
62. Li Z, Begg C. 1994. Random effects models for combining results from controlled and uncontrolled studies in a meta-analysis. Journal of the American Statistical Association 89: 1523-1527.
63. Licata A, DiBona D, Schepis F, Shahied L, Craxi A, Camma C. 2003. When and how to treat acute hepatitis C?" Journal of Hepatology 39: 1056-1062.
64. Lu G, Ades A. 2004. Combination of direct and indirect evidence in mixed treatment comparisons. Statistics in Medicine 23. 3105-3124.
65. Lu G, Ades A. 2006. Assessing evidence inconsistency in mixed treatment comparisons. Journal of the American Statistical Association 101(474): 447-459.
66. Lu G, Ades A. 2009. Modelling between-trial variance structure in mixed treatment comparisons. Biostatistics 10: 792-805.
67. Lu G, Welton NJ, Higgins JPT, White IR, Ades AE. 2012. Linear inference for mixed treatment comparison metaanalysis: a two-stage approach. Research Synthesis Methods 2(1): 43-60.
68. Lumley T. 2002. Network meta-analysis for indirect treatment comparisons. Statistics in Medicine 21(16): 2313-2324.
69. Lunn D, Spiegelhalter D, Thomas A, Best N. 2009. The BUGS project: evolution, critique and future directions. Statistics in Medicine 28(25): 3049-3067.
70. Lunn D, Barrett J, Sweeting M, Thompson S. 2013. Fully Bayesian hierarchical modelling in two stages, with application to meta-analysis. Journal of the Royal Statistical Society: Series C: Applied Statistics 62(4): 551-572.
71. Marshall E, Spiegelhalter D. 2007. Identifying outliers in Bayesian hierarchical models: a simulation-based approach. Bayesian Analysis 2: 409-444.
72. McCarron C, Pullenayegum E, Thabane L, Goerree R, Tarride JE. 2010. The importance of adjusting for potential confounders in Bayesian hierarchical models synthesis evidence from randomized and non-randomized studies: an application comparing treatments for abdominal aortic aneurysms. BMC Medical Research Methodology 10: 64.
73. McCarron C, Pullenayegum E, Thabane L, Goerree R, Tarride JE. 2011. Bayesian hierarchical models combining different study types and adjusting for covariate imbalances: a simulation study to assess model performance. Plos One 6(10)
74. Mesgarpour B, Heidinger B, Schwameis M, Kienbacher C, Walsh C, Schmitz S, Herkner H. 2013. Safety of off-label erythropoiesis stimulating agents in critically ill patients: a meta-analysis. Intensive Care Medicine 39(11): 1896-908.
75. Molitor J, Jackson C, Best N, Richardson S. 2009. Using Bayesian graphical models to model biases in observational studies and to combine multiple data sources: application to low birth weight and water. Journal of the Royal Statistical Society: Series A 32-50 172(3): 615-637.
76. Montague D, Barada JH, Belker AM. 1996. Clinical guidelines panel on erectile dysfunction: summary report on the treatment of organic erectile dysfunction. Journal of Urology 156: 2007-2011.
77. Nanda K, Bastian L, Hasselbad V, Simel D. 1999. Hormone replacement therapy and the risk of colorectal cancer: a meta-analysis. Obstetrics and Gynecology 93: 880-888.
78. Neuenschwander B, Branson M, Spiegelhalter D. 2009. A note on the power prior. Statistics in Medicine 28: 3562-3566.
79. Nixon R, Duffy S. 2002. Cross-issue synthesis: potential application to breast cancer, tamoxifen and genetic susceptibility. Journal of Cancer Epidemiology and Prevention 7: 205-212.
80. Norris S, Moher D, Reeves B, Shea B, Loke Y, Garner S, Anderson L, Tugwell P, Wells G. 2013. Issues relating to selective reporting when including non-randomized studies in systematic reviews on the effects of healthcare interventions. Research Synthesis Methods 4: 36-47.
81. O'Rourke K, Walsh C. 2010. Impact of stroke units on mortality: a Bayesian analysis. European Journal of Neurology 17: 247-251.
82. O'Rourke K, Walsh C, Hutchinson M. 2007. Outcome of beta-interferon treatment in relapsing-remitting multiple sclerosis: a Bayesian analysis. Journal of Neurology 254: 1547-1554.
83. O'Rourke K, Walsh C, Kelly P. 2009. Safety and efficacy of IV-TPA for ischaemic stroke in clinical practice - a Bayesian analysis. Cerebrovascular Diseases 28: 572-581.
84. Orr R. 1999. The impact of prophylactic axillary node dissection on breast cancer survival - a Bayesian metaanalysis. Annals of Surgical Oncology 6(1): 109-116.
85. Committee on quality improvement. 2000. Clinical practice guideline: early detection of developmental dysplasia of the hip. Pediatrics 105.
86. Peters J, Rushton L, Sutton A, Jones D, Abrams K, Mugglestone M. 2005. Bayesian methods for the cross-design synthesis of epidemiological and toxicological evidence. Journal of the Royal Statistical Society, Series C 54: 159-172.
87. Peterson E, Shaw L, Califf R. 1997. Risk stratification after myocardial infarction. Annals of Internal Medicine 126: 561-582.
88. Piepho H, Williams E, Madden L. 2012. The use of two-way linear mixed models in multitreatment meta-analysis. Biometrics 68: 1269-1277.
89. Plummer M. 2003. JAGS: a program for analysis of Bayesian graphical models using gibbs sampling JAGS: Just Another Gibbs Sampler. Proceedings of DSC.
90. Prentice R, Langer R, Stefanick M, Howard B, Pettinger M, Anderson G, Barad D, Curb J, Kotchen J, Kuller L, Limacher M, Wactawski-Wende J. 2006. Combined analysis of Women's Health Initiative observational and clinical trial data on postmenopausal hormone treatment and cardiovascular disease. American Journal of Epidemiology 163: 589-99.
91. Presanis AM, Ohlssen D, Spiegelhalter D, De Angelis D. 2013. Conflict diagnostic in directed acyclic graphs, with applications in Bayesian evidence synthesis. Statistical Science 28(3): 376-397.
92. Prevost T, Abrams K, Jones D. 2000. Hierarchical models in generalized synthesis of evidence: an example based on studies of breast cancer screening. Statistics in Medicine 19: 3359-3376.
93. Reeves B, Higgins J, Ramsay C, Shea B, Tugwell P, Wells G. 2013. An introduction to methodological issues when including non-randomised studies in systematic reviews on the effects of interventions. Research Synthesis Methods 4: 1-11.
94. Riley RD, Lambert PC, Staessen JA, Wang J, Gueyffier F, Thijs L, Boutitie F. 2008. Meta-analysis of continuous outcomes combining individual patient data and aggregate data. Statistics in Medicine 27: 1870-1893.
95. Rothstein HR, Sutton AJ, Borenstein M. 2005. Publication bias in meta-analysis: prevention, assessment and adjustment. Wiley, Chichester.
96. Salanti G, Higgins JP, Ades A, Ioannidis JP. 2008. Evaluation of networks of randomized trials. Statistical Methods in Medical Research 17: 279-301.
97. Sampath S, Moran JL, Graham PL, Rockliff S, Bersten AD, Abrams KR. 2007. The efficacy of loop diuretics in acute renal failure: assessment using Bayesian evidence synthesis techniques. Critical Care Medicine 35: 2516-2524.
98. Schmitz S, Adams R, Walsh C. 2013. Incorporating data from various trial designs into a mixed treatment comparison model. Statistics in Medicine 32: 2935-2949.
99. Schünemann H, Tugwell P, Reeves B, Akl E, Santesso N, Spencer F, Shea B, Wells G, Helfand M. 2013. Nonrandomized studies as a source of complementary, sequential or replacement evidence for randomized controlled trials in systematic reviews on the effects of interventions. Research Synthesis Methods 4: 49-62.
100. Segura J, Preminger G, Assimos D, Dretler S, Kahn R, Lingeman J, Macaluso J, McCullough D. 1994. Nephrolithiasis clinical guidelines panel summary report on the management of staghorn calculi. The Journal of Urology 151: 1648-1651.
101. Segura J, Preminger G, Assimos D, Dretler S, Kahn R, Lingeman J, Macaluso J. 1997. Uteral stones clinical guidelines panel summary report on the management of ureteral calculi. Journal of Urology 158: 1915-1921.
102. Senn S, Gavini F, Magrez D, Scheen A. 2011. Issues in performing a network meta-analysis. Statistical Methods in Medical Research 22(2): 169-189.
103. Shachter RD, Eddy DM, Hasselblad V. 1990. An influence diagram approach to medical technology assessment. In Influence Diagrams, Belief Nets, and Decision Analysis, Oliver RM, Smith JQ (eds.). Wiley, Chichester; 321-350.
104. Shaw L, Peterson E, Kesler K, Hasselbad V, Califf R. 1996. A metaanalysis of predischarge risk stratification after acute myocardial infarction with stress electrocardiographic, myocardial perfusion, and ventricular function imaging. American Journal of Cardiology 78: 1327-1337.
105. Shekelle P, Adams A, Chassin M, Hurwitz E, Brook R. 1992. Spinal manipulation for low-back pain. Ann 117(7): 590-8.
106. Smith JA, Labasky RF, Cockett ATK, Fracchia JA, Montie JE, Rowland RG. 1999. Bladder cancer clinical guidelines panel summary report on the management of nonmuscle invasive bladder cancer (stages Ta, T1 and TIS). The Journal of Urology 162: 1697-1701.
107. Soares MO, Dumville JC, Ades AE, Welton NJ. 2014. Treatment comparisons for decision making: facing the problems of sparse and few data. Journal of the Royal Statistical Society: Series A (Statistics in Society) 177: 259-279.
108. Spiegelhalter D, Best N. 2003. Bayesian approaches to multiple sources of evidence and uncertainty in complex cost-effectiveness modelling. Statistics in Medicine 22: 3687-3709.
109. Spiegelhalter D, Abrams KR, Myles JP. 2004. Bayesian Approaches to Clinical Trials and Health-Care Evaluation. John Wiley & Sons, Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England
110. Sutton AJ, Abrams KR. 2001. Bayesian methods in meta-analysis and evidence synthesis. Statistical Methods in Medical Research 10: 277-303.
111. Sutton A, Higgins J. 2008. Recent developments in meta-analysis. Statistics in Medicine 27: 625-50.
112. Sutton A, Song F, Gilbody S, Abrams K. 2000. Modelling publication bias in meta-analysis: a review. Statistical Methods in Medical Research 9: 421-445.
113. Sutton A, Kendrick D, Coupland C. 2008. Meta-analysis of individual- and aggregate-level data. Statistics in Medicine 27: 651-669.
114. Thorlund K, Thabane L, Mills EJ. 2013. Modelling heterogeneity variance in multiple treatment comparison metaanalysis. Are informative priors the better solution? BMC Medical Research Methodology 13: 2.
115. Turner R, Spiegelhalter D, Smith G, Thompson S. 2009. Bias modelling in evidence synthesis. Journal of the Royal Statistical Society: Series A (Statistics in Society) 172: 21-47.
116. Turner R, Lloyd J, Anumba D, Smith G, Spiegelhalter D, Squires H, Stevens J, Sweeting M, Urbaniak S, Webster R, Thompson S. 2012. Routine antenatal anti-dD prophylaxis in women who are RhD negative: meta-analyses adjusted for differences in study design and quality. PLoS ONE 7 2: e30711.
117. Valentine J, Thompson S. 2013. Issues relating to confounding and meta-analysis when including non-randomized studies in systematic reviews on the effects of interventions. Research Synthesis Methods 4: 26-35.
118. Verde PE. 2010. Meta-analysis of diagnostic test data: a bivariate Bayesian modeling approach." Statistics in Medicine 29: 3088-3102.
119. Verde PE. 2013. Bamdit: Bayesian meta-analysis of diagnostic test data, r package version 1.1.
120. Wells GA, Shea B, Higgins J, Sterne J, Tugwell P, Reeves BC. 2013. Checklists of methodological issues for review authors to consider when including non-randomized studies in systematic reviews. Research Synthesis Methods 4: 63-77.
121. Welton NJ, Ades AE, Carlin JB, Altman DG, Sterne JAC. 2009. Models for potentially biased evidence in metaanalysis using empirically based priors. Journal of the Royal Statistical Society: Series A (Statistics in Society) 172: 119-136.
122. White IR, Barrett JK, Jackson D, Higgins JPT. 2012. Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression. Research Synthesis Methods 3: 111-125.
123. Wolpert RL, Mengersen KL. 2004. Adjusted likelihoods for synthesizing empirical evidence from studies that differ in quality and design: effects of environmental tobacco smoke. Statistical Science 19: 450-471.
124. Zimmerman M, Chelminski I, Posternak M. 2004. Exclusion criteria used in antidepressant efficacy trials: consistency across studies and representativeness of samples included. The Journal of Nervous and Mental Disease 192: 87-94.