What comparative effectiveness research is needed? A framework for using guidelines and systematic reviews to identify evidence gaps and research priorities

Annals of Internal Medicine

Volumn 156, Issue 5, 2012, Pages 367-377

  Li, Tianjing ^a   Vedula, S Swaroop ^a   Scherer, Roberta ^a   Dickersin, Kay ^a  

^a JOHNS HOPKINS BLOOMBERG SCHOOL OF PUBLIC HEALTH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPARATIVE EFFECTIVENESS; CONCEPTUAL FRAMEWORK; HUMAN; MEDICAL RESEARCH; OPEN ANGLE GLAUCOMA; PRACTICE GUIDELINE; PRIORITY JOURNAL; QUESTIONNAIRE; REVIEW; SYSTEMATIC REVIEW;

EID: 84863273835     PISSN: 00034819     EISSN: 15393704     Source Type: Journal    
DOI: 10.7326/0003-4819-156-5-201203060-00009     Document Type: Review

References (101)

1. American Recovery and Reinvestment Act of 2009, Pub. L. No. 111-5, 123 Stat. 226 (2009).
2. Institute of Medicine. Knowing What Works in Health Care: A Roadmap for the Nation. Washington, DC: National Academies Pr; 2008.
3. Patient Protection and Affordable Care Act of 2010. Pub. L. No. 111-148, 24 Stat. 119 (2010).
4. Patient-Centered Outcomes Research Institute. PCORI Funding Announcement: Pilot Projects Grants. Washington, DC: Patient-Centered Outcomes Research Institute; 2011. Accessed at www.pcori.org/funding-opportunities/past-funding-pportunities/pcori-funding-announcement-pilot-projects-grants/on11January2012.
5. Friedman DS, Wolfs RC, O'Colmain BJ, Klein BE, Taylor HR, West S, et al; Eye Diseases Prevalence Research Group. Prevalence of open-angle glaucoma among adults in the United States. Arch Ophthalmol. 2004;122:532-8. [PMID: 15078671]
6. Quigley HA, Broman AT. The number of people with glaucoma worldwide in 2010 and 2020. Br J Ophthalmol. 2006;90:262-7. [PMID: 16488940]
7. Guyatt G, Rennie D. User's Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice. Chicago: American Medical Assoc; 2002.
8. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't [Editorial]. BMJ. 1996;312: 71-2. [PMID: 8555924]
9. Li T, Ervin AM, Scherer R, Jampel H, Dickersin K. Setting priorities for comparative effectiveness research: a case study using primary open-angle glaucoma. Ophthalmology. 2010;117:1937-45. [PMID: 20800896]
10. American Academy of Ophthalmology. Primary Open-Angle Glaucoma Preferred Practice Patterns. San Francisco: American Acad Ophthalmology; 2010. Accessed at http://one.aao.org/CE/PracticeGuidelines/PPP.aspxon11January2012.
11. Haynes RB, Sackett DL, Guyatt GH, Tugwell P. Clinical Epidemiology: How to Do Clinical Practice Research. 3rd ed. Philadelphia: Lippincott Williams & Wilkins; 2005.
12. Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.0.2 [updated March 2011]. The Cochrane Collaboration; 2011. Accessed at www.cochrane-handbook.orgon 11August2011.
13. Montori VM, Wilczynski NL, Morgan D, Haynes RB; Hedges Team. Optimal search strategies for retrieving systematic reviews from Medline: analyt ical survey. BMJ. 2005;330:68. [PMID: 15619601]
14. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10. [PMID: 17302989]
15. Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62:1013-20. [PMID: 19230606]
16. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health-care interventions: explanation and elaboration. Ann Intern Med. 2009;151:W65-94. [PMID: 19622512]
17. Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151:264-9, W64. [PMID: 19622511]
18. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al; GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924-6. [PMID: 18436948]
19. Cheng JW, Xi GL, Wei RL, Cai JP, Li Y. Efficacy and tolerability of latanoprost compared to dorzolamide combined with timolol in the treatment of patients with elevated intraocular pressure: a meta-analysis of randomized, controlled trials. J Ocul Pharmacol Ther. 2009;25:55-64. [PMID: 19232016]
20. Glenny AM, Altman DG, Song F, Sakarovitch C, Deeks JJ, D'Amico R, et al; International Stroke Trial Collaborative Group. Indirect comparisons of competing interventions. Health Technol Assess. 2005;9:1-134, iii-iv. [PMID: 16014203]
21. Abudu M, Kadir J, Abudukadir M, Chen XY, Junire, Arkin A. Non-penetrating trabecular surgery versus trabeculectomy for open aangle glaucoma: a systematic review. Chinese J Evid-Bas Med. 2008;8:1094-9.
22. Mei Li Ke Mu AS, Juerti HD. Dorzolamide/timolol combination versus latanoprost in the treatment of open-angle glaucoma: a systematic review. Chinese J Evid-Bas Med. 2009;9:350-4.
23. Aptel F, Cucherat M, Denis P. Efficacy and tolerability of prostaglandin analogs: a meta-analysis of randomized controlled clinical trials. J Glaucoma. 2008;17:667-73. [PMID: 19092464]
24. Burr J, Azuara-Blanco A, Avenell A. Medical versus surgical interventions for open angle glaucoma. Cochrane Database Syst Rev. 2005:CD004399. [PMID: 15846712]
25. Cheng JW, Ma XY, Wei RL. Efficacy of non-penetrating trabecular surgery for open angle glaucoma: a meta-analysis. Chin Med J (Engl). 2004;117:1006-10. [PMID: 15265373]
26. Cheng JW, Wei RL. Meta-analysis of 13 randomized controlled trials comparing bimatoprost with latanoprost in patients with elevated intraocular pressure. Clin Ther. 2008;30:622-32. [PMID: 18498911]
27. Cheng JW, Li Y, Wei RL. Systematic review of intraocular pressure-lowering effects of adjunctive medications added to latanoprost. Ophthalmic Res. 2009; 42:99-105. [PMID: 19546601]
28. Cox JA, Mollan SP, Bankart J, Robinson R. Efficacy of antiglaucoma fixed combination therapy versus unfixed components in reducing intraocular pressure: a systematic review. Br J Ophthalmol. 2008;92:729-34. [PMID: 18460539]
29. Denis P, Lafuma A, Khoshnood B, Mimaud V, Berdeaux G. A meta-analysis of topical prostaglandin analogues intra-ocular pressure lowering in glaucoma therapy. Curr Med Res Opin. 2007;23:601-8. [PMID: 17355741]
30. Einarson TR, Kulin NA, Tingey D, Iskedjian M. Meta-analysis of the effect of latanoprost and brimonidine on intraocular pressure in the treatment of glaucoma. Clin Ther. 2000;22:1502-15. [PMID: 11192141]
31. Feyi-Waboso A, Ejere HO. Needling for encapsulated trabeculectomy filtering blebs. Cochrane Database Syst Rev. 2004:CD003658. [PMID: 15106211]
32. Friedman DS, Jampel HD, Lubomski LH, Kempen JH, Quigley H, Con-gdon N, et al. Surgical strategies for coexisting glaucoma and cataract: an evidence-based update. Ophthalmology. 2002;109:1902-13. [PMID: 12359612]
33. Hylton C, Congdon N, Friedman D, Kempen J, Quigley H, Bass E, et al. Cataract after glaucoma filtration surgery. Am J Ophthalmol. 2003;135:231-2. [PMID: 12566033]
34. Jampel HD, Lubomski LH, Friedman DS. Treatment of Coexisting Cataract and Glaucoma. Evidence Report/Technology Assessment Number 38. AHRQ Publication No. 03-E041. Rockville, MD: Agency for Healthcare Research and Quality; 2003.
35. Jampel HD, Friedman DS, Lubomski LH, Kempen JH, Quigley H, Con-gdon N, et al. Effect of technique on intraocular pressure after combined cataract and glaucoma surgery: an evidence-based review. Ophthalmology. 2002;109: 2215-24; quiz 2225, 2231. [PMID: 12466161]
36. Fung AT, Reid SE, Jones MP, Healey PR, McCluskey PJ, Craig JC. Meta-analysis of randomised controlled trials comparing latanoprost with bri-monidine in the treatment of open-angle glaucoma, ocular hypertension or normal-tension glaucoma. Br J Ophthalmol. 2007;91:62-8. [PMID: 16956912]
37. Gray TA, Orton LC, Henson D, Harper R, Waterman H. Interventions for improving adherence to ocular hypotensive therapy. Cochrane Database Syst Rev. 2009:CD006132. [PMID: 19370627]
38. Hodge WG, Lachaine J, Steffensen I, Murray C, Barnes D, Foerster V, et al. The efficacy and harm of prostaglandin analogues for IOP reduction in glaucoma patients compared to dorzolamide and brimonidine: a systematic review. Br J Ophthalmol. 2008;92:7-12. [PMID: 18156371]
39. Holmstrom S, Buchholz P, Walt J, Wickstrøm J, Aagren M. Analytic review of bimatoprost, latanoprost and travoprost in primary open angle glaucoma. Curr Med Res Opin. 2005;21:1875-83. [PMID: 16307709]
40. Hondur A, Onol M, Hasanreisoglu B. Nonpenetrating glaucoma surgery: meta-analysis of recent results. J Glaucoma. 2008;17:139-46. [PMID: 18344761]
41. Hong CH, Arosemena A, Zurakowski D, Ayyala RS. Glaucoma drainage devices: a systematic literature review and current controversies. Surv Ophthal-mol. 2005;50:48-60. [PMID: 15621077]
42. Honrubia F, Garćia-Sánchez J, Polo V, de la Casa JM, Soto J. Conjunctival hyperaemia with the use of latanoprost versus other prostaglandin analogues in patients with ocular hypertension or glaucoma: a meta-analysis of randomised clinical trials. Br J Ophthalmol. 2009;93:316-21. [PMID: 19019922]
43. Katz LJ, Simmons ST, Craven ER. Efficacy and safety of brimonidine and dorzolamide for intraocular pressure lowering in glaucoma and ocular hypertension. Curr Med Res Opin. 2007;23:2971-83. [PMID: 17949534]
44. Li N, Chen XM, Zhou Y, Wei ML, Yao X. Travoprost compared with other prostaglandin analogues or timolol in patients with open-angle glaucoma or ocular hypertension: meta-analysis of randomized controlled trials. Clin Experiment Ophthalmol. 2006;34:755-64. [PMID: 17073898]
45. Loon SC, Liew G, Fung A, Reid SE, Craig JC. Meta-analysis of randomized controlled trials comparing timolol with brimonidine in the treatment of glaucoma. Clin Experiment Ophthalmol. 2008;36:281-9. [PMID: 18412600]
46. Maier PC, Funk J, Schwarzer G, Antes G, Falck-Ytter YT. Treatment of ocular hypertension and open angle glaucoma: meta-analysis of randomised controlled trials. BMJ. 2005;331:134. [PMID: 15994659]
47. Minckler DS, Vedula SS, Li TJ, Mathew MC, Ayyala RS, Francis BA. Aqueous shunts for glaucoma. Cochrane Database Syst Rev. 2006:CD004918. [PMID: 16625616]
48. Netland PA; Ophthalmic Technology Assessment Committee Glaucoma Panel, American Academy of Ophthalmology. Nonpenetrating glaucoma surgery. Ophthalmology. 2001;108:416-21. [PMID: 11158823]
49. Pastor SA, Singh K, Lee DA, Juzych MS, Lin SC, Netland PA, et al. Cyclophotocoagulation: a report by the American Academy of Ophthalmology. Ophthalmology. 2001;108:2130-8. [PMID: 11713091]
50. Rolim de Moura C, Paranhos A Jr, Wormald R. Laser trabeculoplasty for open angle glaucoma. Cochrane Database Syst Rev. 2007:CD003919. [PMID: 17943806]
51. Simmons ST, Dirks MS, Noecker RJ. Bimatoprost versus latanoprost in lowering intraocular pressure in glaucoma and ocular hypertension: results from parallel-group comparison trials. Adv Ther. 2004;21:247-62. [PMID: 15605619]
52. Stewart WC, Kolker AE, Sharpe ED, Day DG, Konstas AG, Hollo G, et al. Long-term progression at individual mean intraocular pressure levels in primary open-angle and exfoliative glaucoma. Eur J Ophthalmol. 2008;18:765-70. [PMID: 18850556]
53. Sun W, Cai JP, Li Y, Cheng JW. Meta-analysis of nonpenetrating trabecular surgery combined with implant in treatment of open angle glaucoma. Int J Oph-thalmol. 2008;8:749-52.
54. van der Valk R, Webers CA, Schouten JS, Zeegers MP, Hendrikse F, Prins MH. Intraocular pressure-lowering effects of all commonly used glaucoma drugs: a meta-analysis of randomized clinical trials. Ophthalmology. 2005;112:1177-85. [PMID: 15921747]
55. Vass C, Hirn C, Sycha T, Findl O, Bauer P, Schmetterer L. Medical interventions for primary open angle glaucoma and ocular hypertension. Co-chrane Database Syst Rev. 2007:CD003167. [PMID: 17943780]
56. Vinuesa-Silva JM, Vinuesa-Silva I, Pinazo-Durán MD, Soto-Alvarez J, Delgado-Ortega L, ďiaz-Cerezo S. Development of conjunctival hyperemia with the use of a fixed combination of latanoprost/timolol: systematic review and meta-analysis of clinical trials. Arch Soc Esp Oftalmol. 2009;84:199-207. [PMID: 19384760]
57. Webers CA, van der Valk R, Schouten JS, Zeegers MP, Prins MH, Hen-drikse F. Intraocular pressure-lowering effect of adding dorzolamide or latano-prost to timolol: a meta-analysis of randomized clinical trials. Ophthalmology. 2007;114:40-6. [PMID: 17070584]
58. Webers CA, Beckers HJ, Nuijts RM, Schouten JS. Pharmacological management of primary open-angle glaucoma: second-line options and beyond. Drugs Aging. 2008;25:729-59. [PMID: 18729546]
59. Wilkins M, Indar A, Wormald R. Intra-operative mitomycin C for glaucoma surgery. Cochrane Database Syst Rev. 2005:CD002897. [PMID: 16235305]
60. Wormald R, Wilkins MR, Bunce C. Post-operative 5-fluorouracil for glaucoma surgery. Cochrane Database Syst Rev. 2001:CD001132. [PMID: 11686977]
61. Zhang WY, Po AL, Dua HS, Azuara-Blanco A. Meta-analysis of randomised controlled trials comparing latanoprost with timolol in the treatment of patients with open angle glaucoma or ocular hypertension. Br J Ophthalmol. 2001;85: 983-90. [PMID: 11466259]
62. Whitlock EP, Lopez SA, Chang S, Helfand M, Eder M, Floyd N. AHRQ series paper 3: identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health-care program. J Clin Epidemiol. 2010;63:491-501. [PMID: 19540721]
63. Chalkidou K, Whicher D, Kary W, Tunis S. Comparative effectiveness research priorities: identifying critical gaps in evidence for clinical and health policy decision making. Int J Technol Assess Health Care. 2009;25:241-8. [PMID: 19619341]
64. Iglehart JK. Prioritizing comparative-effectiveness research-IOM recommendations. N Engl J Med. 2009;361:325-8. [PMID: 19567828]
65. O'Leary TJ, Slutsky JR, Bernard MA. Comparative effectiveness research priorities at federal agencies: the view from the Department of Veterans Affairs, National Institute on Aging, and Agency for Healthcare Research and Quality. J Am Geriatr Soc. 2010;58:1187-92. [PMID: 20936736]
66. Smith N, Mitton C, Peacock S, Cornelissen E, MacLeod S. Identifying research priorities for health care priority setting: a collaborative effort between managers and researchers. BMC Health Serv Res. 2009;9:165. [PMID: 19754969]
67. Maciosek MV, Coffield AB, McGinnis JM, Harris JR, Caldwell MB, Teutsch SM, et al. Methods for priority setting among clinical preventive services. Am J Prev Med. 2001;21:10-9. [PMID: 11418252]
68. Douw K, Vondeling H, Oortwijn W. Priority setting for horizon scanning of new health technologies in Denmark: views of health care stakeholders and health economists. Health Policy. 2006;76:334-45. [PMID: 16081185]
69. Noorani HZ, Husereau DR, Boudreau R, Skidmore B. Priority setting for health technology assessments: a systematic review of current practical approaches. Int J Technol Assess Health Care. 2007;23:310-5. [PMID: 17579932]
70. Husereau D, Boucher M, Noorani H. Priority setting for health technology assessment at CADTH. Int J Technol Assess Health Care. 2010;26:341-7. [PMID: 20584365]
71. Johnston SC, Hauser SL. IOM task impossible: set the agenda for neurological comparative effectiveness research [Editorial]. Ann Neurol. 2009;66:A4-6. [PMID: 19938063]
72. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4:e78. [PMID: 17388659]
73. Edwards P, Clarke M, DiGuiseppi C, Pratap S, Roberts I, Wentz R. Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records. Stat Med. 2002;21:1635-40. [PMID: 12111924]
74. Jones AP, Remmington T, Williamson PR, Ashby D, Smyth RL. High prevalence but low impact of data extraction and reporting errors were found in Cochrane systematic reviews. J Clin Epidemiol. 2005;58:741-2. [PMID: 15939227]
75. Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP. Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol. 2006;59:697-703. [PMID: 16765272]
76. GØtzsche PC, Hróbjartsson A, Maric K, Tendal B. Data extraction errors in meta-analyses that use standardized mean differences. JAMA. 2007;298:430-7. [PMID: 17652297]
77. Hrobjartsson A, Gøtzsche PC. Unsubstantiated claims of large effects of placebo on pain: serious errors in meta-analysis of placebo analgesia mechanism studies. J Clin Epidemiol. 2006;59:336-8. [PMID: 16549252]
78. Ford AC, Guyatt GH, Talley NJ, Moayyedi P. Errors in the conduct of systematic reviews of pharmacological interventions for irritable bowel syndrome. Am J Gastroenterol. 2010;105:280-8. [PMID: 19920807]
79. Bhandari M, Morrow F, Kulkarni AV, Tornetta P. Meta-analyses in orthopaedic surgery. A systematic review of their methodologies. J Bone Joint Surg Am. 2001;83-A:15-24. [PMID: 11205853]
80. Delaney A, Bagshaw SM, Ferland A, Manns B, Laupland KB, Doig CJ. A systematic evaluation of the quality of meta-analyses in the critical care literature. Crit Care. 2005;9:R575-82. [PMID: 16277721]
81. Glenny AM, Esposito M, Coulthard P, Worthington HV. The assessment of systematic reviews in dentistry. Eur J Oral Sci. 2003;111:85-92. [PMID: 12648258]
82. Lundh A, Knijnenburg SL, Jørgensen AW, van Dalen EC, Kremer LC. Quality of systematic reviews in pediatric oncology-a systematic review. Cancer Treat Rev. 2009;35:645-52. [PMID: 19836897]
83. Shea B, Moher D, Graham I, Pham B, Tugwell P. A comparison of the quality of Cochrane reviews and systematic reviews published in paper-based journals. Eval Health Prof. 2002;25:116-29. [PMID: 11868441]
84. Institute of Medicine. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: National Academies Pr; 2011.
85. Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One. 2008;3:e3081. [PMID: 18769481]
86. Hopewell S, Clarke M, Lefebvre C, Scherer R. Handsearching versus electronic searching to identify reports of randomized trials. Cochrane Database Syst Rev. 2007:MR000001. [PMID: 17443625]
87. Hopewell S, Clarke M, Stewart L, Tierney J. Time to publication for results of clinical trials. Cochrane Database Syst Rev. 2007:MR000011. [PMID: 17443632]
88. Hopewell S, McDonald S, Clarke M, Egger M. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev. 2007:MR000010. [PMID: 17443631]
89. Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev. 2009:MR000006. [PMID: 19160345]
90. Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ. 2003;326: 1167-70. [PMID: 12775614]
91. Moher D, Pham B, Lawson ML, Klassen TP. The inclusion of reports of randomised trials published in languages other than English in systematic reviews. Health Technol Assess. 2003;7:1-90. [PMID: 14670218]
92. Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007:MR000005. [PMID: 17443628]
93. Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009; 361:1963-71. [PMID: 19907043]
94. Goodman S, Dickersin K. Metabias: a challenge for comparative effectiveness research [Editorial]. Ann Intern Med. 2011;155:61-2.
95. Li T, Puhan MA, Vedula SS, Singh S, Dickersin K; Ad Hoc Network Meta-analysis Methods Meeting Working Group. Network meta-analysis-highly attractive but more methodological research is needed. BMC Med. 2011; 9:79. [PMID: 21707969]
96. Straus S, Moher D. Registering systematic reviews. CMAJ. 2010;182:13-4. [PMID: 19620270]
97. Li T. Register systematic reviews [Letter]. CMAJ. 2010;182:805. [PMID: 20479061]
98. Song F, Loke YK, Walsh T, Glenny AM, Eastwood AJ, Altman DG. Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. BMJ. 2009;338: b1147. [PMID: 19346285].
99. Lumley T. Network meta-analysis for indirect treatment comparisons. Stat Med. 2002;21:2313-24. [PMID: 12210616].
100. Raj D. A note on the variance of the ratio estimate. J Am Stat Assoc. 1964;59:895-8.
101. Weinreb RN, Kaufman PL. The glaucoma research community and FDA look to the future: a report from the NEI/FDA CDER Glaucoma Clinical Trial Design and Endpoints Symposium. Invest Ophthalmol Vis Sci. 2009;50:1497-505. [PMID: 19321793]