Finding evidence for comparing medical interventions: AHRQ and the Effective Health Care Program

Journal of Clinical Epidemiology

Volumn 64, Issue 11, 2011, Pages 1168-1177

  Relevo, Rose ^a   Balshem, Howard ^a  

^a OREGON HEALTH AND SCIENCE UNIVERSITY   (United States)

Author keywords

Avoiding bias; Bibliographic databases; Comparative effectiveness reviews; Controlled vocabulary; Expert searching; Gray literature

Indexed keywords

CINAHL; CLINICAL EFFECTIVENESS; CLINICAL PROTOCOL; CLINICAL TRIAL (TOPIC); COMPARATIVE EFFECTIVENESS; COMPARATIVE STUDY; CONSULTATION; DATA BASE; EMBASE; EVIDENCE BASED PRACTICE; HEALTH CARE; HEALTH PROGRAM; INTERPERSONAL COMMUNICATION; MEDICAL INFORMATION; MEDICAL LITERATURE; MEDLINE; METHODOLOGY; OBSERVATIONAL STUDY; PRIORITY JOURNAL; PSYCINFO; REGISTER; REVIEW;

COMPARATIVE EFFECTIVENESS RESEARCH; DATA COLLECTION; DATABASES, BIBLIOGRAPHIC; EVIDENCE-BASED MEDICINE; HUMANS; OUTCOME ASSESSMENT (HEALTH CARE); PUBLICATION BIAS; RESEARCH DESIGN; REVIEW LITERATURE AS TOPIC; UNITED STATES; UNITED STATES AGENCY FOR HEALTHCARE RESEARCH AND QUALITY;

EID: 80053368677     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2010.11.022     Document Type: Review

References (97)

1. E.P. Whitlock, J.S. Lin, R. Chou, P. Shekelle, and K.A. Robinson Using existing systematic reviews in complex systematic reviews Ann Intern Med 148 2008 776 782 (Pubitemid 351717420)
2. E.P. Whitlock, S.A. Lopez, S. Chang, M. Helfand, M. Eder, and N. Floyd AHRQ series paper 3: identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health care program J Clin Epidemiol 63 2010 491 501
3. Medical Library Association Role of expert searching in health sciences libraries: policy statement by the Medical Library Association adopted September 2003 J Med Libr Assoc 93 2005 42 44
4. J. McGowan, and M. Sampson Systematic reviews need systematic searchers J Med Libr Assoc 93 2005 74 80 (Pubitemid 41711169)
5. S. Golder, Y. Loke, and H.M. McIntosh Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects J Clin Epidemiol 61 2008 440 448 (Pubitemid 351446183)
6. L.B. Mokkink, C.B. Terwee, P.W. Stratford, J. Alonso, D.L. Patrick, and I. Riphagen Evaluation of the methodological quality of systematic reviews of health status measurement instruments Qual Life Res 18 2009 313 333
7. V. Alberani, P. De Castro Pietrangeli, and A.M. Mazza The use of grey literature in health sciences: a preliminary survey Bull Med Libr Assoc 78 1990 358 363
8. J. Illig Archiving "event knowledge": bringing "dark data" to light J Med Libr Assoc 96 2008 189 191
9. Grey Literature Network Service, editor. New frontiers in grey literature. Fourth International conference on Grey Literature. Washington, DC: GL'99 proceedings; October 4-5, 1999.
10. V.S. Conn, J.C. Valentine, H.M. Cooper, and M.J. Rantz Grey literature in meta-analyses [Review] Nurs Res 52 2003 256 261
11. K. Dickersin, R. Scherer, and C. Lefebvre Identifying relevant studies for systematic reviews [Review] BMJ 309 1994 1286 1291 (Pubitemid 24340510)
12. L. McAuley, B. Pham, P. Tugwell, and D. Moher Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet 356 2000 1228 1231
13. S. Hopewell, S. McDonald, J. Clarke Mike, and M. Egger Grey literature in meta-analyses of randomized trials of health care interventions Cochrane Database Syst Rev [Serial on the Internet] 2 2007 Available at http://www.mrw.interscience.wiley.com/cochrane/clsysrev/articles/MR000010/frame. html Accessed April 28, 2011
14. L. Hartling, F.A. McAlister, B.H. Rowe, J. Ezekowitz, C. Friesen, and T.P. Klassen Challenges in systematic reviews of therapeutic devices and procedures Ann Intern Med 142 12 Pt 2 2005 1100 1111 (Pubitemid 40847713)
15. D. Helmer, I. Savoie, C. Green, and A. Kazanjian Evidence-based practice: extending the search to find material for the systematic review Bull Med Libr Assoc 89 2001 346 352
16. P.G. Shekelle, S.C. Morton, M.J. Suttorp, N. Buscemi, and C. Friesen Challenges in systematic reviews of complementary and alternative medicine topics Ann Intern Med 142 12 Pt 2 2005 1042 1047 (Pubitemid 40847706)
17. M. Egger, P. Juni, C. Bartlett, F. Holenstein, and J. Sterne How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study Health Technol Asses 7 1 2003 1 76
18. A.M. Cook, I.G. Finlay, A.G. Edwards, K. Hood, I.J. Higginson, and D.M. Goodwin Efficiency of searching the grey literature in palliative care J Pain Symptom Manage 22 2001 797 801 (Pubitemid 32776591)
19. D. Fergusson, A. Laupacis, L.R. Salmi, F.A. McAlister, and C. Huet What should be included in meta-analyses? An exploration of methodological issues using the ISPOT meta-analyses Int J Technol Assess Health Care 16 2000 1109 1119
20. M.L. van Driel, A. De Sutter, J. De Maeseneer, and T. Christiaens Searching for unpublished trials in Cochrane reviews may not be worth the effort J Clin Epidemiol 62 2009 838 844 e3
21. D.A. Bennett, and A. Jull FDA: untapped source of unpublished trials Lancet 361 2003 1402 1403 (Pubitemid 36515036)
22. C.H. MacLean, S.C. Morton, J.J. Ofman, E.A. Roth, and P.G. Shekelle How useful are unpublished data from the Food and Drug Administration in meta-analysis? J Clin Epidemiol 56 2003 44 51 (Pubitemid 36183314)
23. M. Man-Son-Hing, G. Wells, and A. Lau Quinine for nocturnal leg cramps: a meta-analysis including unpublished data J Gen Intern Med 13 1998 600 606 (Pubitemid 28424031)
24. E.H. Turner, A.M. Matthews, E. Linardatos, R.A. Tell, and R. Rosenthal Selective publication of antidepressant trials and its influence on apparent efficacy N Engl J Med 358 2008 252 260
25. Health Canada Drug Products Database Health Canada Available at http://webprod.hc-sc.gc.ca/dpd-bdpp/index-eng.jsp Accessed September 21, 2010
26. European Public Assessement Reports European Medicines Agency Available at http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/landing/epar- search.jsp&murl=menus/medicines/medicines.jsp&mid=WC0b01ac058001d124 Accessed September 21, 2010
27. I. Savoie, D. Helmer, C.J. Green, and A. Kazanjian Beyond MEDLINE: reducing bias through extended systematic review search [Journal Article] Int J Technol Assess Health Care 19 2003 168 178 (Pubitemid 36356337)
28. S. Mathieu, I. Boutron, D. Moher, D.G. Altman, and P. Ravaud Comparison of registered and published primary outcomes in randomized controlled trials JAMA 302 2009 977 984
29. U.S. National Institutes of Health. ClinicalTrials.gov. 2010; Available at http://clinicaltrials.gov. Accessed September 21, 2010.
30. Current Controlled Trials BioMed Central Available at http://www.controlled-trials.com/ Accessed September 21, 2010
31. Clinical Study Results. Available at http://www.clinicalstudyresults.org/ home/. Accessed September 21, 2010.
32. WHO International Clinical Trials Registry Platform World Health Organization Available at http://apps.who.int/trialsearch/ Accessed September 21, 2010
33. E. von Elm, M.C. Costanza, B. Walder, and M.R. Tramer More insight into the fate of biomedical meeting abstracts: a systematic review BMC Med Res Methodol 3 2003 12
34. M. Toma, F.A. McAlister, L. Bialy, D. Adams, B. Vandermeer, and P.W. Armstrong Transition from meeting abstract to full-length journal article for randomized controlled trials JAMA 295 2006 1281 1287
35. E.J. Matthews, A.G. Edwards, J. Barker, M. Bloor, J. Covey, and K. Hood Efficient literature searching in diffuse topics: lessons from a systematic review of research on communicating risk to patients in primary care Health Libr Rev 16 2 1999 112 120
36. A.J. Brettle, and A.F. Long Comparison of bibliographic databases for information on the rehabilitation of people with severe mental illness Bull Med Libr Assoc 89 2001 353 362
37. N. O'Leary, E. Tiernan, D. Walsh, N. Lucey, J. Kirkova, and M.P. Davis The pitfalls of a systematic MEDLINE review in palliative medicine: symptom assessment instruments Am J Hosp Palliat Care 24 2007 181 184
38. J.M. Glanville, C. Lefebvre, J.N. Miles, and J. Camosso-Stefinovic How to identify randomized controlled trials in MEDLINE: ten years on J Med Libr Assoc 94 2006 130 136
39. K.A. McKibbon, N.L. Wilczynski, R.B. Haynes, and T. Hedges Retrieving randomized controlled trials from MEDLINE: a comparison of 38 published search filters Health Info Libr J 26 2009 187 202
40. R.B. Haynes, N. Wilczynski, K.A. McKibbon, C.J. Walker, and J.C. Sinclair Developing optimal search strategies for detecting clinically sound studies in MEDLINE J Am Med Inform Assoc 1 1994 447 458
41. Cochrane Handbook for Systematic Reviews of Interventions: 6.4.11 search filters 2008: Version 5.0.1 Available at www.cochrane-handbook.org Updated September 2008. Accessed September 21, 2010
42. Scottish Intercollegiate Guidelines Network (SIGN) Search filters. Edingurgh2009 Available at http://www.sign.ac.uk/methodology/filters.html Updated August 3, 2009. Accessed September 21, 2010
43. InterTASC Information Specialists' Sub-Group Search Filter Resource 2009 Available at http://www.york.ac.uk/inst/crd/intertasc/diag.htm Updated July 2, 2009. Accessed September 21, 2010
44. G. Bak, M. Mierzwinski-Urban, H. Fitzsimmons, A. Morrison, and M. Maden-Jenkins A pragmatic critical appraisal instrument for search filters: introducing the CADTH CAI Health Info Libr J 26 2009 211 219
45. M. Sampson, J. McGowan, C. Lefebvre, D. Moher, and J.M. Grimshaw PRESS: peer review of electronic search strategies 2008 Canadian Agency for Drugs and Technologies in Health Ottowa, Canada
46. V.S. Conn, S.A. Isaramalai, S. Rath, P. Jantarakupt, R. Wadhawan, and Y. Dash Beyond MEDLINE for literature searches [Review] J Nurs Scholarsh 35 2 2003 177 182 (Pubitemid 38586662)
47. A. Morrison, K. Moulton, M. Clark, J. Polisena, M. Fiander, and M. Mierzwinski-Urban English-language restriction when conducting systematic review-based meta-analyses: systematic review of published studies [Serial on the Internet] 2009 Canadian Agency for Drugs and Technologies in Health Ottawa, Canada Available at http://www.cadth.ca/media/pdf/H0478-Language-Restriction- Systematic-Review-Pub-Studies-e.pdf Accessed April 28, 2011
48. D.A. Hernandez, M.M. El-Masri, and C.A. Hernandez Choosing and using citation and bibliographic database software (BDS) Diabetes Educ 34 2008 457 474
49. M. Gomis, C. Gall, and F.A. Brahmi Web-based citation management compared to end note: options for medical sciences Med Ref Serv Q 27 3 2008 260 271
50. White CM, Ip S, McPheeters M, Carey TS, Cour R, Lohr KN. Using existing systematic reviews to replace de novo processes in conducting Comparative Effectiveness Reviews. In: Agency for Healthcare Research and Quality. Methods Guide for Comparative Effectiveness Reviews [posted September 2009]. Rockville, MD. Available at http://effectivehealthcare.ahrq.gov/healthInfo.cfm?infotype= rr&ProcessID=60. Accessed September 21, 2010.
51. Y.K. Loke, D. Price, A. Herxheimer, and C.A.E.M. Group Systematic reviews of adverse effects: framework for a structured approach BMC Med Res Methodol 7 2007 32
52. R. Chou, and M. Helfand Challenges in systematic reviews that assess treatment harms [Review] Ann Intern Med 142 12 Pt 2 2005 1090 1099 (Pubitemid 40847712)
53. R. Chou, N. Aronson, D. Atkins, A.S. Ismaila, P. Santaguida, and D.H. Smith AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the Effective Health-Care Program J Clin Epidemiol 63 2010 502 512
54. S. Derry, Y. Kong Loke, and J.K. Aronson Incomplete evidence: the inadequacy of databases in tracing published adverse drug reactions in clinical trials BMC Med Res Methodol 1 2001 7
55. S. Golder, H.M. McIntosh, S. Duffy, and J. Glanville Developing efficient search strategies to identify reports of adverse effects in MEDLINE and EMBASE Health Info Libr J 23 2006 3 12
56. S. Wieland, and K. Dickersin Selective exposure reporting and Medline indexing limited the search sensitivity for observational studies of the adverse effects of oral contraceptives J Clin Epidemiol 58 2005 560 567 (Pubitemid 40692721)
57. Y.K. Loke, D. Price, and A. Herxheimer Appendix 6b. Including adverse effects J.P. Higgins, S. Green, Cochrane handbook for systematic reviews of interventions 425 [Updated May 2005] 2007 Cochrane Colloboration; Cochrane Adverse Effects Subgroup Chichester, UK 194 195
58. C. Fraser, A. Murray, and J. Burr Identifying observational studies of surgical interventions in MEDLINE and EMBASE BMC Med Res Methodol 6 2006 41
59. E. von Elm, D.G. Altman, M. Egger, S.J. Pocock, P.C. Gotzsche, and J.P. Vandenbroucke The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies Ann Intern Med 147 2007 573 577 (Pubitemid 351664504)
60. S. Golder, Y. Loke, and H.M. McIntosh Room for improvement? A survey of the methods used in systematic reviews of adverse effects BMC Med Res Methodol 6 2006 3
61. A.D. Furlan, E. Irvin, and C. Bombardier Limited search strategies were effective in finding relevant nonrandomized studies J Clin Epidemiol 59 2006 1303 1311 (Pubitemid 44708607)
62. Scottish Intercollegiate Guidelines Network Search filters Available at http://www.sign.ac.uk/methodology/filters.html Accessed September 21, 2010
63. J.P. Vandenbroucke, E. von Elm, D.G. Altman, P.C. Gotzsche, C.D. Mulrow, and S.J. Pocock Strengthening the reporting of observational studies in epidemiology (STROBE): explanation and elaboration [Web only] Ann Intern Med 147 2007 W163 W194 (Pubitemid 351664503)
64. M.H. Zheng, X. Zhang, Q. Ye, and Y.P. Chen Searching additional databases except PubMed are necessary for a systematic review Stroke 39 8 2008 e139 author reply e40
65. M.E. Suarez-Almazor, E. Belseck, J. Homik, M. Dorgan, and C. Ramos-Remus Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough Control Clin Trials 21 5 2000 476 487
66. A.P. Betran, L. Say, A.M. Gulmezoglu, T. Allen, and L. Hampson Effectiveness of different databases in identifying studies for systematic reviews: experience from the WHO systematic review of maternal morbidity and mortality BMC Med Res Methodol 5 1 2005 6
67. M. Sampson, N.J. Barrowman, D. Moher, T.P. Klassen, B. Pham, and R. Platt Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol 56 2003 943 955 (Pubitemid 37268081)
68. J.B. DeLuca, M.M. Mullins, C.M. Lyles, N. Crepaz, L. Kay, and S. Thadiparthi Developing a comprehensive search strategy for evidence based systematic reviews Evid Based Libr Inf Pract 3 1 2008 3 32
69. H. Kuper, A. Nicholson, and H. Hemingway Searching for observational studies: what does citation tracking add to PubMed? A case study in depression and coronary heart disease BMC Med Res Methodol 6 2006 4
70. Google Scholar. Available at http://scholar.google.com/. Accessed September 21, 2010.
71. Scopus Elsevier Available at http://www.scopus.com/home.url Accessed September 21, 2010
72. PubFocus. Available at http://pubfocus.com/. Accessed September 21, 2010.
73. PubMed PubReMiner Available at http://bioinfo.amc.uva.nl/human-genetics/ pubreminer/ Accessed September 21, 2010
74. M.E. Falagas, E.I. Pitsouni, G.A. Malietzis, and G. Pappas Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses FASEB J 22 2008 338 342 (Pubitemid 351225599)
75. L. Salisbury Web of Science and Scopus: a comparative review of content and searching capabilities The Charleston Advisor 11 1 2009 5 18
76. P. Jasco As we may search - comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases Curr Sci 89 2005 1537 1547
77. N. Bakkalbasi, K. Bauer, J. Glover, and L. Wang Three options for citation tracking: Google Scholar, Scopus and Web of Science Biomed Digit Libr 3 2006 7
78. A.R. Jadad, and H.J. McQuay Searching the literature. Be systematic in your searching [Comment] BMJ 307 1993 66
79. P.C. Gotzsche Reference bias in reports of drug trials Br Med J (Clin Res Ed) 295 1987 654 656 (Pubitemid 17136544)
80. Armour T, Dingwall O, Sampson M. Contribution of checking reference lists to systematic reviews. Poster presentation at: XIII Cochrane Colloquium. Melbourne: Australia; 2005.
81. A. Al Hajeri, J. Al Sayyad, and A. Eisinga Handsearching the EMHJ for reports of randomized controlled trials by U.K. Cochrane Centre (Bahrain) East Mediterr Health J 12 Suppl 2 2006 S253 S257 (Pubitemid 46039934)
82. A.R. Jadad, D. Moher, and T.P. Klassen Guides for reading and interpreting systematic reviews: II. How did the authors find the studies and assess their quality? Arch Pediatr Adolesc Med 152 1998 812 817 (Pubitemid 28360903)
83. S. Hopewell, M. Clarke, A. Lusher, C. Lefebvre, and M. Westby A comparison of handsearching versus MEDLINE searching to identify reports of randomized controlled trials Stat Med 21 2002 1625 1634 (Pubitemid 34746066)
84. A. Avenell, H.H. Handoll, and A.M. Grant Lessons for search strategies from a systematic review, in The Cochrane Library, of nutritional supplementation trials in patients after hip fracture Am J Clin Nutr 73 2001 505 510 (Pubitemid 32176499)
85. R. Armstrong, N. Jackson, J. Doyle, E. Waters, and F. Howes It's in your hands: the value of handsearching in conducting systematic reviews of public health interventions J Public Health (Oxf) 27 2005 388 391 (Pubitemid 41752474)
86. D.A. Zarin, N.C. Ide, T. Tse, W.R. Harlan, J.C. West, and D.A. Lindberg Issues in the registration of clinical trials JAMA 297 2007 2112 2120 (Pubitemid 46798169)
87. M.R. Tramer, D.J. Reynolds, R.A. Moore, and H.J. McQuay Impact of covert duplicate publication on meta-analysis: a case study BMJ 315 1997 635 640 (Pubitemid 27387173)
88. L. Reveiz, A.F. Cardona, E.G. Ospina, and S. de Agular An e-mail survey identified unpublished studies for systematic reviews J Clin Epidemiol 59 2006 755 758 (Pubitemid 43833496)
89. G.A. Kelley, K.S. Kelley, and Z.V. Tran Retrieval of missing data for meta-analysis: a practical example Int J Technol Assess Health Care 20 2004 296 299 (Pubitemid 39079565)
90. F. Peinemann, N. McGauran, S. Sauerland, and S. Lange Negative pressure wound therapy: potential publication bias caused by lack of access to unpublished study results data BMC Med Res Methodol 8 2008 4
91. D. Rennie Fair conduct and fair reporting of clinical trials JAMA 282 1999 1766 1768 (Pubitemid 29533609)
92. C.A. Gibson, B.W. Bailey, M.J. Carper, J.D. Lecheminant, E.P. Kirk, and G. Huang Author contacts for retrieval of data for a meta-analysis on exercise and diet restriction Int J Technol Assess Health Care 22 2006 267 270
93. A. Yoshii, D.A. Plaut, K.A. McGraw, M.J. Anderson, and K.E. Wellik Analysis of the reporting of search strategies in Cochrane systematic reviews J Med Libr Assoc 97 2009 21 29
94. M. Sampson, J. McGowan, J. Tetzlaff, E. Cogo, and D. Moher No consensus exists on search reporting methods for systematic reviews J Clin Epidemiol 61 2008 748 754
95. M. Egger, P. Juni, C. Bartlett CONSORT Group Value of flow diagrams in reports of randomized controlled trials JAMA 285 2001 1996 1999 (Pubitemid 32366953)
96. S. Hopewell, M. Clarke, D. Moher, E. Wager, P. Middleton, and D.G. Altman CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration PLoS Med 5 1 2008 e20
97. P. Spoor, M. Airey, C. Bennett, J. Greensill, and R. Williams Use of the capture-recapture technique to evaluate the completeness of systematic literature searches BMJ 313 1996 342 343 (Pubitemid 26334820)