Assessing the performance of methodological search filters to improve the efficiency of evidence information retrieval: Five literature reviews and a qualitative study

Health Technology Assessment

Volumn 21, Issue 69, 2017, Pages 1-147

  Lefebvre, Carol ^a,b   Glanville, Julie ^c   Beale, Sophie ^c   Boachie, Charles ^d   Duffy, Steven ^c   Fraser, Cynthia ^d   Harbour, Jenny ^e   McCool, Rachael ^c   Smith, Lynne ^e  

^a UK COCHRANE CENTRE   (United Kingdom)

^b Lefebvre Associates Ltd   (United Kingdom)

^c UNIVERSITY OF YORK   (United Kingdom)

^d UNIVERSITY OF ABERDEEN   (United Kingdom)

^e HEALTHCARE IMPROVEMENT SCOTLAND   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

DATA BASE; DECISION MAKING; DIAGNOSTIC ACCURACY; DIAGNOSTIC TEST; EVIDENCE BASED PRACTICE; INFORMATION PROCESSING; INTERVIEW; MEASUREMENT PRECISION; METHODOLOGY; QUALITATIVE RESEARCH; QUESTIONNAIRE; RANDOMIZED CONTROLLED TRIAL (TOPIC); RESEARCH; REVIEW; SENSITIVITY ANALYSIS; STANDARD; SYSTEMATIC REVIEW (TOPIC); VALIDATION PROCESS; BIBLIOGRAPHIC DATABASE; BIOMEDICAL TECHNOLOGY ASSESSMENT; HUMAN; INFORMATION RETRIEVAL; LITERATURE; PROCEDURES; SEARCH ENGINE;

DATABASES, BIBLIOGRAPHIC; HUMANS; INFORMATION STORAGE AND RETRIEVAL; QUALITATIVE RESEARCH; REVIEW LITERATURE AS TOPIC; SEARCH ENGINE; SURVEYS AND QUESTIONNAIRES; TECHNOLOGY ASSESSMENT, BIOMEDICAL;

EID: 85037622888     PISSN: 13665278     EISSN: 20464924     Source Type: Journal    
DOI: 10.3310/hta21690     Document Type: Review

References (142)

1. Beynon R, Leeflang MM, McDonald S, Eisinga A, Mitchell RL, Whiting P, Glanville JM. Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE. Cochrane Database Syst Rev 2013;9:MR000022. https://doi.org/10.1002/14651858.MR000022.pub3
2. Whiting P, Westwood M, Beynon R, Burke M, Sterne JA, Glanville J. Inclusion of methodological filters in searches for diagnostic test accuracy studies misses relevant studies. J Clin Epidemiol 2011;64:602-7. https://doi.org/10.1016/j.jclinepi.2010.07.006
3. Bak G, Mierzwinski-Urban M, Fitzsimmons H, Morrison A, Maden-Jenkins M. A pragmatic critical appraisal instrument for search filters: introducing the CADTH CAI. Health Info Libr J 2009;26:211-19. https://doi.org/10.1111/j.1471-1842.2008.00830.x
4. Glanville J, Bayliss S, Booth A, Dundar Y, Fernandes H, Fleeman ND, et al. So many filters, so little time: the development of a search filter appraisal checklist. J Med Libr Assoc 2008;96:356-61. https://doi.org/10.3163/1536-5050.96.4.011
5. Jenkins M, Johnson F. Awareness, use and opinions of methodological search filters used for the retrieval of evidence-based medical literature-a questionnaire survey. Health Info Libr J 2004;21:33-43. https://doi.org/10.1111/j.1471-1842.2004.00480.x
6. Glanville J, Lefebvre C, Wright K. ISSG Search Filter Resource. York: The InterTASC Information Specialists’ Sub-Group; 2008 [updated 2017]. URL: https://sites.google.com/a/york.ac.uk/ issg-search-filters-resource/home (accessed 22 August 2017).
7. McKinlay RJ, Wilczynski NL, Haynes RB, Hedges team. Optimal search strategies for detecting cost and economic studies in EMBASE. BMC Health Serv Res 2006;6:67. https://doi.org/10.1186/ 1472-6963-6-67
8. Wilczynski NL, Haynes RB, Lavis JN, Ramkissoonsingh R, Arnold-Oatley AE, HSR Hedges team. Optimal search strategies for detecting health services research studies in MEDLINE. CMAJ 2004;171:1179-85. https://doi.org/10.1503/cmaj.1040512
9. Astin MP, Brazzelli MG, Fraser CM, Counsell CE, Needham G, Grimshaw JM. Developing a sensitive search strategy in MEDLINE to retrieve studies on assessment of the diagnostic performance of imaging techniques. Radiology 2008;247:365-73. https://doi.org/10.1148/ radiol.2472070101
10. Bachmann LM, Coray R, Estermann P, Ter Riet G. Identifying diagnostic studies in MEDLINE: reducing the number needed to read. J Am Med Inform Assoc 2002;9:653-8. https://doi.org/ 10.1197/jamia.M1124
11. Bachmann LM, Estermann P, Kronenberg C, ter Riet G. Identifying diagnostic accuracy studies in EMBASE. J Med Libr Assoc 2003;91:341-6.
12. Berg A, Fleischer S, Behrens J. Development of two search strategies for literature in MEDLINE-PubMed: nursing diagnoses in the context of evidence-based nursing. Int J Nurs Terminol Classif 2005;16:26-32. https://doi.org/10.1111/j.1744-618X.2005.00006.x
13. Haynes RB, McKibbon KA, Wilczynski NL, Walter SD, Werre SR. Optimal search strategies for retrieving scientifically strong studies of treatment from MEDLINE: analytical survey. BMJ 2004;328:1040. https://doi.org/10.1136/bmj.38068.557998.EE
14. Vincent S, Greenley S, Beaven O. Clinical evidence diagnosis: developing a sensitive search strategy to retrieve diagnostic studies on deep vein thrombosis: a pragmatic approach. Health Info Libr J 2003;20:150-9. https://doi.org/10.1046/j.1365-2532.2003.00427.x
15. Wilczynski NL, Haynes RB, Hedges team. EMBASE search strategies for identifying methodologically sound diagnostic studies for use by clinicians and researchers. BMC Med Res Methodol 2005;3:7. https://doi.org/10.1186/1741-7015-3-7
16. Eady AM, Wilczynski NL, Haynes RB. PsycINFO search strategies identified methodologically sound therapy studies and review articles for use by clinicians and researchers. J Clin Epidemiol 2008;61:34-40. https://doi.org/10.1016/j.jclinepi.2006.09.016
17. Montori VM, Wilczynski NL, Morgan D, Haynes RB, Hedges team. Optimal search strategies for retrieving systematic reviews from MEDLINE: analytical survey. BMJ 2005;330:68. https://doi.org/ 10.1136/bmj.38336.804167.47
18. Shojania KG, Bero LA. Taking advantage of the explosion of systematic reviews: an efficient MEDLINE search strategy. Eff Clin Pract 2001;4:157-62.
19. White VJ, Glanville JM, Klefebvre C, Sheldon TA. A statistical approach to designing search filters to find systematic reviews: objectivity enhances accuracy. J Info Sci 2001;27:357-70. https://doi.org/ 10.1177/016555150102700601
20. Wilczynski NL, Haynes RB, Hedges team. EMBASE search strategies achieved high sensitivity and specificity for retrieving methodologically sound systematic reviews. J Clin Epidemiol 2007;60:29-33. https://doi.org/10.1016/j.jclinepi.2006.04.001
21. Wong SS, Wilczynski NL, Haynes RB. Optimal CINAHL search strategies for identifying therapy studies and review articles. J Nurs Scholarsh 2006;38:194-9. https://doi.org/10.1111/j.1547-5069. 2006.00100.x
22. Glanville JM, Lefebvre C, Miles JN, Camosso-Stefinovic J. How to identify randomized controlled trials in MEDLINE: ten years on. J Med Libr Assoc 2006;94:130-6.
23. Haynes RB, McKibbon KA, Wilczynski NL, Walter SD, Werre SR, Hedges team. Optimal search strategies for retrieving scientifically strong studies of treatment from MEDLINE: analytical survey. BMJ 2005;330:21. https://doi.org/10.1136/bmj.38446.498542.8F
24. Lefebvre C, Eisinga A, McDonald S, Paul N. Enhancing access to reports of randomized trials published world-wide-the contribution of EMBASE records to the Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library. Emerg Themes Epidemiol 2008;5:13. https://doi.org/10.1186/1742-7622-5-13
25. Manríquez JJ. A highly sensitive search strategy for clinical trials in Literatura Latino Americana e do Caribe em Ciências da Saúde (LILACS) was developed. J Clin Epidemiol 2008;61:407-11. https://doi.org/10.1016/j.jclinepi.2007.06.009
26. Robinson KA, Dickersin K. Development of a highly sensitive search strategy for the retrieval of reports of controlled trials using PubMed. Int J Epidemiol 2002;31:150-3. https://doi.org/10.1093/ ije/31.1.150
27. Taljaard M, McGowan J, Grimshaw JM, Brehaut JC, McRae A, Eccles MP, Donner A. Electronic search strategies to identify reports of cluster randomized trials in MEDLINE: low precision will improve with adherence to reporting standards. BMC Med Res Methodol 2010;10:15. https://doi.org/10.1186/ 1471-2288-10-15
28. Wong SS, Wilczynski NL, Haynes RB. Developing optimal search strategies for detecting clinically sound treatment studies in EMBASE. J Med Libr Assoc 2006;94:41-7.
29. Zhang L, Ajiferuke I, Sampson M. Optimizing search strategies to identify randomized controlled trials in MEDLINE. BMC Med Res Methodol 2006;6:23. https://doi.org/10.1186/1471-2288-6-23
30. Abhijnhan A, Surcheva Z, Wright J, Adams CE. Searching a biomedical bibliographic database from Bulgaria: the ABS database. Health Info Libr J 2007;24:200-3. https://doi.org/10.1111/ j.1471-1842.2007.00723.x
31. Almerie MQ, Matar HE, Jones V, Kumar A, Wright J, Wlostowska E, Adams CE. Searching the Polish Medical Bibliography (Polska Bibliografia Lekarska) for trials. Health Info Libr J 2007;24:283-6. https://doi.org/10.1111/j.1471-1842.2007.00716.x
32. Chow TK, To E, Goodchild CS, McNeil JJ. A simple, fast, easy method to identify the evidence base in pain-relief research: validation of a computer search strategy used alone to identify quality randomized controlled trials. Anesth Analg 2004;98:1557-65. https://doi.org/10.1213/ 01.ANE.0000114071.78448.2D
33. Corrao S, Colomba D, Arnone S, Argano C, Di Chiara T, Scaglione R, Licata G. Improving efficacy of PubMed clinical queries for retrieving scientifically strong studies on treatment. J Am Med Inform Assoc 2006;13:485-7. https://doi.org/10.1197/jamia.M2084
34. Day D, Furlan A, Irvin E, Bombardier C. Simplified search strategies were effective in identifying clinical trials of pharmaceuticals and physical modalities. J Clin Epidemiol 2005;58:874-81. https://doi.org/10.1016/j.jclinepi.2005.02.005
35. de Freitas AE, Herbert RD, Latimer J, Ferreira PH. Searching the LILACS database for Portugueseand Spanish-language randomized trials in physiotherapy was difficult. J Clin Epidemiol 2005;58:233-7. https://doi.org/10.1016/j.jclinepi.2004.06.014
36. Devillé WL, Buntinx F, Bouter LM, Montori VM, de Vet HCW, van der Windt D, Bezemer PD. Conducting systematic reviews of diagnostic studies: didactic guidelines. BMC Med Res Methodol 2002;2:9. https://doi.org/10.1186/1471-2288-2-9
37. Eisinga A, Siegfried N, Clarke M. The sensitivity and precision of search terms in Phases I, II and III of the Cochrane Highly Sensitive Search Strategy for identifying reports of randomized trials in MEDLINE in a specific area of health care-HIV/AIDS prevention and treatment interventions. Health Info Libr J 2007;24:103-9. https://doi.org/10.1111/j.1471-1842.2007.00698.x
38. Kele I, Bereczki D, Furtado V, Wright J, Adams CE. Searching a biomedical bibliographic database from Hungary-the ‘Magyar Orvosi Bibliografia’. Health Info Libr J 2005;22:286-95. https://doi.org/10.1111/j.1471-1842.2005.00577.x
39. Kumar A, Wright J, Adams CE. Searching a biomedical bibliographic database from the Ukraine: the Panteleimon database. Health Info Libr J 2005;22:223-7. https://doi.org/10.1111/j.1471-1842. 2005.00578.x
40. McDonald S. Improving access to the international coverage of reports of controlled trials in electronic databases: a search of the Australasian Medical Index. Health Info Libr J 2002;19:14-20. https://doi.org/10.1046/j.0265-6647.2001.00359.x
41. Royle P, Waugh N. Literature searching for clinical and cost-effectiveness studies used in health technology assessment reports carried out for the National Institute for Clinical Excellence appraisal system. Health Technology Assessment 2003;7(34). https://doi.org/10.3310/hta7340
42. Royle P, Waugh N. A simplified search strategy for identifying randomised controlled trials for systematic reviews of health care interventions: a comparison with more exhaustive strategies. BMC Med Res Methodol 2005;5:23. https://doi.org/10.1186/1471-2288-5-23
43. Royle P, Waugh N. Making literature searches easier: a rapid and sensitive search filter for retrieving randomized controlled trials from PubMed. Diabet Med 2007;24:308-11. https://doi.org/10.1111/j.1464-5491.2007.02046.x
44. Sassi F, Archard L, McDaid D. Searching literature databases for health care economic evaluations: how systematic can we afford to be? Med Care 2002;40:387-94. https://doi.org/10.1097/ 00005650-200205000-00004
45. Wilczynski NL, Haynes RB. Consistency and accuracy of indexing systematic review articles and meta-analyses in Medline. Health Info Libr J 2009;26:203-10. https://doi.org/10.1111/ j.1471-1842.2008.00823.x
46. Harbour J, Fraser C, Lefebvre C, Glanville J, Beale S, Boachie C, et al. Reporting methodological search filter performance comparisons: a literature review. Health Info Libr J 2014;31:176-94. https://doi.org/10.1111/hir.12070
47. Glanville J, Fleetwood K, Yellowlees A, Kaunelis D, Mensinkai S. Development and Testing of Search Filters to Identify Economic Evaluations in MEDLINE and EMBASE. Ottawa, ON: Canadian Agency for Drugs and Technologies in Health (CADTH); 2009.
48. Leeflang MM, Scholten RJ, Rutjes AW, Reitsma JB, Bossuyt PM. Use of methodological search filters to identify diagnostic accuracy studies can lead to the omission of relevant studies. J Clin Epidemiol 2006;59:234-40. https://doi.org/10.1016/j.jclinepi.2005.07.014
49. Ritchie G, Glanville J, Lefebvre C. Do published search filters to identify diagnostic test accuracy studies perform adequately? Health Info Libr J 2007;24:188-92. https://doi.org/10.1111/ j.1471-1842.2007.00735.x
50. Deurenberg R, Vlayen J, Guillo S, Oliver TK, Fervers B, Burgers J. Standardization of search methods for guideline development: an international survey of evidence-based guideline development groups. Health Info Libr J 2008;25:23-30. https://doi.org/10.1111/j.1471-1842. 2007.00732.x
51. Jenkins M. Evaluation of methodological search filters-a review. Health Info Libr J 2004;21:148-63. https://doi.org/10.1111/j.1471-1842.2004.00511.x
52. Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, et al. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Med Res Methodol 2006;6:33. https://doi.org/10.1186/1471-2288-6-33
53. Boluyt N, Tkosvold L, Lefebvre C, Klassen TP, Offringa M. The usefulness of systematic review search strategies in finding child health systematic reviews in MEDLINE. Arch Pediatr Adolesc Med 2008;162:111-16. https://doi.org/10.1001/archpediatrics.2007.40
54. Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches. Int J Technol Assess Health Care 2003;19:591-603. https://doi.org/ 10.1017/S0266462303000552
55. Bardia A, Wahner-Roedler DL, Erwin PL, Sood A. Search strategies for retrieving complementary and alternative medicine clinical trials in oncology. Integr Cancer Ther 2006;5:202-5. https://doi.org/10.1177/1534735406292146
56. Boynton J, Glanville J, McDaid D, Lefebvre C. Identifying systematic reviews in MEDLINE: developing an objective approach to search strategy design. J Info Sci 1998;24:137-57. https://doi.org/10.1177/016555159802400301
57. Devillé WL, Bezemer PD, Bouter LM. Publications on diagnostic test evaluation in family medicine journals: an optimal search strategy. J Clin Epidemiol 2000;53:65-9. https://doi.org/10.1016/ S0895-4356(99)00144-4
58. Doust JA, Pietrzak E, Sanders S, Glasziou PP. Identifying studies for systematic reviews of diagnostic tests was difficult due to the poor sensitivity and precision of methodologic filters and the lack of information in the abstract. J Clin Epidemiol 2005;58:444-9. https://doi.org/10.1016/ j.jclinepi.2004.09.011
59. Glanville J, Kaunelis D, Mensinkai S. How well do search filters perform in identifying economic evaluations in MEDLINE and EMBASE. Int J Technol Assess Health Care 2009;25:522-9. https://doi.org/10.1017/S0266462309990523
60. Kastner M, Wilczynski NL, McKibbon AK, Garg AX, Haynes RB. Diagnostic test systematic reviews: bibliographic search filters (‘Clinical Queries’) for diagnostic accuracy studies perform well. J Clin Epidemiol 2009;62:974-81. https://doi.org/10.1016/j.jclinepi.2008.11.006
61. McKibbon KA, Wilczynski NL, Haynes RB, Hedges team. Retrieving randomized controlled trials from MEDLINE: a comparison of 38 published search filters. Health Info Libr J 2009;26:187-202. https://doi.org/10.1111/j.1471-1842.2008.00827.x
62. Royle P, Waugh N. A simplified search strategy for identifying randomised controlled trials for systematic reviews of health care interventions: a comparison with more exhaustive strategies. BMC Med Res Methodol 2005;5:2-3. https://doi.org/10.1186/1471-2288-5-2
63. Wong SS, Wilczynski NL, Haynes RB. Comparison of top-performing search strategies for detecting clinically sound treatment studies and systematic reviews in MEDLINE and EMBASE. J Med Libr Assoc 2006;94:451-5.
64. Haynes RB, Wilczynski N, McKibbon KA, Walker CJ, Sinclar JC. Developing optimal search strategies for detecting clinically sound studies in MEDLINE. J Am Med Inform Assoc 1994;1:447-58.
65. Castro AA, Clark OA, Atallah AN. Optimal search strategy for clinical trials in the Latin American and Caribbean Health Science Literature database (LILACS database): update. Sao Paulo Med J 1999;117:138e9. http://dx.doi.org/10.1590/S1516-31801997000300004
66. Bradley SM. Examination of the clinical queries and systematic review ‘hedges’ in EMBASE and MEDLINE. J Can Health Libr Assoc 2010;31:27-37. https://doi.org/10.5596/c10-022
67. Leeflang MM, Deeks JJ, Gatsonis C, Bossuyt PM, Cochrane Diagnostic Test Accuracy Working Group. Systematic reviews of diagnostic test accuracy. Ann Intern Med 2008;149:889-97. https://doi.org/10.7326/0003-4819-149-12-200812160-00008
68. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. BMJ 2003;326:41-4. https://doi.org/10.1136/bmj.326.7379.41
69. Centre for Reviews and Dissemination. Systematic Reviews: CRD’s Guidance for Undertaking Reviews in Health Care. York: University of York; 2009.
70. Hui SL, Zhou XH. Evaluation of diagnostic tests without gold standards. Stat Methods Med Res 1998;7:354-70. https://doi.org/10.1177/096228029800700404
71. Reitsma JB, Rutjes AW, Khan KS, Coomarasamy A, Bossuyt PM. A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard. J Clin Epidemiol 2009;62:797-806. https://doi.org/10.1016/j.jclinepi.2009.02.005
72. Rutjes AW, Reitsma JB, Coomarasamy A, Khan KS, Bossuyt PM. Evaluation of diagnostic tests when there is no gold standard. A review of methods. Health Technol Assess 2007;11(50). https://doi.org/10.3310/hta11500
73. Bossuyt P, Leeflang M. Chapter 6: developing criteria for including studies. In Cochrane Handbook of Systematic Reviews of Diagnostic Test Accuracy Version 0.4. The Cochrane Collaboration; 2008.
74. Deeks JJ. Systematic reviews in health care: systematic reviews of evaluations of diagnostic and screening tests. BMJ 2001;323:157-62. https://doi.org/10.1136/bmj.323.7305.157
75. Food and Drug Administration. Guidance for Industry and Staff: Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests. US Department of Health and Human Services; 2007. URL: www.fda.gov/default.htm (accessed 1 June 2011).
76. Medical Services Advisory Committee. Guidelines for the Assessment of Diagnostic Technologies. Canberra, ACT: Australian Government Department of Health and Ageing; 2005.
77. Lijmer JG, Mol BW, Heisterkamp S, Bonsel GJ, Prins MH, van der Meulen JH, Bossuyt PM. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA 1999;282:1061-6. https://doi.org/10.1001/jama.282.11.1061
78. Westwood ME, Whiting PF, Kleijnen J. How does study quality affect the results of a diagnostic meta-analysis? BMC Med Res Methodol 2005;5:20. https://doi.org/10.1186/1471-2288-5-20
79. Whiting P, Rutjes AW, Reitsma JB, Glas AS, Bossuyt PM, Kleijnen J. Sources of variation and bias in studies of diagnostic accuracy: a systematic review. Ann Intern Med 2004;140:189-202. https://doi.org/10.7326/0003-4819-140-3-200402030-00010
80. Cook C, Cleland J, Huijbregts P. Creation and critique of studies of diagnostic accuracy: use of the STARD and QUADAS methodological quality assessment tools. J Man Manip Ther 2007;15:93-102. https://doi.org/10.1179/106698107790819945
81. Bachmann LM, Puhan MA, ter Riet G, Bossuyt PM. Sample sizes of studies on diagnostic accuracy: literature survey. BMJ 2006;332:1127-9. https://doi.org/10.1136/bmj.38793.637789.2F
82. Bochmann F, Johnson Z, Azuara-Blanco A. Sample size in studies on diagnostic accuracy in ophthalmology: a literature survey. Br J Ophthalmol 2007;91:898-900. https://doi.org/10.1136/ bjo.2006.113290
83. Flahault A, Cadilhac M, Thomas G. Sample size calculation should be performed for design accuracy in diagnostic test studies. J Clin Epidemiol 2005;58:859-62. https://doi.org/10.1016/ j.jclinepi.2004.12.009
84. Whiting P, Rutjes AW, Dinnes J, Reitsma J, Bossuyt PM, Kleijnen J. Development and validation of methods for assessing the quality of diagnostic accuracy studies. Health Technol Assess 2004;8(25). https://doi.org/10.3310/hta8250
85. Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 2003;3:25. https://doi.org/10.1186/1471-2288-3-25
86. Belgian Health Care Knowledge Centre. HTA: Molecular Diagnostics in Belgium. Report no. 20A. Brussels: Federaal Kenniscentrum voor de Gezondheidszorg; 2005. URL: www.kce.fgov.be/ (accessed 1 June 2011).
87. Whiting PF, Sterne JA, Westwood ME, Bachmann LM, Harbord R, Egger M, Deeks JJ. Graphical presentation of diagnostic information. BMC Med Res Methodol 2008;8:20. https://doi.org/ 10.1186/1471-2288-8-20
88. Jaeschke R, Guyatt GH, Sackett DL. Users’ guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group. JAMA 1994;271:703-7. https://doi.org/ 10.1001/jama.1994.03510330081039
89. Deeks J, Altman DG, Bradburn MJ. Chapter 10: Statistical Methods for Examining Heterogeneity and Combining Results from Several Studies in Meta-analysis. In Egger M, Davey Smith G, Altman DG, editors. Systematic Reviews in Health Care: Meta-analysis in Context. London: BMJ Publishing Group; 2001. pp. 285-312. https://doi.org/10.1002/9780470693926.ch15
90. Deeks J, Bossuyt P, Gatsonis C, Macaskill P, Harbord R, Takwoingi Y. Analysing and Presenting Results. In Cochrane Handbook of Systematic Reviews of Diagnostic Test Accuracy Version 0.9.0. The Cochrane Collaboration; 2010.
91. Bossuyt PM. The quality of reporting in diagnostic test research: getting better, still not optimal. Clin Chem 2004;50:465-6. https://doi.org/10.1373/clinchem.2003.029736
92. Coppus SF, van der Veen F, Bossuyt PM, Mol BW. Quality of reporting of test accuracy studies in reproductive medicine: impact of the Standards for Reporting of Diagnostic Accuracy (STARD) initiative. Fertil Steril 2006;86:1321-9. https://doi.org/10.1016/j.fertnstert.2006.03.050
93. Harper R, Reeves B. Reporting of precision of estimates for diagnostic accuracy: a review. BMJ 1999;318:1322-3. https://doi.org/10.1136/bmj.318.7194.1322
94. Rama KR, Poovali S, Apsingi S. Quality of reporting of orthopaedic diagnostic accuracy studies is suboptimal. Clin Orthop Relat Res 2006;447:237-46. https://doi.org/10.1097/01.blo.0000205906. 44103.a3
95. Shunmugam M, Azuara-Blanco A. The quality of reporting of diagnostic accuracy studies in glaucoma using the Heidelberg retina tomograph. Invest Ophthalmol Vis Sci 2006;47:2317-23. https://doi.org/10.1167/iovs.05-1250
96. Siddiqui MA, Azuara-Blanco A, Burr J. The quality of reporting of diagnostic accuracy studies published in ophthalmic journals. Br J Ophthalmol 2005;89:261-5. https://doi.org/10.1136/ bjo.2004.051862
97. Smidt N, Rutjes AW, van der Windt DA, Ostelo RW, Bossuyt PM, Reitsma JB. Reproducibility of the STARD checklist: an instrument to assess the quality of reporting of diagnostic accuracy studies. BMC Med Res Method 2006;6:12. https://doi.org/10.1186/1471-2288-6-12
98. Wilczynski NL. Quality of reporting of diagnostic accuracy studies: no change since STARD statement publication-before-and-after study. Radiology 2008;248:817-23. https://doi.org/ 10.1148/radiol.2483072067
99. Smidt N, Rutjes AW, van der Windt DA, Ostelo RW, Bossuyt PM, Reitsma JB, et al. The quality of diagnostic accuracy studies since the STARD statement: has it improved? Neurology 2006;67:792-7. https://doi.org/10.1212/01.wnl.0000238386.41398.30
100. Agency for Healthcare Research and Quality. A Comprehensive Overview of the Methods and Reporting of Meta-analyses of Test Accuracy. Rockville, MD: Agency for Healthcare Research and Quality; 2011.
101. Honest H, Khan KS. Reporting of measures of accuracy in systematic reviews of diagnostic literature. BMC Health Serv Res 2002;2:4. https://doi.org/10.1186/1472-6963-2-4
102. Mallett S, Deeks JJ, Halligan S, Hopewell S, Cornelius V, Altman DG. Systematic reviews of diagnostic tests in cancer: review of methods and reporting. BMJ 2006;333:413. https://doi.org/ 10.1136/bmj.38895.467130.55
103. Belgian Health Care Knowledge C. Search for Evidence and Critical Appraisal: Health Technology Assessment [Process Notes D200/710.273/40]. Brussels: Belgian Health Care Knowledge Centre; 2007.
104. National Institute for Health and Care Excellence. Interim Methods Statement: Centre for Health Technology Evaluation, Diagnostics Assessment Programme. London: NICE; 2010. URL: www.nice.org.uk/ (accessed 1 June 2011).
105. Morris RK, Selman TJ, Zamora J, Khan KS. Methodological quality of test accuracy studies included in systematic reviews in obstetrics and gynaecology: sources of bias. BMC Womens Health 2011;11:7. https://doi.org/10.1186/1472-6874-11-7
106. Whiting P, Rutjes AW, Dinnes J, Reitsma JB, Bossuyt PM, Kleijnen J. A systematic review finds that diagnostic reviews fail to incorporate quality despite available tools. J Clin Epidemiol 2005;58:1-12. https://doi.org/10.1016/j.jclinepi.2004.04.008
107. Leeflang MM, Bossuyt PM, Irwig L. Diagnostic test accuracy may vary with prevalence: implications for evidence-based diagnosis. J Clin Epidemiol 2009;62:5-12. https://doi.org/10.1016/ j.jclinepi.2008.04.007
108. Rutjes AW, Reitsma JB, Di Nisio M, Smidt N, van Rijn JC, Bossuyt PM. Evidence of bias and variation in diagnostic accuracy studies. CMAJ 2006;174:469-76. https://doi.org/10.1503/ cmaj.050090
109. Dinnes J, Deeks J, Kirby J, Roderick P. A methodological review of how heterogeneity has been examined in systematic reviews of diagnostic test accuracy. Health Technol Assess 2005;9(12). https://doi.org/10.3310/hta9120
110. Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 2005;58:982-90. https://doi.org/10.1016/j.jclinepi.2005.02.022
111. Rutter CM, Gatsonis CA. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med 2001;20:2865-84. https://doi.org/10.1002/sim.942
112. Willis BH, Quigley M. Uptake of newer methodological developments and the deployment of meta-analysis in diagnostic test research: a systematic review. BMC Med Res Methodol 2011;11:27. https://doi.org/10.1186/1471-2288-11-27
113. Davis J, Goadrich M. The Relationship between Precision-Recall and ROC Curves. Proceedings of the 23rd Annual Conference on Machine Learning, Pittsburgh, PA, 2006. New York, NY: ACM; 2006. pp. 233-40. https://doi.org/10.1145/1143844.1143874
114. Higgins J, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011. URL: www.handbook.cochrane.org (accessed 25 August 2017).
115. Jha S, Ho A, Bhargavan M, Owen JB, Sunshine JH. Imaging evaluation for suspected pulmonary embolism: what do emergency physicians and radiologists say? AJR Am J Roentgenol 2010;194:W38-48. https://doi.org/10.2214/AJR.09.2694
116. McGinnis PQ, Hack LM, Nixon-Cave K, Michlovitz SL. Factors that influence the clinical decision making of physical therapists in choosing a balance assessment approach. Phys Ther 2009;89:233-47. https://doi.org/10.2522/ptj.20080131
117. Perneger TV, Martin DP, Bovier PA. Physicians’ attitudes toward health care rationing. Med Decis Making 2002;22:65-70. https://doi.org/10.1177/0272989X0202200106
118. Sox CM, Koepsell TD, Doctor JN, Christakis DA. Pediatricians’ clinical decision making: results of 2 randomized controlled trials of test performance characteristics. Arch Pediatr Adolesc Med 2006;160:487-92. https://doi.org/10.1001/archpedi.160.5.487
119. Stein PD, Sostman HD, Dalen JE, Bailey DL, Bajc M, Goldhaber SZ, et al. Controversies in diagnosis of pulmonary embolism. Clin Appl Thromb Hemost 2011;17:140-9. https://doi.org/10. 1177/1076029610389027
120. Wackerbarth SB, Tarasenko YN, Curtis LA, Joyce JM, Haist SA. Using decision tree models to depict primary care physicians CRC screening decision heuristics. J Gen Intern Med 2007;22:1467-9. https://doi.org/10.1007/s11606-007-0338-6
121. Zettler M, Mollon B, da Silva V, Howe B, Speechley M, Vinden C. Family physicians’ choices of and opinions on colorectal cancer screening modalities. Can Fam Physician 2010;56:e338-44.
122. UK National Screening Committee. Criteria for Appraising the Viability, Effectiveness and Appropriateness of a Screening Programme. 2011. URL: www.screening.nhs.uk/criteria (accessed 1 June 2011).
123. US Preventive Services Task Force. Procedure Manual. Publication No. 08-05118-EF. US Preventive Services Task Force; 2008. URL: www.uspreventiveservicestaskforce.org/uspstf08/methods/ procmanual.htm (accessed 1 June 2011).
124. Australian Population Health Development Principal Committee Screening Subcommittee. Population Based Screening Framework. Canberra, ACT, Australian Population Health Development Principal Committee Screening Subcommittee; 2008. URL: www.health.gov.au (accessed June 2011).
125. World Health Organization. Screening for Various Cancers. Geneva: WHO; 2011. URL: www.who.int/cancer/detection/variouscancer/en/ (accessed 1 June 2011).
126. Agoritsas T, Courvoisier DS, Combescure C, Deom M, Perneger TV. Does prevalence matter to physicians in estimating post-test probability of disease? A randomized trial. J Gen Intern Med 2011;26:373-8. https://doi.org/10.1007/s11606-010-1540-5
127. Bramwell R, West H, Salmon P. Health professionals’ and service users’ interpretation of screening test results: experimental study. BMJ 2006;333:284. https://doi.org/10.1136/bmj.38884.663102.AE
128. Cahan A, Gilon D, Manor O, Paltiel O. Probabilistic reasoning and clinical decision-making: do doctors overestimate diagnostic probabilities? Q J Med 2003;96:763-9. https://doi.org/10.1093/ qjmed/hcg122
129. Heller RF, Sandars JE, Patterson L, McElduff P. GPs’ and physicians’ interpretation of risks, benefits and diagnostic test results. Fam Pract 2004;21:155-9. https://doi.org/10.1093/fampra/cmh209
130. Sox CM, Doctor JN, Koepsell TD, Christakis DA. The influence of types of decision support on physicians’ decision making. Arch Dis Child 2009;94:185-90. https://doi.org/10.1136/adc. 2008.141903
131. Steurer J, Fischer JE, Bachmann LM, Koller M, ter Riet G. Communicating accuracy of tests to general practitioners: a controlled study. BMJ 2002;324:824-6. https://doi.org/10.1136/ bmj.324.7341.824
132. Beale S, Duffy S, Glanville J, Lefebvre C, Wright D, McCool R, et al. Choosing and using methodological search filters: searchers’ views. Health Info Libr J 2014;31:133-47. https://doi.org/ 10.1111/hir.12062
133. Lefebvre C, Manheimer E, Glanville J. Chapter 6: Searching for Studies. In Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011. URL: http://training.cochrane.org/handbook (accessed 22 September 2017).
134. EBLIP Editorial Team. Evidence Based Library and Information Practice. Edmonton, AB: University of Alberta; 2014. URL: http://ejournals.library.ualberta.ca/index.php/EBLIP/index (accessed 1 September 2014).
135. Hausner E, Waffenschmidt S, Kaiser T, Simon M. Routine development of objectively derived search strategies. Syst Rev 2012;1:19. https://doi.org/10.1186/2046-4053-1-19
136. Noyes P, Popay J, Pearson A, Hannes K, Booth A. Chapter 20: Qualitative Research and Cochrane Reviews. In Higgins J, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011. URL: http://training.cochrane.org/handbook (accessed 22 September 2017).
137. Sampson M, Tetzlaff J, Urquhart C. Precision of healthcare systematic review searches in a cross-sectional sample. Res Synth Methods 2011;2:119-25. https://doi.org/10.1002/jrsm.42
138. Hopewell S, Clark M, Lefebvre C, Scherer R. Handsearching still a valuable element of the systematic review. Evid Based Dent 2008;9:85. https://doi.org/10.1038/sj.ebd.6400602
139. Hopewell S, Clarke M, Lefebvre C, Scherer R. Handsearching versus electronic searching to identify reports of randomized trials. Cochrane Database Syst Rev 2007;2:MR000001. https://doi.org/10.1002/14651858.MR000001.pub2
140. McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 guideline statement. J Clin Epidemiol 2016;75:40-6. https://doi.org/10.1016/j.jclinepi.2016.01.021
141. McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS-Peer Review of Electronic Search Strategies: 2015 Guideline Explanation and Elaboration (PRESS E&E). Ottawa, ON: Canadian Agency for Drugs and Technologies in Health (CADTH); 2016. URL: www.cadth.ca/sites/default/files/pdf/CP0015_PRESS_Update_Report_2016.pdf (accessed 1 January 2017).
142. Whiting P, Westwood M, Bojke L, Palmer S, Richardson G, Cooper J, et al. Clinical effectiveness and cost-effectiveness of tests for the diagnosis and investigation of urinary tract infection in children: a systematic review and economic model. Health Technol Assess 2006;10(36). https://doi.org/10.3310/hta10360