Searching for evidence or approval? A commentary on database search in systematic reviews and alternative information retrieval methodologies

Research Synthesis Methods

Volumn 9, Issue 1, 2018, Pages 124-131

  Delaney, Aogán ^a   Tamás, Peter A ^b  

^a Wageningen University ^*  (Luxembourg)

^b WAGENINGEN UNIVERSITY   (Netherlands)

Author keywords

information science; qualitative research; realist reviews; systematic reviews meta analyses

Indexed keywords

BIBLIOGRAPHIC DATABASE; EVIDENCE BASED MEDICINE; HUMAN; INFORMATION RETRIEVAL; LITERATURE; PROCEDURES; QUALITATIVE RESEARCH; RISK; SEARCH ENGINE; STATISTICAL MODEL; STATISTICS;

DATABASES, BIBLIOGRAPHIC; EVIDENCE-BASED MEDICINE; HUMANS; INFORMATION STORAGE AND RETRIEVAL; MODELS, STATISTICAL; QUALITATIVE RESEARCH; REVIEW LITERATURE AS TOPIC; RISK; SEARCH ENGINE; STATISTICS AS TOPIC;

EID: 85043603884     PISSN: None     EISSN: 17592887     Source Type: Journal    
DOI: 10.1002/jrsm.1282     Document Type: Article

References (72)

1. Kuhn TS. The Structure of Scientific Revolutions. Chicago: University of Chicago Press; 1970.
2. Dixon-Woods M, Kirk D, Agarwal S. Vulnerable Groups and Access to Health Care: A Critical Interpretive Review. National Co-ordinating Centre for NHS Service Delivery and Organisation R & D (NCCSDO), London; 2005.
3. Magarey JM. Elements of a systematic review. Int J Nurs Pract. 2001;7(6):376-382. https://doi.org/10.1046/j.1440-172X.2001.00295.x
4. Dixon-Woods M, Cavers D, Agarwal S, et al. Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol. 2006;6(1):35. https://doi.org/10.1186/1471-2288-6-35
5. Booth A. Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Syst Rev. 2016;5(1):74. https://doi.org/10.1186/s13643-016-0249-x
6. O'Leary BC, Kvist K, Bayliss HR, et al. The reliability of evidence review methodology in environmental science and conservation. Environ Sci Policy. 2016;64:75-82. https://doi.org/10.1016/j.envsci.2016.06.012
7. Hammerstrøm K, Wade A, Jørgensen A-MK. Searching for studies: a guide to information retrieval for Campbell Systematic Reviews. Campbell Systematic Reviews 2010: Supplement 1. Oslo: The Campbell Collaboration; 2010.
8. Thompson J, Davis J, Mazerolle L. A systematic method for search term selection in systematic reviews. Res Synth Methods. 2014;5(2):87-97. https://doi.org/10.1002/jrsm.1096
9. Booth A, Harris J, Croot E, Springett J, Campbell F, Wilkins E. Towards a methodology for cluster searching to provide conceptual and contextual “richness” for systematic reviews of complex interventions: case study (CLUSTER). BMC Med Res Methodol. 2013b;13(1):118. https://doi.org/10.1186/1471-2288-13-118
10. Shemilt I, Simon A, Hollands GJ, et al. Pinpointing needles in giant haystacks: use of text mining to reduce impractical screening workload in extremely large scoping reviews. Res Synth Methods. 2014;5(1):31-49. https://doi.org/10.1002/jrsm.1093
11. Crane TA, Delaney A, Tamás PA, Chesterman S, Ericksen P. A systematic review of local vulnerability to climate change in developing country agriculture. Wiley Interdiscip Rev Clim Change. 2017;8(4):E464. https://doi.org/10.1002/wcc.464
12. Delaney A, Evans T, John M, et al. Strengthening the food systems governance evidence base: supporting commensurability of research through a systematic review of methods (no. 167), Working Paper. CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS), Copenhagen, Denmark; 2016.
13. Boell SK, Cecez-Kecmanovic D. On being “systematic” in literature reviews in IS. J Inf Technol. 2014;30(2):161-173. https://doi.org/10.1057/jit.2014.26
14. Ripple AM, Mork JG, Rozier JM, Knecht LS. Structured abstracts in MEDLINE: twenty-five years later. National Library of Medicine; 2012.
15. Evans D. Database searches for qualitative research. J Med Libr Assoc. 2002;90(3):290-293.
16. Macpherson A, Holt R. Knowledge, learning and small firm growth: a systematic review of the evidence. Res Policy. 2007;36(2):172-192. https://doi.org/10.1016/j.respol.2006.10.001
17. Shaw RL, Booth A, Sutton AJ, et al. Finding qualitative research: an evaluation of search strategies. BMC Med Res Methodol. 2004;4(1):5. https://doi.org/10.1186/1471-2288-4-5
18. Waddington H, White H, Snilstveit B, et al. How to do a good systematic review of effects in international development: a tool kit. J Dev Eff. 2012;4(3):359-387. https://doi.org/10.1080/19439342.2012.711765
19. Delaney A, Tamás PA, Tobi H. Which standards from which disciplines? A test of systematic review for designing quality interdisciplinary evaluations. J Dev Eff. 2017;9(1):82-100. https://doi.org/10.1080/19439342.2016.1160419
20. Mallett R, Hagen-Zanker J, Slater R, Duvendack M. The benefits and challenges of using systematic reviews in international development research. J Dev Eff. 2012;4(3):445-455. https://doi.org/10.1080/19439342.2012.711342
21. Hartley J, Benjamin M. An evaluation of structured abstracts in journals published by the British Psychological Society. Br J Educ Psychol. 1998;68(3):443-456. https://doi.org/10.1111/j.2044-8279.1998.tb01303.x
22. Bing-Jonsson PC, Bjørk IT, Hofoss D, Kirkevold M, Foss C. Instruments measuring nursing staff competence in community health care: a systematic literature review. Home Health Care Manag Pract. 2013;25(6):282-294. https://doi.org/10.1177/1084822313494784
23. Dubois C-A, D'Amour D, Pomey M-P, Girard F, Brault I. Conceptualizing performance of nursing care as a prerequisite for better measurement: a systematic and interpretive review. BMC Nurs. 2013;12(7). https://doi.org/10.1186/1472-6955-12-7
24. van der Lee J, Mokkink L, Grootenhuis M, Heymans H, Offringa M. Definitions and measurement of chronic health conditions in childhood: a systematic review. JAMA J Am Med Assoc. 2007;297:2741-2751.
25. Stephan BCM, Minett T, Pagett E, Siervo M, Brayne C, McKeith IG. Diagnosing mild cognitive impairment (MCI) in clinical trials: a systematic review. BMJ Open. 2013;3:e001909 https://doi.org/10.1136/bmjopen-2012-001909, 2, e001909
26. Le Reste JY, Nabbe P, Manceau B, et al. The European general practice research network presents a comprehensive definition of multimorbidity in family medicine and long term care, following a systematic review of relevant literature. J Am Med Dir Assoc. 2013;14(5):319-325. https://doi.org/10.1016/j.jamda.2013.01.001
27. Eriksson T. Methodological issues in dynamic capabilities research—a critical review. Balt J Manag. 2013;8(3):306-327. https://doi.org/10.1108/BJOM-Jul-2011-0072
28. Sterne JAC, Egger M, Moher D. Chapter 10.4 Detecting reporting biases. In: Higgins J, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [Updated March 2011]. London: The Cochrane Collaboration; 2011.
29. Lexchin J. Sponsorship bias in clinical research. Int J Risk Saf Med. 2012;24(4):233-242. https://doi.org/10.3233/JRS-2012-0574
30. Stansfield C, Brunton G, Rees R. Search wide, dig deep: literature searching for qualitative research. An analysis of the publication formats and information sources used for four systematic reviews in public health. Res Synth Methods. 2014;5(2):142-151. https://doi.org/10.1002/jrsm.1100
31. Guruge S, Jayasuriya-Illesinghe V, Gunawardena N. A review of the Sri Lankan health-sector response to intimate partner violence: looking back, moving forward. WHO South-East Asia J Public Hea. 2015;4(1):6-11.
32. Stewart R, van Rooyen C, de Wet T. Purity or pragmatism? Reflecting on the use of systematic review methodology in development. J Dev Eff. 2012;4(3):430-444. https://doi.org/10.1080/19439342.2012.711341
33. Lane D, Dykeman J, Ferri M, Goldsmith CH, Stelfox HT. Capture-mark-recapture as a tool for estimating the number of articles available for systematic reviews in critical care medicine. J Crit Care. 2013;28(4):469-475. https://doi.org/10.1016/j.jcrc.2012.10.071
34. Cooper H. Research Synthesis and Meta-analysis: A Step-by-Step Approach. Applied Social Research Methods Series. 5th ed. California: SAGE; 2017.
35. Higgins J, Green S (Eds). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. London: The Cochrane Collaboration; 2011.
36. Reveiz L, Cardona AF, Ospina EG, de Agular S. An e-mail survey identified unpublished studies for systematic reviews. J Clin Epidemiol. 2006;59(7):755-758. https://doi.org/10.1016/j.jclinepi.2005.11.022
37. Song F, Loke Y, Hooper L. Why are medical and health-related studies not being published? A systematic review of reasons given by investigators. PLoS One. 2014;9, e110418. https://doi.org/10.1371/journal.pone.0110418(10):e110418
38. Cooke A, Smith D, Booth A. Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual Health Res. 2012;22(10):1435-1443. https://doi.org/10.1177/1049732312452938
39. Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14(1):579. https://doi.org/10.1186/s12913-014-0579-0
40. Wagemann C, Schneider CQ. Set-Theoretic Methods for the Social Sciences: A Guide to Qualitative Comparative Analysis. New York: Cambridge University Press; 2012.
41. Thomas J, O'Mara-Eves A, Brunton G. Using qualitative comparative analysis (QCA) in systematic reviews of complex interventions: a worked example. Syst Rev. 2014;3(1):1-14. https://doi.org/10.1186/2046-4053-3-67
42. Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8(1):45. https://doi.org/10.1186/1471-2288-8-45
43. Booth A, Carroll C, Ilott I, Low LL, Cooper K. Desperately seeking dissonance identifying the disconfirming case in qualitative evidence synthesis. Qual Health Res. 2013a;23(1):126-141. https://doi.org/10.1177/1049732312466295
44. Noyes J, Popay J, Pearson A, Hannes K, Booth A. Chapter 20: Qualitative research and Cochrane reviews. In: Higgins J, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [Updated March 2011]. London: The Cochrane Collaboration; 2011.
45. Kavanagh J, Campbell F, Harden A, Thomas J. Mixed methods synthesis: a worked example. In: Hannes K, Lockwood C, eds. Synthesizing Qualitative Research: Choosing the Right Approach. Chichester: BMJ Books; 2012:113-136. https://doi.org/10.1002/9781119959847.ch6.
46. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10(1_suppl):21-34. https://doi.org/10.1258/1355819054308530
47. Berrang-Ford L, Pearce T, Ford JD. Systematic review approaches for climate change adaptation research. Reg Environ Change. 2015;15(5):755-769. https://doi.org/10.1007/s10113-014-0708-7
48. Jones L, Champalle C, Chesterman S, Cramer L, Crane TA. Constraining and enabling factors to using long-term climate information in decision-making. Clim Policy. 2016;0(5):1-22. https://doi.org/10.1080/14693062.2016.1191008
49. Bates MJ. The design of browsing and berrypicking techniques for the online search interface. Online Rev. 1989;13(5):407-424. https://doi.org/10.1108/eb024320
50. Sandelowski M, Docherty S, Emden C. Focus on qualitative methods. Qualitative metasynthesis: issues and techniques. Res Nurs Health. 1997;20(4):365-371. https%3A%2F%2Fdoi.org%2F10.1002%2F(SICI)1098-240X(199708)20%3A4%253C365%3A%3AAID-NUR9%253E3.0.CO%3B2-E
51. Finfgeld-Connett D, Johnson ED. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews: discussion paper. J Adv Nurs. 2013;69(1):194-204. https://doi.org/10.1111/j.1365-2648.2012.06037.x
52. Kuruppu N, Willie R. Barriers to reducing climate enhanced disaster risks in least developed country-small islands through anticipatory adaptation. Weather Clim Extrem, SI: IGBP APN. 2015;7:72-83. https://doi.org/10.1016/j.wace.2014.06.001
53. Plummer R, Armitage DR, de Loe RC. Adaptive comanagement and its relationship to environmental governance. Ecol Soc. 2013;18(1):21. https://doi.org/10.5751/ES-05383-180121
54. Porter JJ, Dessai S, Tompkins EL. What do we know about UK household adaptation to climate change? A systematic review. Clim Change. 2014;127(2):371-379. https://doi.org/10.1007/s10584-014-1252-7
55. Bernier P, Fenner RA, Ainger C. Assessing the sustainability merits of retrofitting existing homes. Proc Inst Civil Eng - Eng Sustain. 2010;163(4):197-207.
56. Glenk K, Fischer A. Insurance, prevention or just wait and see? Public preferences for water management strategies in the context of climate change. Ecol Econ. 2010;69(11):2279-2291. https://doi.org/10.1016/j.ecolecon.2010.06.022
57. Gul MS, Menzies GF. Designing domestic buildings for future summers: attitudes and opinions of building professionals. Energy Policy. 2012;45:752-761. https://doi.org/10.1016/j.enpol.2012.03.046
58. Harries T. The anticipated emotional consequences of adaptive behaviour-impacts on the takeup of household flood-protection measures. Environment and Planning A. 2012;44(3):649-668. https://doi.org/10.1068/a43612
59. Chapman AL, Morgan LC, Gartlehner G. Semi-automating the manual literature search for systematic reviews increases efficiency. Health Info Libr J. 2010;27(1):22-27. https://doi.org/10.1111/j.1471-1842.2009.00865.x
60. Pearson M, Moxham T, Ashton K. Effectiveness of search strategies for qualitative research about barriers and facilitators of program delivery. Eval Health Prof. 2011;34(3):297-308. https://doi.org/10.1177/0163278710388029
61. Preston L, Carroll C, Gardois P, Paisley S, Kaltenthaler E. Improving search efficiency for systematic reviews of diagnostic test accuracy: an exploratory study to assess the viability of limiting to MEDLINE, EMBASE and reference checking. Syst Rev. 2015;4(1):82. https://doi.org/10.1186/s13643-015-0074-7
62. McManus RJ, Wilson S, Delaney BC, et al. Review of the usefulness of contacting other experts when conducting a literature search for systematic reviews. BMJ. 1998;317(7172):1562-1563. https://doi.org/10.1136/bmj.317.7172.1562
63. CADTH. Grey matters: a practical search tool for evidence-based medicine; 2011.
64. Haddaway NR, Bayliss HR. Shades of grey: two forms of grey literature important for reviews in conservation. Biol Conserv. 2015;191:827-829. https://doi.org/10.1016/j.biocon.2015.08.018
65. Delaney A, Evans T, McGreevy J, et al. Governance of food systems across scales in times of social-ecological change: a review of indicators. Food Secur. In press;
66. Rücker G, Reiser V, Motschall E, et al. Boosting qualifies capture-recapture methods for estimating the comprehensiveness of literature searches for systematic reviews. J Clin Epidemiol. 2011;64(12):1364-1372. https://doi.org/10.1016/j.jclinepi.2011.03.008
67. Abilio R, Pereira D, Oliviera C, Morais F, Costa H. Systematic literature review supported by information retrieval techniques: a case study, in: Computing Conference (CLEI), 2014 XL Latin American. Presented at the Computing Conference (CLEI), IEEE, Montevideo; 2014, https://doi.org/10.1109/CLEI.2014.6965144
68. Piroi F, Lipani A, Lupu M, Hanbury A. DASyR(IR)—document analysis system for systematic reviews (in information retrieval), in: 2015 13th International Conference on Document Analysis and Recognition (ICDAR). Presented at the 2015 13th International Conference on Document Analysis and Recognition (ICDAR). 2015, 591–595. https://doi.org/10.1109/ICDAR.2015.7333830
69. Ogilvie D, Hamilton V, Egan M, Petticrew M. Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? J Epidemiol Community Health. 2005;59(9):804-808. https://doi.org/10.1136/jech.2005.034181
70. Savoie I, Helmer D, Green CJ, Kazanjian A. Beyond Medline: reducing bias through extended systematic review search. Int J Technol Assess Health Care. 2003;19(01):168-178. https://doi.org/10.1017/S0266462303000163
71. Booth A. The number needed to retrieve: a practically useful measure of information retrieval? Health Info Libr J. 2006;23(3):229-232. https://doi.org/10.1111/j.1471-1842.2006.00663.x
72. Booth A. How much searching is enough? Comprehensive versus optimal retrieval for technology assessments. Int J Technol Assess Health Care. 2010;26(4):431-435. https://doi.org/10.1017/S0266462310000966