Single-reviewer abstract screening missed 13 percent of relevant studies: a crowd-based, randomized controlled trial

Journal of Clinical Epidemiology

Volumn 121, Issue , 2020, Pages 20-28

  Gartlehner, Gerald ^a,b   Affengruber, Lisa ^a,c   Titscher, Viktoria ^a   Noel Storr, Anna ^d   Dooley, Gordon ^e   Ballarini, Nicolas ^f   König, Franz ^f  

^a DANUBE UNIVERSITY KREMS   (Austria)

^b RTI INTERNATIONAL   (United States)

^c MAASTRICHT UNIVERSITY   (Netherlands)

^d UNIVERSITY OF OXFORD   (United Kingdom)

^e Metaxis Ltd   (United Kingdom)

^f MEDICAL UNIVERSITY OF VIENNA   (Austria)

Author keywords

Accuracy; Literature screening; Randomized controlled trial; Rapid reviews; Systematic reviews

Indexed keywords

ADULT; ARTICLE; EXERCISE; FEMALE; HUMAN; HUMAN EXPERIMENT; MALE; PARALLEL DESIGN; PUBLIC HEALTH; RANDOMIZED CONTROLLED TRIAL (TOPIC); SENSITIVITY AND SPECIFICITY; STANDARD; SYNTHESIS; SYSTEMATIC REVIEW; DEPRESSION; DOCUMENTATION; MEASUREMENT ACCURACY; PEER REVIEW; RANDOMIZATION; REGRESSION ANALYSIS; SAMPLE SIZE;

ABSTRACTING AND INDEXING; ADULT; DATA ACCURACY; DEPRESSION; FEMALE; HUMANS; MALE; PEER REVIEW, RESEARCH; RANDOM ALLOCATION; REGRESSION ANALYSIS; SAMPLE SIZE; SENSITIVITY AND SPECIFICITY; SUGAR-SWEETENED BEVERAGES; SYSTEMATIC REVIEWS AS TOPIC;

EID: 85079216446     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2020.01.005     Document Type: Article

References (23)

1. Institute of Medicine. Finding what works in health care: standards for systematic reviews. 2011, The National Academies Press, Washington, DC Available at http://www.nationalacademies.org/hmd/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews.aspx. (Accessed 26 December 2019)
2. University of York CfRaD. CRD's guidance for undertaking reviews in health care. Available at https://www.york.ac.uk/media/crd/Systematic_Reviews.pdf, 2008. (Accessed 26 December 2019)
3. Institute for Quality and Efficiency in Health. General Methods Version 5.0 2019. Available at https://www.iqwig.de/en/methods/methods-paper.3020.html. (Accessed 26 December 2019)
4. Shea, B.J., Reeves, B.C., Wells, G., Thuku, M., Hamel, C., Moran, J., et al. Amstar 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, 358, 2017, j4008.
5. Whiting, P., Savovic, J., Higgins, J.P., Caldwell, D.M., Reeves, B.C., Shea, B., et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 69 (2016), 225–234.
6. Agency for Healthcare Research and Quality. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Chapters available at: Rockville, MD. 2014 [AHRQ Publication No. 10(4)-EHC063-EF]. Available at https://effectivehealthcare.ahrq.gov/products/methods-guidance-bias/methods. (Accessed 26 December 2019)
7. Campbell Systematic Reviews. Campbell Policies and Guidelines Series No. 3. Methodological expectations of Campbell Collaboration intervention reviews: Conduct standards. Available at https://onlinelibrary.wiley.com/page/journal/18911803/homepage/author-guidelines, 2019. (Accessed 26 December 2019)
8. JHiggins, J.P.T., Lasserson, T., Chandler, J., Tovey, D., Thomas, J., Flemyng, E., et al. Methodological Expectations of Cochrane Intervention Reviews. 2019, Cochrane, London Available at https://community.cochrane.org/sites/default/files/uploads/inline-files/MECIR%20October%202019%20Final%20Online%20version.pdf. (Accessed 26 December 2019)
9. National Institute for Health and Care Excelllence. Developing NICE guidelines: the manual 2014. Available at https://www.nice.org.uk/process/pmg20/chapter/reviewing-research-evidence#identifying-and-selecting-relevant-evidence. (Accessed 26 December 2019)
10. Higgins, J.P.T.T.J., Chandler, J., Cumpston, M., Li, T., Page, M.J., Welch, V.A., (eds.) Cochrane Handbook for Systematic Reviews of Interventions, Version 6 (updated July 2019); [Chapter 4]: Searching for and selecting studies. Cochrane, 2019 Available at https://training.cochrane.org/handbook/current. (Accessed 26 December 2019)
11. Tricco, A.C., Antony, J., Zarin, W., Strifler, L., Ghassemi, M., Ivory, J., et al. A scoping review of rapid review methods. BMC Med, 13, 2015, 224.
12. Waffenschmidt, S., Knelangen, M., Sieben, W., Buhn, S., Pieper, D., Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review. BMC Med Res Methodol, 19, 2019, 132.
13. Stoll, C.R.T., Izadi, S., Fowler, S., Green, P., Suls, J., Colditz, G.A., The value of a second reviewer for study selection in systematic reviews. Res Synth Methods 10 (2019), 539–545.
14. Gartlehner, G., Gaynes, B.N., Forneris, C., Lohr, K.N., Comparative benefits and harms of antidepressant, psychological, complementary, and exercise treatments for major depression. Ann Intern Med, 165, 2016, 454.
15. von Philipsborn, P., Stratil, J.M., Burns, J., Busert, L.K., Pfadenhauer, L.M., Polus, S., et al. Environmental interventions to reduce the consumption of sugar-sweetened beverages and their effects on health. Cochrane Database Syst Rev(6), 2019, CD012292.
16. Cochrane Collaboration. Cochrane Crowd. Available at https://community.cochrane.org/help/data-management-tools/cochrane-crowd, 2019. (Accessed 6 February 2020)
17. Doust, J.A., Pietrzak, E., Sanders, S., Glasziou, P.P., Identifying studies for systematic reviews of diagnostic tests was difficult due to the poor sensitivity and precision of methodologic filters and the lack of information in the abstract. J Clin Epidemiol 58 (2005), 444–449.
18. Edwards, P., Clarke, M., DiGuiseppi, C., Pratap, S., Roberts, I., Wentz, R., Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records. Stat Med 21 (2002), 1635–1640.
19. Pham, M.T., Waddell, L., Rajić, A., Sargeant, J.M., Papadopoulos, A., McEwen, S.A., Implications of applying methodological shortcuts to expedite systematic reviews: three case studies using systematic reviews from agri-food public health. Res Synth Methods 7:4 (2016), 433–446.
20. Shemilt, I., Khan, N., Park, S., Thomas, J., Use of cost-effectiveness analysis to compare the efficiency of study identification methods in systematic reviews. Syst Rev, 5(1), 2016, 140.
21. Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., Moher, D., Evidence summaries: the evolution of a rapid review approach. Syst Rev, 1, 2012, 10.
22. Wagner, G., Nussbaumer-Streit, B., Greimel, J., Ciapponi, A., Gartlehner, G., Trading certainty for speed - how much uncertainty are decisionmakers and guideline developers willing to accept when using rapid reviews: an international survey. BMC Med Res Methodol, 17, 2017, 121.
23. Nussbaumer-Streit, B., Klerings, I., Wagner, G., Heise, T.L., Dobrescu, A.I., Armijo-Olivo, S., et al. Abbreviated literature searches were viable alternatives to comprehensive searches: a meta-epidemiological study. J Clin Epidemiol 102 (2018), 1–11.