Living systematic reviews: 2. Combining human and machine effort

Journal of Clinical Epidemiology

Volumn 91, Issue , 2017, Pages 31-37

  Thomas, James ^a,i   Noel Storr, Anna ^b,i   Marshall, Iain ^c,i   Wallace, Byron ^d,i   McDonald, Steve ⁱ   Mavergames, Chris ^f,i   Glasziou, Paul ^g,i   Shemilt, Ian ^a,i   Synnot, Anneliese ^e,h,i   Turner, Tari ^e,i   Elliott, Julian ^e,i   Agoritsas, Thomas ⁱ   Hilton, John ⁱ   Perron, Caroline ⁱ   Akl, Elie ⁱ   Hodder, Rebecca ⁱ   Pestridge, Charlotte ⁱ   Albrecht, Lauren ⁱ   Horsley, Tanya ⁱ   Platt, Joanne ⁱ   Armstrong, Rebecca ⁱ   Nguyen, Phi Hung ⁱ   Plovnick, Robert ⁱ   Arno, Anneliese ⁱ   Ivers, Noah ⁱ   Quinn, Gail ⁱ   Au, Agnes ⁱ   Johnston, Renea ⁱ   Rada, Gabriel ⁱ   Bagg, Matthew ⁱ   Jones, Arwel ⁱ   Ravaud, Philippe ⁱ   Boden, Catherine ⁱ   Kahale, Lara ⁱ   Richter, Bernt ⁱ   Boisvert, Isabelle ⁱ   Keshavarz, Homa ⁱ   Ryan, Rebecca ⁱ   Brandt, Linn ⁱ   Kolakowsky Hayner, Stephanie A ⁱ   Salama, Dina ⁱ   Brazinova, Alexandra ⁱ   Nagraj, Sumanth Kumbargere ⁱ   Salanti, Georgia ⁱ   Buchbinder, Rachelle ⁱ   Lasserson, Toby ⁱ   Santaguida, Lina ⁱ   Champion, Chris ⁱ   Lawrence, Rebecca ⁱ   Santesso, Nancy ⁱ   Chandler, Jackie ⁱ   Les, Zbigniew ⁱ   Schünemann, Holger J ⁱ   Charidimou, Andreas ⁱ   Leucht, Stefan ⁱ   Chou, Roger ⁱ   Low, Nicola ⁱ   Sherifali, Diana ⁱ   Churchill, Rachel ⁱ   Maas, Andrew ⁱ   Siemieniuk, Reed ⁱ   Cnossen, Maryse C ⁱ   MacLehose, Harriet ⁱ   Simmonds, Mark ⁱ   Cossi, Marie Joelle ⁱ   Macleod, Malcolm ⁱ   Skoetz, Nicole ⁱ   Counotte, Michel ⁱ   Soares Weiser, Karla ⁱ   Craigie, Samantha ⁱ   Marshall, Rachel ⁱ   Srikanth, Velandai ⁱ   Dahm, Philipp ⁱ   Martin, Nicole ⁱ   Sullivan, Katrina ⁱ   Danilkewich, Alanna ⁱ   García, Laura Martínez ⁱ   Danko, Kristen ⁱ   Taylor, Mark ⁱ   Donoghue, Emma ⁱ   Maxwell, Lara J ⁱ   Thayer, Kris ⁱ   Dressler, Corinna ⁱ   McAuley, James ⁱ   Egan, Cathy ⁱ   Tritton, Roger ⁱ   McKenzie, Joanne ⁱ   Tsafnat, Guy ⁱ   Elliott, Sarah A ⁱ   Meerpohl, Joerg ⁱ   Tugwell, Peter ⁱ   Etxeandia, Itziar ⁱ   Merner, Bronwen ⁱ   Featherstone, Robin ⁱ   Mondello, Stefania ⁱ   Foxlee, Ruth ⁱ   Morley, Richard ⁱ   van Valkenhoef, Gert ⁱ   Garner, Paul ⁱ   Munafo, Marcus ⁱ   Vandvik, Per ⁱ   Gerrity, Martha ⁱ   Munn, Zachary ⁱ   Murano, Melissa ⁱ   Wallace, Sheila A ⁱ   Green, Sally ⁱ   Newman, Kristine ⁱ   Watts, Chris ⁱ   Grimshaw, Jeremy ⁱ   Nieuwlaat, Robby ⁱ   Weeks, Laura ⁱ   Gurusamy, Kurinchi ⁱ   Nikolakopoulou, Adriani ⁱ   Weigl, Aaron ⁱ   Haddaway, Neal ⁱ   Wells, George ⁱ   Hartling, Lisa ⁱ   O'Connor, Annette ⁱ   Wiercioch, Wojtek ⁱ   Hayden, Jill ⁱ   Page, Matthew ⁱ   Wolfenden, Luke ⁱ   Helfand, Mark ⁱ   Pahwa, Manisha ⁱ   Yepes Nuñez, Juan José ⁱ   Higgins, Julian ⁱ   Pardo, Jordi Pardo ⁱ   Yost, Jennifer ⁱ   Hill, Sophie ⁱ   Pearson, Leslea ⁱ   more..

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c KING'S COLLEGE LONDON   (United Kingdom)

^d Northeastern University   (United States)

^e MONASH UNIVERSITY   (Australia)

^f Cochrane ^*  (United Kingdom)

^g BOND UNIVERSITY   (Australia)

^h LA TROBE UNIVERSITY   (Australia)

ⁱ Monash University and Alfred Hospital ^*

Author keywords

Automation; Citizen science; Crowdsourcing; Machine learning; Systematic review; Text mining

Indexed keywords

AUTOMATION; COMMUNITY; ECONOMIC EVALUATION; GOLD STANDARD; HUMAN; HUMAN FACTORS RESEARCH; MACHINE LEARNING; PRIORITY JOURNAL; RANDOMIZED CONTROLLED TRIAL (TOPIC); REVIEW; SYSTEMATIC REVIEW (TOPIC); WORKFLOW; DATA MINING; EVIDENCE BASED MEDICINE; LITERATURE; PROCEDURES;

DATA MINING; EVIDENCE-BASED MEDICINE; HUMANS; MACHINE LEARNING; REVIEW LITERATURE AS TOPIC;

EID: 85028965305     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2017.08.011     Document Type: Review

References (15)

1. Higgins, J.P.T., Green, S., editors, Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. 2011, The Cochrane Collaboration Available at www.cochrane-handbook.org.
2. Elliott, J.H., Synnot, A., Turner, T., Simmonds, M., Akl, E., McDonald, S., et al. Living systematic review: 1. Introduction—the why, what, when and how. J Clin Epidemiol 91 (2017), 23–30.
3. Elliott, J.H., Turner, T., Clavisi, O., Thomas, J., Higgins, J.P.T., Mavergames, C., et al. Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Med, 11(2), 2014, e1001603.
4. Thomas, J., Noel-Storr, A., Elliott, J., Human and machine effort in Project Transform: how intersecting technologies will help us to identify studies reliably, efficiently and at scale. Cochrane Methods 1 (2016), 37–41.
5. Wallace, B.C., Noel-Storr, A., Marshall, I.J., Cohen, A.M., Smalheiser, N.R., Thomas, J., Identifying reports of randomized controlled trials (RCTs) via a hybrid machine learning and crowdsourcing approach. J Am Med Inform Assoc 24 (2017), 1165–1168.
6. Marshall, I., Kuiper, J., Wallace, B.C., RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials. J Am Med Inform Assoc 23 (2016), 193–201.
7. Tsafnat, G., Dunn, A., Glasziou, P., Coiera, E., The automation of systematic reviews. BMJ, 346, 2013, f139.
8. Tsafnat, G., Glasziou, P., Choong, M.K., Dunn, A., Galgani, F., Coiera, E., Systematic review automation technologies. Syst Rev, 3(1), 2014, 74.
9. O'Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., Ananiadou, S., Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev, 4(1), 2015, 5.
10. Thomas, J., McNaught, J., Ananiadou, S., Applications of text mining within systematic reviews. Res Synth Methods 2:1 (2011), 1–14.
11. Bastian, H., Glasziou, P., Chalmers, I., Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?. PLoS Med, 7(9), 2010, e1000326.
12. Shemilt, I., Khan, N., Park, S., Thomas, J., Use of cost-effectiveness analysis to compare the efficiency of study identification methods in systematic reviews. Syst Rev, 5(1), 2016, 140.
13. Higgins, J.P.T., Lasserson, T., Chandler, J., Tovey, D., Churchill, R., Methodological Expectations of Cochrane Intervention Reviews (MECIR). Standards for the conduct and reporting of new Cochrane Intervention Reviews, reporting of protocols and the planning, conduct and reporting of updates. 2016, Cochrane, London.
14. Wallace, B.C., Trikalinos, T.A., Lau, J., Brodley, C.E., Schmid, C.H., Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinformatics, 11(1), 2010, 55.
15. Hoffmann, T., Glasziou, P., Beller, E., Goldacre, B., Chalmers, I., Focus on sharing individual patient data distracts from other ways of improving trial transparency. BMJ, 357, 2017, j2782.