Machine learning to assist risk-of-bias assessments in systematic reviews

International Journal of Epidemiology

Volumn 45, Issue 1, 2016, Pages 266-277

  Millard, Louise A C ^a   Flach, Peter A ^a   Higgins, Julian P T ^a  

^a UNIVERSITY OF BRISTOL   (United Kingdom)

Author keywords

Machine learning; Risk of bias; Systematic review; Text mining

Indexed keywords

LITERATURE REVIEW; MACHINE LEARNING; METHODOLOGY; REGRESSION ANALYSIS; RESEARCH WORK; RISK ASSESSMENT;

CLINICAL RESEARCH; DATA MINING; PREDICTION; PRIORITY JOURNAL; REVIEW; RISK OF BIAS ASSESSMENT; SELECTION BIAS; STATISTICAL MODEL; SUPERVISED MACHINE LEARNING; SYSTEMATIC REVIEW; HUMAN; INFORMATION PROCESSING; LITERATURE; MACHINE LEARNING; PROCEDURES; RECEIVER OPERATING CHARACTERISTIC; STATISTICAL BIAS; UTILIZATION;

BIAS (EPIDEMIOLOGY); DATA MINING; DATASETS AS TOPIC; HUMANS; LOGISTIC MODELS; MACHINE LEARNING; REVIEW LITERATURE AS TOPIC; ROC CURVE;

EID: 84963999519     PISSN: 03005771     EISSN: 14643685     Source Type: Journal    
DOI: 10.1093/ije/dyv306     Document Type: Review

References (33)

1. Higgins JP, Green S (eds). Cochrane Handbook for Systematic Reviews of Interventions. Chichester, UK: Wiley, 2008.
2. Schulz KF, Grimes DA. Allocation concealment in randomised trials: defending against deciphering. Lancet 2002;359:614-18.
3. Higgins JP, Altman DG, Gøtzsche PC et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928.
4. Savovic J, Weeks L, Sterne J et al. Evaluation of the Cochrane Collaboration's tool for assessing the risk of bias in randomized trials: focus groups, online survey, proposed recommendations and their implementation. Syst Rev 2014;3:10.1186.
5. Hartling L, Hamm MP, Milne A et al. Testing the Risk of Bias tool showed low reliability between individual reviewers and across consensus assessments of reviewer pairs. J Clin Epidemiol 2013;66:973-81.
6. Hartling L, Ospina M, Liang Y et al. Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ 2009;339:b4012.
7. Hartling L, Bond K, Vandermeer B, Seida J, Dryden DM, Rowe BH. Applying the risk of bias tool in a systematic review of combination long-acting beta-agonists and inhaled corticosteroids for persistent asthma. PLoS One 2011;6:e17242.
8. Lensen S, Farquhar C, Jordon V. Risk of bias: are judgements consistent between reviews? Cochrane Database Syst Rev Suppl 2014;1:30.
9. Armijo-Olivo S, Ospina M, da Costa BR et al. Poor reliability between Cochrane reviewers and blinded external reviewers when applying the Cochrane risk of bias tool in physical therapy trials. PloS One 2014;9:e96920.
10. Tsafnat G, Glasziou P, Choong MK, Dunn A, Galgani F, Coiera E. Systematic review automation technologies. Syst Rev 2014; 3:74.
11. Wallace BC, Trikalinos TA, Lau J, Brodley C, Schmid CH. Semiautomated screening of biomedical citations for systematic reviews. BMC Bioinformatics 2010;11:55.
12. Cohen AM, Hersh WR, Peterson K, Yen P-Y. Reducing workload in systematic review preparation using automated citation classification. J Am Med Inform Assoc 2006;13:206-19.
13. Cohen AM. Optimizing feature representation for automated systematic review work prioritization. AMIA Annu Symp Proc 2008;2008:121-15.
14. Thomas J, McNaught J, Ananiadou S. Applications of text mining within systematic reviews. ResSynthesis Methods 2011;2:1-14.
15. Cohen AM, Adams CE,Davis JM et al. (eds). Evidence-basedmedicine, the essential role of systematic reviews, and the need for automated text mining tools. Proceedings of the 1st ACM international Health Informatics Symposium 11-12 November 2010. Arlington, VA: Association for ComputingMachinery, 2010.
16. Ananiadou S, Rea B, Okazaki N, Procter R, Thomas J. Supporting systematic reviews using text mining. Soc Sci Comput Rev 2009;27:509-23.
17. Matwin S, Kouznetsov A, Inkpen D, Frunza O, O'Blenis P. A new algorithm for reducing the workload of experts in performing systematic reviews. J Am Med Inform Assoc 2010;17:446-53.
18. Kiritchenko S, de Bruijn B, Carini S, Martin J, Sim I. ExaCT:automatic extraction of clinical trial characteristics from journal publications. BMC Med Inform Decis Mak 2010;10:56.
19. de Bruijn B, Carini S, Kiritchenko S, Martin J, Sim I. Automated information extraction of key trial design elements from clinical trial publications. AMIA Annu Symp Proc 2008;2008:141-45.
20. Hsu W, Speier W, Taira RK. Automated extraction of reported statistical analyses: towards a logical representation of clinical trial literature. AMIA Annu Symp Proc 2012;2012:350-59.
21. Zhao J, Kan M-Y, Procter PM, Zubaidah S, Yip WK, Li GM (eds). Improving search for evidence-based practice using information extraction. AMIA Annu Symp Proc 2010;2010:937-41.
22. Hassanzadeh H, Groza T, Hunter J. Identifying scientific artefacts in biomedical literature: the Evidence Based Medicine use case. J Biomed Inform 2014;49:159-70.
23. Marshall IJ, Kuiper J, Wallace BC. Automating risk of bias assessment for clinical trials. Proceedings of the 5th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics, 26-28 June 2014, Cambridge UK. New York, NY: Association for Computing Machinery, 2014.
24. Kuiper J, Marshall I, Wallace B, Swertz M. Spá: A web-based viewer for text mining in evidence based medicine. In: Machine Learning and Knowledge Discovery in Databases. New York, NY: Springer, 2014.
25. Marshall IJ, Kuiper J, Wallace BC. RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials. J Am Med Inform Assoc 2015 Jun 22. pii: ocv044. doi: 10.1093/ jamia/ocv044. [Epub ahead of print.]
26. Thomas J, Newman M, Oliver S. Rapid evidence assessments of research to inform social policy: taking stock and moving forward. Evidence&Policy 2013;9:5-27.
27. Millard LA, Flach PA, Higgins JP. Rate-constrained ranking and the rate-weighted AUC. In: Machine Learning and Knowledge Discovery in Databases. New York, NY: Springer, 2014.
28. Niculescu-Mizil A, Caruana R. Predicting good probabilities with supervised learning. Proceedings of the 22nd International Conference on Machine Learning. New York, NY: Association for Computing Machinery, 2005.
29. Flach PA. ROC analysis. In: Encyclopedia of Machine Learning. New York, NY: Springer, 2011.
30. Fawcett T. An introduction to ROC analysis. Pattern Recognition Letters 2006;27:861-74.
31. Hopewell S, Dutton S, Yu L-M, Chan A-W, Altman DG. The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. BMJ 2010;340:c723.
32. Pildal J, Chan A-W, Hróbjartsson A, Forfang E, Altman DG, Gøtzsche PC. Comparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study. BMJ 2005;330:1049.
33. Millard LA, Kull M, Flach PA. Rate-oriented point-wise confidence bounds for ROC curves. In: Machine Learning and Knowledge Discovery in Databases. Berlin Heidelberg: Springer, 2014, pp. 404-21.