Using text mining for study identification in systematic reviews: A systematic review of current approaches

Systematic Reviews

Volumn 4, Issue 1, 2015, Pages

  O'Mara Eves, Alison ^a   Thomas, James ^a   McNaught, John ^b   Miwa, Makoto ^c   Ananiadou, Sophia ^b  

^a UNIVERSITY OF LONDON   (United Kingdom)

^b UNIVERSITY OF MANCHESTER   (United Kingdom)

^c TOYOTA TECHNOLOGICAL INSTITUTE   (Japan)

Author keywords

Automation; Review efficiency; Screening; Study selection; Text mining

Indexed keywords

ALGORITHM; ARTICLE; DATA BASE; DATA EXTRACTION; DATA PROCESSING; HASTY GENERALISATION; KNOWLEDGE; LEARNING; PRIORITY JOURNAL; PUBLICATION; SCREENING; SYSTEMATIC REVIEW; TEXT MINING; WORKFLOW; WORKLOAD; BIOLOGY; DATA MINING; EVIDENCE BASED MEDICINE; FACTUAL DATABASE; HUMAN; INFORMATION RETRIEVAL; PROCEDURES; TRENDS;

COMPUTATIONAL BIOLOGY; DATA MINING; DATABASES, FACTUAL; EVIDENCE-BASED MEDICINE; HUMANS; INFORMATION STORAGE AND RETRIEVAL; PUBLICATIONS;

EID: 84937924687     PISSN: None     EISSN: 20464053     Source Type: Journal    
DOI: 10.1186/2046-4053-4-5     Document Type: Article

References (93)

1. Gough D, Elbourne D: Systematic research synthesis to inform policy, practice and democratic debate. Soc Policy Soc 2002, 1:225-36.
2. Gough D, Oliver S, Thomas J: An Introduction to Systematic Reviews. London: Sage; 2012.
3. Gough D, Thomas J, Oliver S: Clarifying differences between review designs and methods. Syst Rev 2012.,1(28): doi:10.1186/2046-4053-1-28
4. Chalmers I, Hedges L, Cooper H: A brief history of research synthesis. Eval Health Prof 2002, 25:12-37. 10.1177/0163278702025001003
5. Mulrow C: Rationale for systematic reviews. BMJ 1994, 309:597-9. 10.1136/bmj.309.6954.597
6. Bastian H, Glasziou P, Chalmers I: Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med 2010.,7(9):
7. Lefebvre C, Manheimer E, Glanville J: Searching for studies (chapter 6). In Cochrane Handbook for Systematic Reviews of Interventions Version 510 [updated March 2011]. Edited by: Higgins J, Green S. Oxford: The Cochrane Collaboration; 2011.
8. Gomersall A, Cooper C: Database selection bias and its affect on systematic reviews: a United Kingdom perspective. In Joint Colloquium of the Cochrane and Campbell Collaborations. Keystone, Colorado: The Campbell Collaboration; 2010.
9. Harden A, Peersman G, Oliver S, Oakley A: Identifying primary research on electronic databases to inform decision-making in health promotion: the case of sexual health promotion. Health Educ J 1999, 58:290-301. 10.1177/001789699905800310
10. Sampson M, Barrowman N, Moher D, Clifford T, Platt R, Morrison A, et al.: Can electronic search engines optimize screening of search results in systematic reviews: an empirical study. BMC Med Res Methodol 2006.,6(7)
11. Wallace B, Trikalinos T, Lau J, Brodley C, Schmid C: Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinformatics 2010.,11(55):
12. Allen I, Olkin I: Estimating time to conduct a meta-analysis from number of citations retrieved. JAMA 1999,282(7):634-5. 10.1001/jama.282.7.634
13. Felizardo K, Andery G, Paulovich F, Minghim R, Maldonado J: A visual analysis approach to validate the selection review of primary studies in systematic reviews. Inf Softw Technol 2012,54(10):1079-91. 10.1016/j.infsof.2012.04.003
14. Malheiros V, Hohn E, Pinho R, Mendonca M: A visual text mining approach for systematic reviews. In Empirical Software Engineering and Measurement, 2007 ESEM 2007 First International Symposium on: 2007 2007. Piscataway: IEEE; 2007:245-54.
15. Miroslav K, Matwin S: Addressing the curse of imbalanced training sets: one-sided selection. Proceedings of the Fourteenth International Conference on Machine Learning: 1997 1997.
16. Watt A, Cameron A, Sturm L, Lathlean T, Babidge W, Blamey S, et al.: Rapid reviews versus full systematic reviews: an inventory of current methods and practice in health technology assessment. Int J Technol Assess Health Care 2008,24(2):133-9.
17. Ananiadou S, McNaught J: Text Mining for Biology and Biomedicine. Boston/London: Artech House; 2006.
18. Hearst M: Untangling Text Data Mining. Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics (ACL 1999): 1999 1999, 3-10.
19. Thomas J, McNaught J, Ananiadou S: Applications of text mining within systematic reviews. Res Synth Methods 2011,2(1):1-14. 10.1002/jrsm.27
20. Ananiadou S, Okazaki N, Procter R, Rea B, Sasaki Y, Thomas J: Supporting systematic reviews using text mining. Soc Sci Comput Rev 2009, 27:509-23. 10.1177/0894439309332293
21. Thomas J: Diffusion of innovation in systematic review methodology: why is study selection not yet assisted by automation? OA Evid Based Med 2013,1(2):12.
22. Thomas J, Brunton J, Graziosi S: EPPI-Reviewer 4.0: Software for Research Synthesis. London: EPPI-Centre Software, Social Science Research Unit, Institute of Education; 2010.
23. Jonnalagadda S, Petitti D: A new iterative method to reduce workload in systematic review process. Int J Comput Biol Drug Des 2013,6(1-2):5-17.
24. Frunza O, Inkpen D, Matwin S: Building systematic reviews using automatic text classification techniques. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters: 2010 2010. Beijing China: Association for Computational Linguistics; 2010:303-11.
25. Wallace B, Small K, Brodley C, Trikalinos T: Active learning for biomedical citation screening. KDD 2010; Washington USA 2010.
26. Lavoie M, Verbeek J: Devices for preventing percutaneous exposure injuries caused by needles in healthcare personnel. Cochrane Database Syst Rev 2014.,2014(3):
27. Mischke C, Verbeek J, Saarto A, Lavoie MC, Pahwa M, Ijaz S: Gloves, extra gloves or special types of gloves for preventing percutaneous exposure injuries in healthcare personnel. Cochrane Database Syst Rev 2014.,2014(3):
28. Martin A, Saunders D, Shenkin S, Sproule J: Lifestyle intervention for improving school achievement in overweight or obese children and adolescents. Cochrane Database Syst Rev 2014.,2014(3):
29. Fletcher-Watson S, McConnell F, Manola E, McConachie H: Interventions based on the Theory of Mind cognitive model for autism spectrum disorder (ASD). Cochrane Database Syst Rev 2014.,2014(3):
30. Bekhuis T, Demner-Fushman D: Screening nonrandomized studies for medical systematic reviews: a comparative study of classifiers. Artif Intell Med 2012,55(3):197-207. 10.1016/j.artmed.2012.05.002
31. Shemilt I, Simon A, Hollands G, Marteau T, Ogilvie D, O'Mara-Eves A, et al.: Pinpointing needles in giant haystacks: use of text mining to reduce impractical screening workload in extremely large scoping reviews. Res Synth Methods 2013, 13:1218. n/a-n/a
32. Hammerstrøm K, Wade A, Jørgensen A: Searching for Studies: A Guide to Information Retrieval for Campbell Systematic Reviews. Keystone, Colorado: Campbell Collaboration; 2010.
33. Institute of Medicine of the National Academies: Finding what works in health care: standards for systematic reviews. Washington, DC: Institute of Medicine of the National Academies; 2011.
34. Cohen A: Performance of support-vector-machine-based classification on 15 systematic review topics evaluated with the WSS@95 measure. J Am Med Inform Assoc 2011, 18:104-4.
35. Cohen A, Ambert K, McDonagh M: A prospective evaluation of an automated classification system to support evidence-based medicine and systematic review. AMIA Annual Symposium 2010, 121-5.
36. Cohen A, Hersh W, Peterson K, Yen P-Y: Reducing workload in systematic review preparation using automated citation classification. J Am Med Inform Assoc 2006,13(2):206-19. 10.1197/jamia. M1929
37. Cohen A: An effective general purpose approach for automated biomedical document classification. In AMIA Annual Symposium Proceedings, vol. 13. Washington, DC: American Medical Informatics Association; 2006:206-19.
38. Fiszman M, Ortiz E, Bray BE, Rindflesch TC: Semantic Processing to Support Clinical Guideline Development. AMIA 2008 Symposium Proceedings: 2008 2008 2008, 187-91.
39. Kim S, Choi J: Improving the performance of text categorization models used for the selection of high quality articles. Healthc Informatics Res 2012,18(1):18-28. 10.4258/hir.2012.18.1.18
40. Ma Y: Text Classification on Imbalanced Data: Application to Systematic Reviews Automation. Ottawa: University of Ottawa; 2007.
41. Matwin S, Kouznetsov A, Inkpen D, Frunza O, O'Blenis P: A new algorithm for reducing the workload of experts in performing systematic reviews. J Am Med Inform Assoc 2010,17(4):446-53. 10.1136/jamia.2010.004325
42. Matwin S, Kouznetsov A, Inkpen D, Frunza O, O'Blenis P: Performance of SVM and Bayesian classifiers on the systematic review classification task. J Am Med Inform Assoc 2011, 18:104-5.
43. Matwin S, Sazonova V: Correspondence. J Am Med Inform Assoc 2012, 19:917--7. 10.1136/amiajnl-2012-001072
44. Razavi A, Matwin S, Inkpen D, Kouznetsov A: Parameterized Contrast in Second Order Soft Co-Occurrences: A Novel Text Representation Technique in Text Mining and Knowledge Extraction. In 2009 Ieee International Conference on Data Mining Workshops: 2009 2009. New York: Ieee; 2009:471-6.
45. Miwa M, Thomas J, O'Mara-Eves A, Ananiadou S: Reducing systematic review workload through certainty-based screening. J Biomed Inform 2014, 51:242-53. doi:10.1016/j.jbi.2014.06.005
46. Sun Y, Yang Y, Zhang H, Zhang W, Wang Q: Towards evidence-based ontology for supporting Systematic Literature Review. In Proceedings of the EASE Conference 2012: 2012 2012. Ciudad Real Spain: IET; 2012.
47. Tomassetti F, Rizzo G, Vetro A, Ardito L, Torchiano M, Morisio M: Linked data approach for selection process automation in systematic reviews. Evaluation & Assessment in Software Engineering (EASE 2011), 15th Annual Conference on: 2011 2011; Durham 2011, 31-5.
48. Wallace B, Small K, Brodley C, Lau J, Schmid C, Bertram L, et al.: Toward modernizing the systematic review pipeline in genetics: efficient updating via data mining. Genet Med 2012, 14:663-9. 10.1038/gim.2012.7
49. Wallace B, Small K, Brodley C, Lau J, Trikalinos T: Modeling Annotation Time to Reduce Workload in Comparative Effectiveness Reviews. Proc ACM International Health Informatics Symposium: 2010 2010 2010, 28-35.
50. Wallace B, Small K, Brodley C, Lau J, Trikalinos T: Deploying an interactive machine learning system in an evidence-based practice center: abstrackr. In Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium: 2012. New York: ACM; 2012:819-24.
51. Yu W, Clyne M, Dolan S, Yesupriya A, Wulf A, Liu T, et al.: GAPscreener: an automatic tool for screening human genetic association literature in PubMed using the support vector machine technique. BMC Bioinformatics 2008.,205(9):
52. Choi S, Ryu B, Yoo S, Choi J: Combining relevancy and methodological quality into a single ranking for evidence-based medicine. Inf Sci 2012, 214:76-90.
53. Fiszman M, Bray BE, Shina D, Kilicoglu H, Bennett GC, Bodenreider O, et al.: Combining relevance assignment with quality of the evidence to support guideline development. Stud Health Technol Inform 2010,160(1):709-13.
54. Kouznetsov A, Japkowicz N: Using classifier performance visualization to improve collective ranking techniques for biomedical abstracts classification. In Advances in Artificial Intelligence, Proceedings: 2010. Berlin: Springer-Verlag Berlin; 2010:299-303.
55. Kouznetsov A, Matwin S, Inkpen D, Razavi A, Frunza O, Sehatkar M, et al.: Classifying biomedical abstracts using committees of classifiers and collective ranking techniques. In Advances in Artificial Intelligence, Proceedings: 2009. Berlin: Springer-Verlag Berlin; 2009:224-8.
56. Martinez D, Karimi S, Cavedon L, Baldwin T: Facilitating biomedical systematic reviews using ranked text retrieval and classification. Proceedings of the 13th Australasian Document Computing Symposium: 2008; Hobart Australia 2008, 53.
57. Thomas J, O'Mara A: How can we find relevant research more quickly? In NCRM MethodsNews. UK: NCRM; 2011:3.
58. Wallace B, Small K, Brodley C, Trikalinos T: Who should label what? Instance allocation in multiple expert active learning. Proc SIAM International Conference on Data Mining: 2011 2011, 176-87.
59. Bekhuis T, Demner-Fushman D: Towards automating the initial screening phase of a systematic review. Stud Health Technol Inform 2010,160(1):146-50.
60. Bekhuis T, Tseytlin E, Mitchell K, Demner-Fushman D: Feature engineering and a proposed decision-support system for systematic reviewers of medical evidence. PLoS One 2014,9(1):e86277. 10.1371/journal.pone.0086277
61. Frunza O, Inkpen D, Matwin S, Klement W, O'Blenis P: Exploiting the systematic review protocol for classification of medical abstracts. Artif Intell Med 2011,51(1):17-25. 10.1016/j.artmed.2010.10.005
62. García Adevaa J, Pikatza-Atxa J, Ubeda-Carrillo M, Ansuategi-Zengotitabengoa E: Automatic text classification to support systematic reviews in medicine. Expert Syst Appl 2014,41(4):1498-508. 10.1016/j.eswa.2013.08.047
63. Felizardo K, Maldonado J, Minghim R, MacDonell S, Mendes E: An extension of the systematic literature review process with visual text mining: a case study on software engineering. 16. Unpublished. Downloadable from: (p.164) http://www.teses.usp.br/teses/disponiveis/55/55134/tde-18072012-102032/publico/Thesis.pdf
64. Felizardo K, Salleh N, Martins R, Mendes E, MacDonell S, Maldonado J: Using visual text mining to support the study selection activity in systematic literature reviews. Empirical Software Engineering and Measurement (ESEM), 2011 International Symposium on: 2011; Banff2011, 77-86.
65. Felizardo R, Souza S, Maldonado J: The use of visual text mining to support the study selection activity in systematic literature reviews: a replication study. Replication in Empirical Software Engineering Research (RESER), 2013 3rd International Workshop on: 2013; Baltimore 2013, 91-100.
66. Cohen A, Ambert K, McDonagh M: Cross-topic learning for work prioritization in systematic review creation and update. J Am Med Inform Assoc 2009, 16:690-704. 10.1197/jamia. M3162
67. Brunton G, Caird J, Sutcliffe K, Rees R, Stokes G, Stansfield C, et al.: Depression, Anxiety, Pain and Quality of Life in People Living with Chronic Hepatitis C: A Systematic Review and Meta-Analysis. London: EPPI Centre, Social Science Research Unit, Institute of Education, University of London; 2014.
68. Cohen A: Optimizing feature representation for automated systematic review work prioritization. AMIA Annual Symposium Proceedings: 2008 2008, 121-5.
69. Cohen A, Ambert K, McDonagh M: Studying the potential impact of automated document classification on scheduling a systematic review update. BMC Med Inform Decis Mak 2012,12(1):33. 10.1186/1472-6947-12-33
70. Dalal S, Shekelle P, Hempel S, Newberry S, Motala A, Shetty K: A pilot study using machine learning and domain knowledge to facilitate comparative effectiveness review updating. Med Decis Making 2013,33(3):343-55. 10.1177/0272989X12457243
71. Small K, Wallace B, Brodley C, Trikalinos T: The constrained weight space SVM: learning with ranked features. In Proceedings of the 28th International Conference on Machine Learning. Bellevue, WA, USA: ICML; 2011.
72. Sampson M, TetzlaffJ, Urquhart C: Precision of healthcare systematic review searches in a cross-sectional sample. Res Synth Methods 2011, 2:119-25. 10.1002/jrsm.42
73. Sasaki Y: Automatic text classification. University of Manchester: presentation 2008.
74. Tomek I: Two modifications of CNN. IEEE Trans Syst Man Cybern 1976, SMC-6(11):769-72.
75. Brinker K: Incorporating diversity in active learning with support vector machines. In Proceedings of the 20th International Conference on Machine Learning: 2003. Palo Alto: AAAI Press; 2003:59-66.
76. Gama J, Žliobaite A, Bifet A, Pechenizkiy M, Bouchachia A: A survey on concept drift adaptation. ACM Comput Surv (CSUR) 2014,46(4):44.
77. Pan S, Qiang Y: A survey on transfer learning. Knowledge and Data Engineering. IEEE Trans Syst Man Cybern 2010,22(10):1345-59.
78. Davis J, Goadrich M: The relationship between Precision-Recall and ROC curves. In ICML '06 Proceedings of the 23rd international conference on Machine learning 2006. New York, NY, USA: ACM; 2006.
79. García V, Mollineda R, Sánchez J: A bias correction function for classification performance assessment in two-class imbalanced problems. Knowl Based Syst 2014, 59:66-74.
80. Tsafnat G, Dunn A, Glasziou P, Coiera E: The automation of systematic reviews. BMJ 2013.,346(f139):
81. Settles B: Active Learning Literature Survey. Computer Sciences Technical Report 1648. Wisconsin: University of Wisconsin-Madison; 2009.
82. Sarveniazi A: An actual survey of dimensionality reduction. Am J Comput Math 2014, 4:55-72. 10.4236/ajcm.2014.42006
83. Elkan C: The foundations of cost-sensitive learning. In International Joint Conference on Artificial Intelligence: 2001. Seattle, Washington: Morgan Kaufmann Publishers Inc; 2001.
84. Cao P, Zhao D, Zaiane O: An optimized cost-sensitive SVM for imbalanced data learning. In Advances in Knowledge Discovery and Data Mining: 2013. Berlin Heidelberg: Springer; 2013:280-92.
85. Margineantu D: Active cost-sensitive learning. In Proceedings of the 19th International Joint Conference on Artificial Intelligence: 2005. Burlington: Morgan Kaufmann Publishers Inc; 2005.
86. Blake C: Beyond genes, proteins, and abstracts: identifying scientific claims from full-text biomedical articles. J Biomed Inform 2010, 43:173-89. 10.1016/j.jbi.2009.11.001
87. Cohen K, Johnson H, Verspoor K, Roeder C, Hunter L: The structural and content aspects of abstracts versus bodies of full text journal articles are different. BMC Bioinformatics 2010.,11(492):
88. Truyens M, Van Eecke P: Legal aspects of text mining. Comput Law Secur Rev 2014,302(2):153-70.
89. Reichman J, Okediji R: When copyright law and science collide: empowering digitally integrated research methods on a global scale. Minn Law Rev 2012,96(4):1362-480.
90. Tsafnat G, Glasziou P, Choong M, Dunn A, Galgani F, Coiera E: Systematic review automation technologies. Syst Rev 2014,3(1):74. 10.1186/2046-4053-3-74
91. Kiritchenko S, de Bruijn B, Carini S, Martin J, Sim I: ExaCT: automatic extraction of clinical trial characteristics from journal publications. BMC Med Inform Decis Mak 2010,10(1):56. 10.1186/1472-6947-10-56
92. Marshall I, Kuiper J, Wallace B: Automating risk of bias assessment for clinical trials. In BCB '14 Proceedings of the 5th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics. New York, NY, USA: ACM; 2014:88-95.
93. Summerscales R: Automatic Summarization of Clinical Abstracts for Evidence-Based Medicine. Chicago, Illinois: Graduate College of the Illinois Institute of Technology; 2013.