Feature engineering and a proposed decision-support system for systematic reviewers of medical evidence

PLoS ONE

Volumn 9, Issue 1, 2014, Pages

  Bekhuis, Tanja ^a   Tseytlin, Eugene ^a   Mitchell, Kevin J ^a   Demner Fushman, Dina ^b  

^a UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE   (United States)

^b NATIONAL INSTITUTES OF HEALTH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALPHABETIC FEATURE; ALPHANUMERIC FEATURE; AUTOMATION; CLASSIFICATION; CLASSIFICATION ERROR; CLASSIFIER; COMPUTER PROGRAM; DECISION SUPPORT SYSTEM; ERROR; EVIDENCE BASED MEDICINE; INFORMATION RETRIEVAL; LINGUISTICS; MACHINE LEARNING; MATHEMATICAL PHENOMENA; MEASUREMENT PRECISION; MEDICAL INFORMATICS; QUALITY CONTROL; REVIEW; SYSTEMATIC REVIEW;

ARTIFICIAL INTELLIGENCE; BAYES THEOREM; DATABASES, BIBLIOGRAPHIC; DECISION SUPPORT TECHNIQUES; EVIDENCE-BASED MEDICINE; PUBLICATIONS; REVIEW LITERATURE AS TOPIC;

EID: 84899946454     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0086277     Document Type: Review

References (61)

1. Patient-Centered Outcomes Research Institute (PCORI) Mission and vision. Available: http://www.pcori.org/about-us/mission-and-vision/. Accessed 19 July 2013.
2. Luce BR, Drummond M, Jonsson B, Neumann PJ, Schwartz JS, et al. (2010) EBM, HTA, and CER: clearing the confusion. Milbank Q 88: 256-276.
3. Cochrane Collaboration. Available: http://www.cochrane.org/. Accessed 7 August 2011.
4. Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Centers. Available: http://www.ahrq.gov/clinic/epc/. Accessed 19 July 2013.
5. Bastian H, Glasziou P, Chalmers I (2010) Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med 7(9): e1000326.
6. The Cochrane Collaboration Cochrane Database of Systematic Reviews in numbers. Available: http://www.cochrane.org/cochrane-reviews/cochrane-database- systematic-reviews-numbers. Accessed 19 July 2013.
7. Higgins JPT, Green S (2008) Cochrane Handbook for Systematic Reviews of Interventions. Chichester, UK: Wiley.
8. Institute of Medicine of the National Academies (March 2011) Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC.
9. Patient-Centered Outcomes Research Institute (December 2012) PCORI Methodology Standards. Washington, DC. pp. 1-16.
10. Bekhuis T, Demner-Fushman D (2012) Screening nonrandomized studies for medical systematic reviews: a comparative study of classifiers. Artif Intell Med 55: 197-207.
11. Reeves B, Deeks J, Higgins J, Wells G (2008) Chapter 13: Including Non-randomized Studies. In: Higgins J, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions. Chichester (UK): Wiley.
12. Cohen AM, Ambert K, McDonagh M (2009) Cross-topic learning for work prioritization in systematic review creation and update. J Am Med Inform Assoc 16: 690-704.
13. Dalal SR, Shekelle PG, Hempel S, Newberry SJ, Motala A, et al. (2013) A pilot study using machine learning and domain knowledge to facilitate comparative effectiveness review updating. Med Decis Making 33: 343-355.
14. Cohen AM (2008) Optimizing feature representation for automated systematic review work prioritization. AMIA Annu Symp Proc.
15. Matwin S, Kouznetsov A, Inkpen D, Frunza O (2010) A new algorithm for reducing the workload of experts in performing systematic reviews. J Am Med Inform Assoc 17: 446-453.
16. Cohen AM, Hersh WR, Peterson K, Yen PY (2006) Reducing workload in systematic review preparation using automated citation classification. J Am Med Inform Assoc 13: 206-219. (Pubitemid 43332912)
17. Wallace BC, Trikalinos TA, Lau J, Brodley CE, Schmid CH (2010) Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinformatics 11: 55.
18. Frunza O, Inkpen D, Matwin S, Klement W, O'Blenis P (2011) Exploiting the systematic review protocol for classification of medical abstracts. Artif Intell Med 51: 17-25.
19. Cohen AM, Adams CE, Davis JM, Yu C, Yu PS, et al. (2010) Evidence-based Medicine, the Essential Role of Systematic Reviews, and the Need for Automated Text Mining Tools. IHI'10. Arlington, Virginia, USA: ACM (978-1-4503-0030-8/10/ 11). pp. 376-380.
20. Yaffe J, Montgomery P, Hopewell S, Shepard LD (2012) Empty reviews: a description and consideration of Cochrane systematic reviews with no included studies. PLoS ONE 7: e36626.
21. Bekhuis T, Stein CD (2013) Feasibility scans could improve prioritization and selection of systematic review topics. 21st Cochrane Colloquium. Quebec, Canada. Available: http://colloquium.cochrane.org/abstracts/feasibility-scans- could-improve-prioritization-and-selection-systematic-review-topics. Accessed 19 November 2013.
22. Abba K, Deeks JJ, Olliaro PL, Naing CM, Jackson SM, et al. (2011) Rapid diagnostic tests for diagnosing uncomplicated P. falciparum malaria in endemic countries. Cochrane Database of Systematic Reviews Issue 7. Art. No.: CD008122. doi:10.1002/14651858.CD008122.pub2.
23. Leeflang MM, Debets-Ossenkopp YJ, Visser CE, Scholten RJPM, Hooft L, et al. (2008) Galactomannan detection for invasive aspergillosis in immunocompromized patients. Cochrane Database of Systematic Reviews Issue 4. Art. No.: CD007394. doi:10.1002/14651858.CD007394.
24. Bekhuis T, Thyvalikakath TP, Oliver R (2009) Interventions for treating ameloblastomas of the jaws [protocol]. Cochrane Database of Systematic Reviews doi:101002/14651858CD003975pub2. Chichester, UK: John Wiley & Sons, Ltd.
25. Oremus M, Zeidler J, Ensom MHH, Matsuda-Abedini M, Balion C, et al. (Feb 2008) Utility of monitoring mycophenolic acid in solid organ transplant patients. Evidence Reports/Technology Assessments, No 164 AHRQ Publication No 08-E006. Rockville, MD: US Agency for Healthcare Research and Quality. Available: http://www.ncbi.nlm.nih.gov/books/NBK38475/. Accessed 24 July 2013.
26. Jefferson T, Di Pietrantonj C, Al-Ansary LA, Ferroni E, Thorning S, et al. (2010) Vaccines for preventing influenza in the elderly [updated review]. Cochrane Database of Systematic Reviews Issue 2 doi:101002/14651858CD004876pub3. Chichester, UK: Wiley.
27. National Center for Biotechnology Information PubMed.gov: US National Library of Medicine, National Institutes of Health. Available: http://www.ncbi.nlm.nih.gov/pubmed/. Accessed 27 May 2013
28. EMBASE: Biomedical Answers. Available: http://www.embase.com/. Accessed 27 May 2013.
29. Porter MF (1980) An algorithm for suffix stripping. Program 14: 130-137.
30. US National Library of Medicine, National Institutes of Health Medical Subject Headings: MeSH. Available: http://www.nlm.nih.gov/mesh/. Accessed 29 May 2013.
31. Elsevier BV Indexing/Emtree. Available: http://www.elsevier.com/online- tools/embase/emtree. Accessed 29 May 2013.
32. Huang X, Lin J, Demner-Fushman D (2006) Evaluation of PICO as a knowledge representation for clinical questions. AMIA Annu Symp Proc: 359-363.
33. Boudin F, Nie JY, Bartlett JC, Grad R, Pluye P, et al. (2010) Combining classifiers for robust PICO element detection. BMC medical informatics and decision making 10: 29.
34. US National Library of Medicine (2009) UMLS Reference Manual. Bethesda, MD: US National Library of Medicine. Available: http://www.ncbi.nlm.nih.gov/ books/NBK9676/. Accessed 24 July 2013.
35. Zou Q, Chu WW, Morioka C, Leazer GH, Kangarloo H (2003) IndexFinder: a method of extracting key concepts from clinical texts for indexing. AMIA Annu Symp Proc: 763-767.
36. Bekhuis T, Demner-Fushman D, Crowley RS (2013) Comparative effectiveness research designs: an analysis of terms and coverage in Medical Subject Headings (MeSH) and Emtree. J Med Libr Assoc 101: 92-100.
37. EDDA Group: Bekhuis T, Tseytlin E, Mitchell K (2013) EDDA study design terminology. National Center for Biomedical Ontology (NCBO) BioPortal Available: http://bioportalbioontologyorg/ontologies/3247 Accessed 24 July 2013.
38. Blei DM, Ng AY, Jordan MI (2003) Latent Dirichlet allocation. The Journal of Machine Learning Research 3: 993-1022.
39. Hofmann T (2001) Unsupervised learning by probabilistic latent semantic analysis. Machine Learning 42: 177-196.
40. Steyvers M, Griffiths T (2007) Probabilistic topic models. Handbook of Latent Semantic Analysis. pp. 424-440.
41. Griffiths TL, Steyvers M (2004) Finding scientific topics. Proceedings of the National Academy of Sciences of the United States of America 101 Suppl 1: 5228-5235. (Pubitemid 38469131)
42. McCallum AK MALLET: A Machine Learning for Language Toolkit. Available: http://mallet.cs.umass.edu. Accessed 17 July 2013.
43. Manning CD, Raghavan P, Schutze H (2008) Introduction to Information Retrieval. New York: Cambridge University Press.
44. Mierswa I, Wurst M, Klinkenberg R, Scholz M, Euler T (2006) YALE (now RapidMiner): rapid prototyping for complex data mining tasks. Proc ACM SIGKDD Int Conf on Knowl Discov Data Mining
45. RapidMiner [data mining suite]. Available: http://rapid-i.com/ Accessed 26 May 2013.
46. EDDA Group: Bekhuis T, Mitchell K, Tseytlin E (2013) EDDA extension for binominal text classification (topic models and regex) [open source code]. Available: http://marketplace.rapid-i.com/UpdateServer/faces/product-details. xhtml?productId=rmx-edda. Acessed 24 July 2013
47. Rennie J, Shih L, Teevan J, Karger D (2003) Tackling the poor assumptions of naive Bayes text classifiers. Proceedings of the Twentieth International Conference on Machine Learning (ICML). Washington, DC
48. IBM SPSS Statistics [computer program], version 20. Armonk, NY.
49. Bekhuis T, Demner-Fushman D (2010) Towards automating the initial screening phase of a systematic review. Stud Health Technol Inform 160: 146-150.
50. Wilczynski NL, Morgan D, Haynes RB (2005) An overview of the design and methods for retrieving high-quality studies for clinical care. BMC Med Inform Decis Mak 5: 20.
51. Wilczynski NL, McKibbon KA, Walter SD, Garg AX, Haynes RB (2013) MEDLINE clinical queries are robust when searching in recent publishing years. J Am Med Inform Assoc 20: 363-368.
52. McKibbon KA, Lokker C, Handler SM, Dolovich LR, Holbrook AM, et al. (2012) The effectiveness of integrated health information technologies across the phases of medication management: a systematic review of randomized controlled trials. J Am Med Inform Assoc 19: 22-30.
53. Kilicoglu H, Demner-Fushman D, Rindflesch TC, Wilczynski NL, Haynes RB (2009) Towards automatic recognition of scientifically rigorous clinical research evidence. J Am Med Inform Assoc 16: 25-31.
54. Keerthi SS, Duan KB, Shevade SK, Poo AN (2005) A Fast Dual Algorithm for Kernel Logistic Regression. Machine Learning 61: 151-165. (Pubitemid 43049366)
55. Zeng QT, Redd D, Rindflesch T, Nebeker J (2012) Synonym, topic model and predicate-based query expansion for retrieving clinical documents. AMIA Annu Symp Proc 2012: 1050-1059.
56. Jimeno-Yepes A, Wilkowski B, Mork JG, Van Lenten E, Fushman DD, et al. (2011) A bottom-up approach to MEDLINE indexing recommendations. AMIA Annu Symp Proc: American Medical Informatics Association. pp. 1583.
57. National Institutes of Health (April 2012) NLM Administrative Supplements for Informationist Services in NIH-funded Research Projects. Available: http://grants.nih.gov/grants/guide/pa-files/PA-12-158.html. Accessed 24 July 2013.
58. Chandler J, Churchill R, Higgins J, Lasserson T, Tovey D (December 2012) Methodological standards for the conduct of Cochrane Intervention Reviews (version 2.2). Cochrane Library.
59. Del Fiol G, Haug PJ (2008) Infobuttons and classification models: a method for the automatic selection of on-line information resources to fulfill clinicians' information needs. J Biomed Inform 41: 655-666.
60. Klinkenberg R, Joachims T (2000) Detecting concept drift with support vector machines. Proceedings of the Seventeenth International Conference on Machine Learning (ICML): Morgan Kaufmann. pp. 487-494.
61. Wilczynski NL, McKibbon KA, Haynes RB (2011) Search filter precision can be improved by NOTing out irrelevant content. AMIA Annu Symp Proc 2011: 1506-1513.