Automatic identification of high impact articles in PubMed to support clinical decision making

Journal of Biomedical Informatics

Volumn 73, Issue , 2017, Pages 95-103

  Bian, Jiantao ^a   Morid, Mohammad Amin ^b   Jonnalagadda, Siddhartha ^c   Luo, Gang ^d   Del Fiol, Guilherme ^a  

^a UNIVERSITY OF UTAH SCHOOL OF MEDICINE   (United States)

^b UNIVERSITY OF UTAH   (United States)

^c MICROSOFT   (United States)

^d UNIVERSITY OF WASHINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; AUTOMATION; EDUCATION; LEARNING SYSTEMS; METADATA; SOCIAL NETWORKING (ONLINE);

AUTOMATIC IDENTIFICATION; CLINICAL DECISION MAKING; CLINICAL DECISION SUPPORT; CLINICAL STUDY; EVIDENCE-BASED MEDICINE; PRIMARY SOURCES; RANDOMIZED CONTROLLED TRIAL; STATE OF THE ART;

LEARNING ALGORITHMS;

ACCURACY; ARTICLE; BAYESIAN LEARNING; BIBLIOMETRICS; CITATION ANALYSIS; CLASSIFICATION ALGORITHM; CLINICAL DECISION MAKING; CLINICAL DECISION SUPPORT SYSTEM; EVIDENCE BASED MEDICINE; HUMAN; INFORMATION RETRIEVAL; JOURNAL IMPACT FACTOR; LEARNING ALGORITHM; MEDICAL LITERATURE; MEDICAL ONTOLOGY; MEDLINE; METADATA; PATIENT CARE; PERFORMANCE; PRIORITY JOURNAL; SOCIAL MEDIA; STATISTICAL SIGNIFICANCE; ALGORITHM; MACHINE LEARNING;

ALGORITHMS; BIBLIOMETRICS; CLINICAL DECISION-MAKING; EVIDENCE-BASED MEDICINE; HUMANS; INFORMATION STORAGE AND RETRIEVAL; MACHINE LEARNING; MEDLINE; METADATA; PUBMED; SOCIAL MEDIA;

EID: 85026473430     PISSN: 15320464     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jbi.2017.07.015     Document Type: Article

References (70)

1. Covell, D.G., Uman, G.C., Manning, P.R., Information needs in office practice: are they being met?. Ann. Intern. Med. 103 (1985), 596–599 (accessed July 18, 2012).
2. Del Fiol, G., Workman, T.E., Gorman, P.N., Clinical questions raised by clinicians at the point of care: a systematic review. JAMA Int. Med. 174 (2014), 710–718, 10.1001/jamainternmed.2014.368.
3. L.L. Leape, D.W. Bates, D.J. Cullen, J. Cooper, H.J. Demonaco, T. Gallivan, R. Hallisey, J. Ives, N. Laird, G. Laffel, Systems Analysis of Adverse Drug Events. ADE Prevention Study Group, JAMA. 274 (1995) 35–43. http://www.ncbi.nlm.nih.gov/pubmed/7791256 (accessed May 13, 2015).
4. Maviglia, S.M., Yoon, C.S., Bates, D.W., Kuperman, G., KnowledgeLink: impact of context-sensitive information retrieval on clinicians’ information needs. J. Am. Med. Inform. Assoc. 13 (2006), 67–73, 10.1197/jamia.M1861.
5. Sackett, D.L., Rosenberg, W.M.C., Gray, J.A.M., Haynes, R.B., Richardson, W.S., Evidence based medicine: what it is and what it isn't. BMJ 312 (1996), 71–72, 10.1136/bmj.312.7023.71.
6. Hung, B.T., Long, N.P., Hung, L.P., Luan, N.T., Anh, N.H., Nghi, T.D., Van Hieu, M., Trang, N.T.H., Rafidinarivo, H.F., Anh, N.K., Hawkes, D., Huy, N.T., Hirayama, K., Research trends in evidence-based medicine: a joinpoint regression analysis of more than 50 years of publication data. PLoS One, 10, 2015, e0121054, 10.1371/journal.pone.0121054.
7. Hoogendam, A., Stalenhoef, A.F.H., Robbé, P.F. de V., Overbeke, A.J.P.M., Answers to questions posed during daily patient care are more likely to be answered by UpToDate than PubMed. J. Med. Internet Res., 10, 2008, e29, 10.2196/jmir.1012.
8. Ioannidis, J.P.A., Why most published research findings are false. PLoS Med., 2, 2005, e124, 10.1371/journal.pmed.0020124.
9. Cook, D.A., Sorensen, K.J., Hersh, W., Berger, R.A., Wilkinson, J.M., Features of effective medical knowledge resources to support point of care learning: a focus group study. PLoS One, 8, 2013, e80318, 10.1371/journal.pone.0080318.
10. Haynes, R.B., Wilczynski, N., McKibbon, K.A., Walker, C.J., Sinclair, J.C., Developing optimal search strategies for detecting clinically sound studies in MEDLINE. J. Am. Med. Inform. Assoc. 1 (1994), 447–458 (accessed November 28, 2013).
11. Kilicoglu, H., Demner-Fushman, D., Rindflesch, T.C., Wilczynski, N.L., Haynes, R.B., Towards automatic recognition of scientifically rigorous clinical research evidence. J. Am. Med. Inform. Assoc. 16 (2009), 25–31, 10.1197/jamia.M2996.
12. Wilczynski, N.L., McKibbon, K.A., Walter, S.D., Garg, A.X., Haynes, R.B., MEDLINE clinical queries are robust when searching in recent publishing years. J. Am. Med. Informatics Assoc. 20 (2013), 363–368, 10.1136/amiajnl-2012-001075.
13. Bernstam, E.V., Herskovic, J.R., Aphinyanaphongs, Y., Aliferis, C.F., Sriram, M.G., Hersh, W.R., Using citation data to improve retrieval from MEDLINE. J. Am. Med. Inform. Assoc. 13 (2006), 96–105, 10.1197/jamia.M1909.
14. Aphinyanaphongs, Y., Tsamardinos, I., Statnikov, A., Hardin, D., Aliferis, C.F., Text categorization models for high-quality article retrieval in internal medicine. J. Am. Med. Inform. Assoc. 12 (2005), 207–216, 10.1197/jamia.M1641.
15. Aphinyanaphongs, Y., Statnikov, A., Aliferis, C.F., A comparison of citation metrics to machine learning filters for the identification of high quality MEDLINE documents. J. Am. Med. Inform. Assoc. 13 (2006), 446–455, 10.1197/jamia.M2031.
16. Brin, S., Page, L., Brin, S., Page, L., The anatomy of a large-scale hypertextual Web search engine. Comput. Networks ISDN Syst. 30 (1998), 107–117, 10.1016/S0169-7552(98)00110-X.
17. Wilczynski, N.L., Morgan, D., Haynes, R.B., An overview of the design and methods for retrieving high-quality studies for clinical care. BMC Med. Inform. Decis. Mak., 5, 2005, 20, 10.1186/1472-6947-5-20.
18. Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., Altman, D., Preferred reporting items for systematic reviews and meta-analyses: the prisma statement. PLoS Med., 6, 2009, e1000097, 10.1371/journal.pmed.1000097.
19. J.P.T. Higgins, S. (Sally E. Green, Cochrane Collaboration., Wiley InterScience (Online service), Cochrane handbook for systematic reviews of interventions, Wiley-Blackwell, 2008. < https://books.google.com/books?hl = en&lr = &id = NKMg9sMM6GUC&oi = fnd&pg = PT13&dq = systematic + review + methodology + cochrane&ots = LIRAR-HGC6&sig = qAntasDRxidNqJG_KFz4wOHD3HU#v = onepage&q = systematic review methodology cochrane&f = false> (accessed July 20, 2017).
20. Singh, J.A., Furst, D.E., Bharat, A., Curtis, J.R., Kavanaugh, A.F., Kremer, J.M., Moreland, L.W., O'Dell, J., Winthrop, K.L., Beukelman, T., Bridges, S.L., Chatham, W.W., Paulus, H.E., Suarez-Almazor, M., Bombardier, C., Dougados, M., Khanna, D., King, C.M., Leong, A.L., Matteson, E.L., Schousboe, J.T., Moynihan, E., Kolba, K.S., Jain, A., Volkmann, E.R., Agrawal, H., Bae, S., Mudano, A.S., Patkar, N.M., Saag, K.G., 2012 update of the 2008 American College of Rheumatology recommendations for the use of disease-modifying antirheumatic drugs and biologic agents in the treatment of rheumatoid arthritis. Arthritis Care Res. (Hoboken) 64 (2012), 625–639, 10.1002/acr.21641.
21. D. Curran, A. Blain, K. Orio, C. Daughaday, D. Bray, L. Potter, D. Khachikian, C. Colluraburke, J. Quinn, M. Littner, D. Dodson, N. Radebaugh, G. Sees, T. Davenport, K. Robson, D. Hsu, C. Roach, R. Lee, W. Schriner, J. Liss, M. Tucker, J. Manning, H. Wojtczak, M. Morris, J. Jacknewitz-Woolard, J. Markusfeld, C. Cassidy, E. Degenhardt, A. Klar, O. Susskind, L. Lux, M. Viswanathan, K. Lohr, R. Fishman, J. Marko, VA/DoD Clinical Practice Guideline For Management of Asthma in Children and Adults, 2009, 127.
22. C. Cassidy, L. Carchedi, T. Craig, C. Engel, M. Gerrity, D. Knittel, L. Labbate, P. Lowry, J. McQuaid, G. Manos, A. Muller, E. McLaughlin, D. Oslin, M. Ramos, J. Ray, S. K.Trice, T. Semla, R. Wilson, J. Williams, A. Zeiss, O. Susskind, V. Coats, E. Erinoff, K. Schoelles, D. Snyder, M. D'Erasmo, R. Fishman, J. Marko, VA/DoD Clinical Practice Guideline For Management of Major Depressive Disorder (MDD), (2009) 203.
23. P. Almenoff, C. Cote, S. Doman, M. Habib, D. Doherty, D. Khachikian, M. Littner, K. Mahutte, P. McGlashan, S. Sethi, A. Sharafkhaneh, D. Carnahan, M. Kallish, C. Kang, A. Klar, J. Mitchell, M. Musket, M. Stephens, O. Susskind, V. Coats, E. Erinoff, D. Snyder, C. Turkelson, M.D'Erasmo, R. Fishman, J. Marko, VA/DoD Clinical Practice Guideline For Management of Outpatient COPD, 2007, 138.
24. M. Render, J. Sherner, K. Rice, T. Anekwe, A. Sharafkhaneh, J. Feinberg, J. Ellis, J. Sall, A. Fritz, M.J. Sapp, C. Hintz, R.M. Selvester, D. Khachikian, S. Shaw, C. Staropoli, M. Tennant, J. Wallinger, M.E. Rodgers, E. Degenhardt, R.M. Sutton, J. Sall, C. Goodman, F. Murphy, E. Beam, N. Stettler, S. Uhl, H. Kleiner, K. D'Anci, C. Jones, O. Akinyede, R. Rishar, A. Tsou, K. Laws, A. Ramanathan, VA/DoD Clinical Practice Guideline for the Management of Chronic Obstructive Pulmonary Disease, 2014, 94.
25. Brott, T.G., Halperin, J.L., Abbara, S., Bacharach, J.M., Barr, J.D., Bush, R.L., Cates, C.U., Creager, M.A., Fowler, S.B., Friday, G., Hertzberg, V.S., McIff, E.B., Moore, W.S., Panagos, P.D., Riles, T.S., Rosenwasser, R.H., Taylor, A.J., ASA/ACCF/AHA/AANN/AANS/ACR/ASNR/CNS/SAIP/SCAI/SIR/SNIS/SVM/SVS guideline on the management of patients with extracranial carotid and vertebral artery disease. J. Am. Coll. Cardiol. 57:2011 (2011), e16–e94, 10.1016/j.jacc.2010.11.006.
26. Fihn, S.D., Gardin, J.M., Abrams, J., Berra, K., Blankenship, J.C., Dallas, A.P., Douglas, P.S., Foody, J.M., Gerber, T.C., Hinderliter, A.L., King, S.B., Kligfield, P.D., Krumholz, H.M., Kwong, R.Y.K., Lim, M.J., Linderbaum, J.A., Mack, M.J., Munger, M.A., Prager, R.L., Sabik, J.F., Shaw, L.J., Sikkema, J.D., Smith, C.R., Smith, S.C., Spertus, J.A., Williams, S.V., ACCF/AHA/ACP/AATS/PCNA/SCAI/STS Guideline for the Diagnosis and Management of Patients With Stable Ischemic Heart Disease: Executive Summary. J. Am. Coll. Cardiol. 60:2012 (2012), 2564–2603, 10.1016/j.jacc.2012.07.012.
27. O'Gara, P.T., Kushner, F.G., Ascheim, D.D., Casey, D.E., Chung, M.K., de Lemos, J.A., Ettinger, S.M., Fang, J.C., Fesmire, F.M., Franklin, B.A., Granger, C.B., Krumholz, H.M., Linderbaum, J.A., Morrow, D.A., Newby, L.K., Ornato, J.P., Ou, N., Radford, M.J., Tamis-Holland, J.E., Tommaso, C.L., Tracy, C.M., Woo, Y.J., Zhao, D.X., Anderson, J.L., Jacobs, A.K., Halperin, J.L., Albert, N.M., Brindis, R.G., Creager, M.A., DeMets, D., Guyton, R.A., Hochman, J.S., Kovacs, R.J., Ohman, E.M., Stevenson, W.G., Yancy, C.W., 2013 ACCF/AHA guideline for the management of ST-elevation myocardial infarction: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. J. Am. Coll. Cardiol. 61 (2013), e78–e140, 10.1016/j.jacc.2012.11.019.
28. Yancy, C.W., Jessup, M., Bozkurt, B., Butler, J., Casey, D.E., Drazner, M.H., Fonarow, G.C., Geraci, S.A., Horwich, T., Januzzi, J.L., Johnson, M.R., Kasper, E.K., Levy, W.C., Masoudi, F.A., McBride, P.E., McMurray, J.J.V., Mitchell, J.E., Peterson, P.N., Riegel, B., Sam, F., Stevenson, L.W., Tang, W.H.W., Tsai, E.J., Wilkoff, B.L., 2013 ACCF/AHA guideline for the management of heart failure: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. J. Am. Coll. Cardiol. 62 (2013), e147–e239, 10.1016/j.jacc.2013.05.019.
29. Nishimura, R.A., Otto, C.M., Bonow, R.O., Carabello, B.A., Erwin, J.P., Guyton, R.A., O'Gara, P.T., Ruiz, C.E., Skubas, N.J., Sorajja, P., Sundt, T.M., Thomas, J.D., 2014 AHA/ACC guideline for the management of patients with valvular heart disease: a report of the American college of cardiology/American heart association task force on practice guidelines. J. Am. Coll. Cardiol. 63 (2014), e57–e185, 10.1016/j.jacc.2014.02.536.
30. January, C.T., Wann, L.S., Alpert, J.S., Calkins, H., Cleveland, J.C., Cigarroa, J.E., Conti, J.B., Ellinor, P.T., Ezekowitz, M.D., Field, M.E., Murray, K.T., Sacco, R.L., Stevenson, W.G., Tchou, P.J., Tracy, C.M., Yancy, C.W., 2014 AHA/ACC/HRS guideline for the management of patients with atrial fibrillation: a report of the American college of cardiology/American heart association task force on practice guidelines and the heart rhythm society. j. Am. Coll. Cardiol. 64 (2014), e1–e76, 10.1016/j.jacc.2014.03.022.
31. Demner-Fushman, D., Lin, J., Answering clinical questions with knowledge-based and statistical techniques. Comput. Linguist. 33 (2007), 63–103, 10.1162/coli.2007.33.1.63.
32. Garfield, E., The history and meaning of the journal impact factor. JAMA 295 (2006), 90–93, 10.1001/jama.295.1.90.
33. McVeigh, M.E., Mann, S.J., The journal impact factor denominator. JAMA., 302, 2009, 1107, 10.1001/jama.2009.1301.
34. Journal Citation Reports® (JCR®) by Thomson Reuters, (n.d.). https://jcr.incites.thomsonreuters.com/ (accessed April 4, 2017).
35. Lundh, A., Sismondo, S., Lexchin, J., Busuioc, O.A., Bero, L., Industry sponsorship and research outcome. Cochrane Database Syst. Rev., 12, 2012, MR000033, 10.1002/14651858.MR000033.pub2.
36. Sismondo, S., Pharmaceutical company funding and its consequences: a qualitative systematic review. Contemp. Clin. Trials. 29 (2008), 109–113, 10.1016/j.cct.2007.08.001.
37. Lexchin, J., Bero, L.A., Djulbegovic, B., Clark, O., Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 326 (2003), 1167–1170, 10.1136/bmj.326.7400.1167.
38. Schumock, G.T., Pickard, A.S., Comparative effectiveness research: relevance and applications to pharmacy. Am. J. Health. Syst. Pharm. 66 (2009), 1278–1286, 10.2146/ajhp090150.
39. Scopus Abstract Citations Count API, n.d. http://api.elsevier.com/documentation/AbstractCitationCountAPI.wadl (accessed April 4, 2017).
40. About Altmetric and the Altmetric score, n.d. https://help.altmetric.com/support/solutions/articles/6000059309-about-altmetric-and-the-altmetric-score (accessed April 4, 2017).
41. How is the Altmetric score calculated, n.d. https://help.altmetric.com/support/solutions/articles/6000060969-how-is-the-altmetric-score-calculated (accessed April 4, 2017).
42. Altmetric API, n.d. http://api.altmetric.com/ (accessed April 4, 2017).
43. MEDLINE Abridged Index Medicus (AIM or “Core Clinical”) Journal Titles, n.d. https://www.nlm.nih.gov/bsd/aim.html (accessed April 4, 2017).
44. McMaster Plus (Premium LiteratUre Service) journals, n.d. http://hiru.mcmaster.ca/hiru/journalslist.asp (accessed April 4, 2017).
45. Journal selection for index medicus®/Medline®, J. Can. Chiropr. Assoc. 40 (1996) 47. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2485146/ (accessed March 14, 2016).
46. Committed selection: abridged index medicus, N. Engl. J. Med. 282 (1970) 220–1. doi:10.1056/NEJM197001222820410.
47. Hemens, B.J., Haynes, R.B., McMaster Premium LiteratUre Service (PLUS) performed well for identifying new studies for updated Cochrane reviews. J. Clin. Epidemiol. 65 (2012), 62–72, 10.1016/j.jclinepi.2011.02.010.
48. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H., The WEKA data mining software. ACM SIGKDD Explor. Newsl., 11, 2009, 10, 10.1145/1656274.1656278.
49. Hall, M.A., Holmes, G., Benchmarking attribute selection techniques for discrete class data mining. IEEE Trans. Knowl. Data Eng. 15 (2003), 1437–1447, 10.1109/TKDE.2003.1245283.
50. Charles, L., Sheng, V., Cost-sensitive learning. Encycl. Mach. Learn., 2011, Springer, US, Boston, MA, 231–235, 10.1007/978-0-387-30164-8_181.
51. Soni, J., Ansari, U., Sharma, D., Soni, S., Predictive data mining for medical diagnosis: an overview of heart disease prediction. Int. J. Comput. Appl. 17 (2011), 43–48, 10.5120/2237-2860.
52. Seera, M., Lim, C.P., A hybrid intelligent system for medical data classification. Expert Syst. Appl. 41 (2014), 2239–2249, 10.1016/j.eswa.2013.09.022.
53. K. Canese, PubMed Relevance Sort, NLM Tech. Bull, 2013. < https://www.nlm.nih.gov/pubs/techbull/so13/so13_pm_relevance.html> (accessed July 20, 2017).
54. Hoogendam, A., Stalenhoef, A.F.H., Robbé, P.F. de V., Overbeke, A.J.P.M., Analysis of queries sent to PubMed at the point of care: observation of search behaviour in a medical teaching hospital. BMC Med. Inform. Decis. Mak., 8, 2008, 42, 10.1186/1472-6947-8-42.
55. Manning, C.D., Raghavan, P., Schütze, H., Introduction to Information Retrieval. 2008, Cambridge University Press.
56. Charles, D., Charles, G., Statistics in the pharmaceutical industry. Buncher, R., Tsay, J.-Y., (eds.) Stat. Pharm. Ind., 3rd ed., 2006, Chapman & Hall/CRC, New York, New York, USA, 421–452.
57. A. Dmitrienko, A. Tamhane, F. Bretz, Multiple Testing Problems in Pharmaceutical Statistics, 2009, New York, New York, USA.
58. International Conference on Harmonisation E9 Expert Working Group, ICH harmonised tripartite guideline: statistical principles for clinical trials, Stat Med. 18 (1999) 1905–1942.
59. Van Cutsem, E., Köhne, C.-H., Hitre, E., Zaluski, J., Chang Chien, C.-R., Makhson, A., D'Haens, G., Pintér, T., Lim, R., Bodoky, G., Roh, J.K., Folprecht, G., Ruff, P., Stroh, C., Tejpar, S., Schlichting, M., Nippgen, J., Rougier, P., Cetuximab and chemotherapy as initial treatment for metastatic colorectal cancer. N. Engl. J. Med. 360 (2009), 1408–1417, 10.1056/NEJMoa0805019.
60. Jansen, B.J., Spink, A., Saracevic, T., Real life, real users, and real needs: a study and analysis of user queries on the web. Inf. Process. Manage. 36 (2000), 207–227, 10.1016/S0306-4573(99)00056-4.
61. Ely, J.W., Osheroff, J.A., Chambliss, M.L., Ebell, M.H., Rosenbaum, M.E., Answering physicians’ clinical questions: obstacles and potential solutions. J. Am. Med. Inform. Assoc. 12 (2005), 217–224, 10.1197/jamia.M1608.
62. Ely, J.W., Osheroff, J.A., Ebell, M.H., Bergus, G.R., Levy, B.T., Chambliss, M.L., Evans, E.R., Analysis of questions asked by family doctors regarding patient care. BMJ 319 (1999), 358–361 (accessed March 14, 2017).
63. Margolis, J., Citation indexing and evaluation of scientific papers. Science 155 (1967), 1213–1219 (accessed March 5, 2017).
64. Bornmann, L., Daniel, H., What do citation counts measure? A review of studies on citing behavior. J. Doc. 64 (2008), 45–80, 10.1108/00220410810844150.
65. Barbic, D., Tubman, M., Lam, H., Barbic, S., An analysis of altmetrics in emergency medicine. Acad. Emerg. Med. 23 (2016), 251–268, 10.1111/acem.12898.
66. Rosenkrantz, A.B., Ayoola, A., Singh, K., Duszak, R., Alternative metrics (“altmetrics”) for assessing article impact in popular general radiology. J. Acad. Radiol., 2017, 10.1016/j.acra.2016.11.019.
67. Widmer, G., Kubat, M., Learning in the presence of concept drift and hidden contexts. Mach. Learn. 23 (1996), 69–101, 10.1007/BF00116900.
68. A. Tsymbal, The Problem of Concept Drift: Definitions and Related Work, 2004. citeulike-article-id:2350391.
69. Irwin, A.N., Rackham, D., Comparison of the time-to-indexing in PubMed between biomedical journals according to impact factor, discipline, and focus. Res. Soc. Adm. Pharm. 13 (2017), 389–393, 10.1016/j.sapharm.2016.04.006.
70. Mork, J., Aronson, A., Demner-Fushman, D., 12 years on – is the NLM medical text indexer still useful and relevant?. J. Biomed. Semantics., 8, 2017, 8, 10.1186/s13326-017-0113-5.