MLBCD: A machine learning tool for big clinical data

Health Information Science and Systems

Volumn 3, Issue 1, 2015, Pages

  Luo, Gang ^a  

^a UNIVERSITY OF UTAH SCHOOL OF MEDICINE   (United States)

Author keywords

Automatic algorithm selection; Automatic hyper parameter value selection; Big clinical data; Entity attribute value; Machine learning; Pivot

Indexed keywords

ABSENTEEISM; ARTICLE; ASTHMA; BRONCHIOLITIS; DOCUMENTATION QUALITY; HUMAN; HYPERPARAMETER; INFORMATION PROCESSING; LEARNING ALGORITHM; MACHINE LEARNING; MEASUREMENT ACCURACY; MEDICAL RESEARCH; NACHINE LEARNING TOOL FOR BIG CLINICAL DATA; PHYSICAL PERFORMANCE; PREDICTION AND FORECASTING; PRIORITY JOURNAL; SCHOOL ATTENDANCE; STATISTICAL PARAMETERS; TRAINING;

EID: 85019317965     PISSN: None     EISSN: 20472501     Source Type: Journal    
DOI: 10.1186/s13755-015-0011-0     Document Type: Article

References (138)

1. Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating. New York: Springer; 2009.
2. Kuhn M, Johnson K. Applied predictive modeling. New York: Springer; 2013.
3. Axelrod RC, Vogel D. Predictive modeling in health plans. Dis Manag Health Outcomes. 2003;11(12):779-87.
4. Asadi H, Dowling R, Yan B, Mitchell P. Machine learning for outcome prediction of acute ischemic stroke post intra-arterial therapy. PLoS One. 2014;9(2):e88225.
5. Witten IH, Frank E, Hall MA. Data mining: practical machine learning tools and techniques. 3rd ed. Burlington: Morgan Kaufmann; 2011.
6. Jovic A, Brkic K, Bogunovic N. An overview of free software tools for general data mining. In: Proceedings of MIPRO; 2014. p. 1112-7.
7. Kraska T, Talwalkar A, Duchi JC, Griffith R, Franklin MJ, Jordan MI. MLbase: a distributed machine-learning system. CIDR: Proc; 2013.
8. Thornton C, Hutter F, Hoos HH, Leyton-Brown K. Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms. In: Proceedings of KDD; 2013. p. 847-55.
9. Petrak J. Fast subsampling performance estimates for classification algorithm selection. In: Proceedings in ECML Workshop on Meta-Learning: Building Automatic Advice Strategies for Model Selection and Method Combination; 2000. pp 3-14.
10. Snoek J, Larochelle H, Adams RP. Practical Bayesian optimization of machine learning algorithms. In: Proceedings of NIPS; 2012. p. 2960-8.
11. Nickson T, Osborne MA, Reece S, Roberts SJ. Automated machine learning on big data using stochastic algorithm tuning. 2014. Available at http://arxiv.org/abs/1407.7969.
12. Luo G. A review of automatic selection methods for machine learning algorithms and hyper-parameter values. 2015. Available at http://pages. cs.wisc.edu/~gangluo/automatic_selection_review.pdf.
13. Komer B, Bergstra J, Eliasmith C. Hyperopt-sklearn: automatic hyperparameter configuration for scikit-learn. In: Proceedings of SciPy; 2014. p. 33-9.
14. Sparks ER, Talwalkar A, Franklin MJ, Jordan MI, Kraska T. TuPAQ: an efficient planner for large-scale predictive analytic queries. 2015. Available at http://arxiv.org/abs/1502.00068.
15. Dinu V, Nadkarni PM. Guidelines for the effective use of entityattribute-value modeling for biomedical databases. I J Med Inform. 2007;76(11-12):769-79.
16. Cerner's electronic medical record homepage. 2015. https:// www.cerner.com/solutions/Hospitals_and_Health_Systems/ Electronic_Medical_Record/.
17. McDonald CJ, Blevins L, Tierney WM, Martin DK. The Regenstrief medical records. MD Comput. 1988;5(5):34-47.
18. Warner HR, Olmsted CM, Rutherford BD. HELP-a program for medical decision-making. Comput Biomed Res. 1972;5(1):65-74.
19. Stead WW, Hammond WE, Straube MJ. A chartless record-Is it adequate? In: Proceedings of Annual Symposium on Computer Appliction Medical Care; 1982. p. 89-94.
20. Friedman C, Hripcsak G, Johnson SB, Cimino JJ, Clayton PD. A generalized relational schema for an integrated clinical patient database. In: Proceedings of Annual Symposium Computer Appliction in Medical Care; 1990. p. 335-9.
21. Oracle Clinical homepage. 2015. http://www.oracle.com/us/products/ applications/health-sciences/e-clinical/clinical/index.html.
22. Oracle Health Sciences Clintrial homepage. 2015. http://www.oracle. com/us/industries/life-sciences/health-sciences-clintrial-363570.html.
23. Brandt CA, Nadkarni P, Marenco L, Karras BT, Lu C, Schacter L, et al. Reengineering a database for clinical trials management: lessons for system architects. Control Clin Trials. 2000;21(5):440-61.
24. CohortExplorer. 2015. https://metacpan.org/pod/CohortExplorer.
25. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)-a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-81.
26. Eccher C, Piras EM, Stenico M. TreC-a REST-based regional PHR. Stud Health Technol Inform. 2011;169:108-12.
27. Nadkarni PM. Metadata-driven software systems in biomedicine: designing systems that can adapt to changing knowledge. New York: Springer; 2011.
28. Lyman JA, Scully K, Harrison JH Jr. The development of health care data warehouses to support data mining. Clin Lab Med. 2008;28(1):55-71.
29. Observational Medical Outcomes Partnership (OMOP) Common Data Model homepage. 2015. http://omop.org/CDM.
30. The National Patient-Centered Clinical Research Network (PCORnet) Common Data Model homepage. 2015. http://www.pcornet.org/ resource-center/pcornet-common-data-model/.
31. Informatics for Integrating Biology and the Bedside (i2b2) Design Document Data Repository (CRC) Cell. 2015. https://www.i2b2.org/software/ files/PDF/current/CRC_Design.pdf.
32. Einbinder JS, Scully KW, Pates RD, Schubart JR, Reynolds RE. Case study: a data warehouse for an academic medical center. J Healthc Inf Manag. 2001;15(2):165-75.
33. Luo G, Frey LJ. Efficient execution methods of pivoting for bulk extraction of Entity-Attribute-Value-modeled data. IEEE J Biomed Health Inform. 2015. doi:10.1109/JBHI.2015.2392553.
34. Cunningham C, Graefe G, Galindo-Legaria CA. PIVOT and UNPIVOT: optimization and execution strategies in an RDBMS. In: Proceedings of VLDB; 2004. p. 998-1009.
35. Dinu V, Nadkarni PM, Brandt C. Pivoting approaches for bulk extraction of entity-attribute-value data. Comput Methods Programs Biomed. 2006;82(1):38-43.
36. Pyle D. Data preparation for data mining. San Francisco: Morgan Kaufmann; 1999.
37. Dean J, Ghemawat S. MapReduce: Simplified data processing on large clusters. In: Proceedings of OSDI; 2004. p. 137-50.
38. White T. Hadoop: the definitive guide. 4th ed. Sebastopol: O'Reilly Media; 2015.
39. Zaharia M, Chowdhury M, Franklin MJ, Shenker S, Stoica I. Spark: cluster computing with working sets. In: Proceedings of HotCloud; 2010.
40. Chang F, Dean J, Ghemawat S, Hsieh WC, Wallach DA, Burrows M, et al. Bigtable: a distributed storage system for structured data. In: Proceedings of OSDI; 2006. 205-18.
41. Thusoo A, Sen Sarma J, Jain N, Shao Z, Chakka P, Zhang N, et al. Hive-a petabyte scale data warehouse using Hadoop. In: Proceedings of ICDE; p. 996-1005.
42. Xin RS, Rosen J, Zaharia M, Franklin MJ, Shenker S, Stoica I. Shark: SQL and rich analytics at scale. In: Proceedings of SIGMOD Conference; p. 13-24.
43. Armbrust M, Xin R, Huai Y, Liu D, Bradley JK, Meng X, et al. Spark SQL: relational data processing in Spark. In: Proceedings of SIGMOD; 2015.
44. Sparks ER, Talwalkar A, Smith V, Kottalam J, Pan X, Gonzalez JE, et al. MLI: an API for distributed machine learning. In: Proceedings of ICDM; 2013. p. 1187-92.
45. MLlib homepage. 2015. https://spark.apache.org/mllib/.
46. SQLeo homepage. 2015. http://sourceforge.net/projects/sqleo/.
47. Hosmer DW Jr, Lemeshow S, Sturdivant RX. Applied logistic regression. 3rd ed. Hoboken: Wiley; 2013.
48. Freitas AA. Comprehensible classification models: a position paper. SIGKDD Explor. 2013;15(1):1-10.
49. Feurer M, Springenberg T, Hutter F. Initializing Bayesian hyperparameter optimization via meta-learning. In: Proceedings of AAAI; 2015. p. 1128-35.
50. Provost FJ, Jensen D, Oates T. Efficient progressive sampling. In: Proceedings of KDD; 1999. p. 23-32.
51. Leite R, Brazdil P. Predicting relative performance of classifiers from samples. In: Proceedings of ICML; 2005. p. 497-503.
52. Fürnkranz J, Petrak J. An evaluation of landmarking variants. In: Proceedings of ECML/PKDD Workshop on Integrating Aspects of Data Mining, Decision Support and Meta-Learning; 2001. p. 57-68.
53. John GH, Langley P. Static versus dynamic sampling for data mining. In: Proceedings of KDD; 1996. p. 367-70.
54. Soares C, Petrak J, Brazdil P. Sampling-based relative landmarks: systematically test-driving algorithms before choosing. In: Proceedings of EPIA; p. 88-95.
55. Leite R, Brazdil P. Active testing strategy to predict the best classification algorithm via sampling and metalearning. In: Proceedings of ECAI; p. 309-14.
56. Gu B, Liu B, Hu F, Liu H. Efficiently determining the starting sample size for progressive sampling. In: Proceedings of ECML; 2001. p. 192-202.
57. Leite R, Brazdil P, Vanschoren J. Selecting classification algorithms with active testing. In: Proceedings of MLDM; 2012. p. 117-31.
58. Hutter F, Hoos HH, Leyton-Brown K. Sequential model-based optimization for general algorithm configuration. In: Proceedings of LION; 2011. p. 507-23.
59. Bergstra J, Bardenet R, Bengio Y, Kégl B. Algorithms for hyper-parameter optimization. In: Proceedings of NIPS; 2011. p. 2546-54.
60. Eggensperger K, Hutter F, Hoos HH, Leyton-Brown K. Efficient benchmarking of hyperparameter optimizers via surrogates. In: Proceedings of AAAI; 2015. p. 1114-20.
61. Bergstra J, Bengio Y. Random search for hyper-parameter optimization. J Mach Learn Res. 2012;13:281-305.
62. Smith-Miles K. Cross-disciplinary perspectives on meta-learning for algorithm selection. ACM Comput Surv. 2008;41(1).
63. Roemer M. Health Care Expenditures for the Five Most Common Children's Conditions, 2008: Estimates for U.S. Civilian Noninstitutionalized Children, Ages 0-17. MEPS Statistical Brief #349. Rockville, MD: AHRQ; 2011.
64. Malveaux FJ. The state of childhood asthma: introduction. Pediatrics. 2009;123(Suppl 3):S129-30.
65. Akinbami LJ, Moorman JE, Liu X. Asthma prevalence, health care use, and mortality: United States, 2005-2009. Natl Health Stat Report. 2011;32:1-14.
66. Akinbami LJ, Moorman JE, Bailey C, Zahran HS, King M, Johnson CA, et al. Trends in asthma prevalence, health care use, and mortality in the United States, 2001-2010. NCHS Data Brief. 2012;94:1-8.
67. Weissman JS, Gatsonis C, Epstein AM. Rates of avoidable hospitalization by insurance status in Massachusetts and Maryland. JAMA. 1992;268(17):2388-94.
68. Wang LY, Zhong Y, Wheeler L. Direct and indirect costs of asthma in school-age children. Prev Chronic Dis. 2005;2(1):A11.
69. Carroll KN, Wu P, Gebretsadik T, Griffin MR, Dupont WD, Mitchel EF, et al. The severity-dependent relationship of infant bronchiolitis on the risk and morbidity of early childhood asthma. J Allergy Clin Immunol. 2009;123(5):1055-61.
70. Zorc JJ, Hall CB. Bronchiolitis: recent evidence on diagnosis and management. Pediatrics. 2010;125(2):342-9.
71. Piippo-Savolainen E, Korppi M. Wheezy babies-wheezy adults? Review on long-term outcome until adulthood after early childhood wheezing. Acta Paediatr. 2008;97(1):5-11.
72. Perlstein PH, Kotagal UR, Bolling C, Steele R, Schoettker PJ, Atherton HD, et al. Evaluation of an evidence-based guideline for bronchiolitis. Pediatrics. 1999;104(6):1334-41.
73. Hyvärinen M, Piippo-Savolainen E, Korhonen K, Korppi M. Teenage asthma after severe infantile bronchiolitis or pneumonia. Acta Paediatr. 2005;94(10):1378-83.
74. Larouch V, Rivard G, Deschesnes F, Goulet R, Turcotte H, Boulet LP. Asthma and airway hyper-responsiveness in adults who required hospital admission for bronchiolitis in early childhood. Respir Med. 2000;94(3):288-94.
75. McConnochie KM, Roghmann KJ. Bronchiolitis as a possible cause of wheezing in childhood: new evidence. Pediatrics. 1984;74(1):1-10.
76. Piippo-Savolainen E, Remes S, Kannisto S, Korhonen K, Korppi M. Asthma and lung function 20 years after wheezing in infancy: results from a prospective follow-up study. Arch Pediatr Adolesc Med. 2004;158(11):1070-6.
77. Sigurs N, Bjarnason R, Sigurbergsson F, Kjellman B. Respiratory syncytial virus bronchiolitis in infancy is an important risk factor for asthma and allergy at age 7. Am J Respir Crit Care Med. 2000;161(5):1501-7.
78. Ruotsalainen M, Piippo-Savolainen E, Hyvärinen MK, Korppi M. Adulthood asthma after wheezing in infancy: a questionnaire study at years of age. Allergy. 2010;65(4):503-9.
79. James KM, Gebretsadik T, Escobar GJ, Wu P, Carroll KN, Li SX, et al. Risk of childhood asthma following infant bronchiolitis during the respiratory syncytial virus season. J Allergy Clin Immunol. 2013;132(1):227-9.
80. Luo G, Nkoy FL, Stone BL, Schmick D, Johnson MD. A systematic review of predictive models for asthma development in children. Available at http://pages.cs.wisc.edu/~gangluo/asthma_development.pdf.
81. Altman DG, Bland JM. Diagnostic tests 2: predictive values. BMJ. 1994;309(6947):102.
82. Nolte H, Nepper-Christensen S, Backer V. Unawareness and undertreatment of asthma and allergic rhinitis in a general population. Respir Med. 2006;100(2):354-62.
83. Yeatts K, Davis KJ, Sotir M, Herget C, Shy C. Who gets diagnosed with asthma? Frequent wheeze among adolescents with and without a diagnosis of asthma. Pediatrics. 2003;111(5 Pt 1):1046-54.
84. Speight AN, Lee DA, Hey EN. Underdiagnosis and undertreatment of asthma in childhood. Br Med J (Clin Res Ed). 1983;286(6373):1253-6.
85. Speight AN. Is childhood asthma being underdiagnosed and undertreated? Br Med J. 1978;2(6133):331-2.
86. Majak P, Bak-Walczak E, Stelmach I, Jerzyn'ska J, Krakowiak J, Stelmach W. An increasing trend of the delay in asthma diagnosis after the discontinuation of a population-based intervention. J Asthma. 2011;48(4):414-8.
87. Pedersen S. Preschool asthma-not so easy to diagnose. Prim Care Respir J. 2007;16(1):4-6.
88. Caudri D, Wijga A, Schipper CM, Hoekstra M, Postma DS, Koppelman GH, et al. Predicting the long-term prognosis of children with symptoms suggestive of asthma at preschool age. J Allergy Clin Immunol. 2009;124(5):903-10.
89. Humbert M. The right tools at the right time. Chest. 2006;130(1 Suppl):29S-40S.
90. Frey U, von Mutius E. The challenge of managing wheezing in infants. N Engl J Med. 2009;360(20):2130-3.
91. Holt PG, Sly PD. Prevention of allergic respiratory disease in infants: current aspects and future perspectives. Curr Opin Allergy Clin Immunol. 2007;7(6):547-55.
92. Mansbach JM, Camargo CA Jr. Respiratory viruses in bronchiolitis and their link to recurrent wheezing and asthma. Clin Lab Med. 2009;29(4):741-55.
93. Krueger RA, Casey MA. Focus groups: a practical guide for applied research. 5th ed. Thousand Oaks: SAGE Publications Inc.; 2014.
94. Patton MQ. Qualitative research and evaluation methods. 3rd ed. Thousand Oaks: SAGE Publications; 2001.
95. Sandelowski M. Sample size in qualitative research. Res Nurs Health. 1995;18(2):179-83.
96. Morae at usability testing software. 2015. http://www.techsmith.com/ morae.html, 2015.
97. McLafferty I. Focus group interviews as a data collection strategy. J Adv Nurs. 2004;48:187-94.
98. Twohig PL, Putnam W. Group interviews in primary care research: advancing the state of the art or ritualized research? Fam Pract. 2002;19:278-84.
99. Powell RA, Single HM. Focus groups. Int J Qual Health Care. 1996;8:499-504.
100. Sharts-Hopko NC. Focus group methodology: when and why? J Assoc Nurses AIDS Care. 2001;12:89-91.
101. Nielsen J. Usability engineering. Cambridge: Morgan Kaufmann; 1993.
102. Rubin J, Chisnell D. Handbook of usability testing: how to plan, design and conduct effective tests. 2nd ed. Indianapolis: John Wiley & Sons; 2008.
103. Nielsen J. How many test users in a usability study? 2012. http://www.nngroup.com/articles/how-many-test-users/.
104. Hasegawa K, Tsugawa Y, Brown DF, Mansbach JM, Camargo CA Jr. Temporal trends in emergency department visits for bronchiolitis in the United States, 2006 to 2010. Pediatr Infect Dis J. 2014;33(1):11-8.
105. Byington CL, Reynolds CC, Korgenski K, Sheng X, Valentine KJ, Nelson RE, et al. Costs and infant outcomes after implementation of a care process model for febrile infants. Pediatrics. 2012;130(1):e16-24.
106. Bradford W, Hurdle JF, LaSalle B, Facelli JC. Development of a HIPAAcompliant environment for translational research data and analytics. J Am Med Inform Assoc. 2014;21(1):185-9.
107. Schatz M, Cook EF, Joshua A, Petitti D. Risk factors for asthma hospitalizations in a managed care organization: development of a clinical prediction rule. Am J Manag Care. 2003;9(8):538-47.
108. Desai JR, Wu P, Nichols GA, Lieu TA, O'Connor PJ. Diabetes and asthma case identification, validation, and representativeness when using electronic health data to construct registries for comparative effectiveness and epidemiologic research. Med Care. 2012;50(Suppl):S30-5.
109. Wakefield DB, Cloutier MM. Modifications to HEDIS and CSTE algorithms improve case recognition of pediatric asthma. Pediatr Pulmonol. 2006;41(10):962-71.
110. The APCD (all-payer claims database) Council homepage. 2015. http:// www.apcdcouncil.org/.
111. Andersson M, Hedman L, Bjerg A, Forsberg B, Lundbäck B, Rönmark E. Remission and persistence of asthma followed from 7 to 19 years of age. Pediatrics. 2013;132(2):e435-42.
112. Savenije OE, Kerkhof M, Koppelman GH, Postma DS. Predicting who will have asthma at school age among preschool children. J Allergy Clin Immunol. 2012;130(2):325-31.
113. Yunginger JW, Reed CE, O'Connell EJ, Melton LJ 3rd, O'Fallon WM, Silverstein MD 3rd. A community-based study of the epidemiology of asthma. Incidence rates, 1964-1983. Am Rev Respir Dis. 1992;146(4):888-94.
114. Taussig LM, Wright AL, Holberg CJ, Halonen M, Morgan WJ, Martinez FD. Tucson children's respiratory study: 1980 to present. J Allergy Clin Immunol. 2003;111(4):661-75.
115. Matricardi PM, Illi S, Grüber C, Keil T, Nickel R, Wahn U, et al. Wheezing in childhood: incidence, longitudinal patterns and factors predicting persistence. Eur Respir J. 2008;32(3):585-92.
116. Duncan I. Healthcare risk adjustment and predictive modeling. Winsted: ACTEX Publications Inc.; 2011.
117. Luo G, Nkoy FL, Gesteland PH, Glasgow TS, Stone BL. A systematic review of predictive modeling for bronchiolitis. Int J Med Inform. 2014;83(10):691-714.
118. Matricardi PM, Illi S, Keil T, Wagner P, Wahn U, Lau S. Predicting persistence of wheezing: one algorithm does not fit all. Eur Respir J. 2010;35(3):701-3.
119. Korppi M. Asthma predictive factors in infants with bronchiolitis: asthma risk at 13-20 years of age. Eur Respir J. 2010;36(1):221-2.
120. Brooke J. SUS-a quick and dirty usability scale. 1996. http://hell.meiert. org/core/pdf/sus.pdf.
121. Tullis T, Albert W. Measuring the user experience: collecting, analyzing and presenting usability metrics. Morgan Kaufmann, 2008.
122. Sauro J. Measuring usability with the System Usability Scale (SUS). 2011. http://www.measuringusability.com/sus.php.
123. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Human Comp Interact. 2008;24(6):574-94.
124. Thomas DR. A general inductive approach for analyzing qualitative evaluation data. Am J Eval. 2006;27(2):237-46.
125. ATLAS qualitative analysis software. 2015. http://www.atlasti.com/index. html.
126. Schuirmann DJ. A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability. J Pharmacokinet Biopharm. 1987;15(6):657-80.
127. Google Prediction API homepage. 2015. https://cloud.google.com/ prediction/docs.
128. Amazon Machine Learning homepage. 2015. http://aws.amazon.com/ machine-learning/.
129. Chen G, Gully SM, Eden D. Validation of a new general self-efficacy scale. Organ Res Methods. 2001;4(1):62-83.
130. Pfahringer B, Bensusan H, Giraud-Carrier CG. Meta-learning by landmarking various learning algorithms. In: Proceedings of ICML; 2000. p.743-50.
131. Maron O, Moore AW. Hoeffding races: Accelerating model selection search for classification and function approximation. In: Proceedings of NIPS; 1993. p. 59-66.
132. Brazdil P, Soares C, da Costa JP. Ranking learning algorithms: using IBL and meta-learning on accuracy and time results. Mach Learn. 2003;50(3):251-77.
133. Bengio Y. Gradient-based optimization of hyperparameters. Neural Comput. 2000;12(8):1889-900.
134. Guo XC, Yang JH, Wu CG, Wang CY, Liang YC. A novel LS-SVMs hyperparameter selection based on particle swarm optimization. Neurocomputing. 2008;71(16-18):3211-5.
135. Adankon MM, Cheriet M. Model selection for the LS-SVM. Application to handwriting recognition. Pattern Recogn. 2009;42(12):3264-70.
136. Bardenet R, Brendel M, Kégl B, Sebag M. Collaborative hyperparameter tuning. In: Proceedings of ICML; 2013. p. 199-207.
137. Swersky K, Snoek J, Adams RP. Multi-task Bayesian optimization. In: Proceedings of NIPS; 2013. p. 2004-12.
138. Yogatama D, Mann G. Efficient transfer learning method for automatic hyperparameter tuning. In: Proceedings of AISTATS; 2014. p. 1077-85.