Applications of the MapReduce programming framework to clinical big data analysis: Current landscape and future trends

BioData Mining

Volumn 7, Issue 1, 2014, Pages

  Mohammed, Emad A ^a   Far, Behrouz H ^a   Naugler, Christopher ^b  

^a UNIVERSITY OF CALGARY   (Canada)

^b CALGARY LABORATORY SERVICES   (Canada)

Author keywords

Big data; Bioinformatics; Clinical big data analysis; Clinical data analysis; Distributed programming; Hadoop; MapReduce

Indexed keywords

BIOINFORMATICS; BIOMETRY; COMPUTER ANALYSIS; DATA ANALYSIS; DATA BASE; DATA EXTRACTION; DATA MINING; DATA SYNTHESIS; GENOTYPE; IMAGE ANALYSIS; INFORMATION STORAGE; MEDICAL INFORMATICS; PRIORITY JOURNAL; REVIEW; SOFTWARE;

EID: 84988962477     PISSN: None     EISSN: 17560381     Source Type: Journal    
DOI: 10.1186/1756-0381-7-22     Document Type: Review

References (93)

1. Shuman S: Structure, mechanism, and evolution of the mRNA capping apparatus. Prog Nucleic Acid Res Mol Biol 2000, 66:1-40.
2. Rajaraman A, Ullman JD: Mining of Massive Datasets. Cambridge-United Kingdom: Cambridge University Press; 2012.
3. Coulouris GF, Dollimore J, Kindberg T: Distributed Systems: Concepts and Design: Pearson Education; 2005.
4. de Oliveira Branco M: Distributed Data Management for Large Scale Applications. Southampton-United Kingdom: University of Southampton; 2009.
5. Raghupathi W, Raghupathi V: Big data analytics in healthcare: promise and potential. Health Inform Sci Syst 2014, 2(1):3.
6. Bell DE, Raiffa H, Tversky A: Descriptive, normative, and prescriptive interactions in decision making. Decis Mak 1988, 1:9-32.
7. Foster I, Kesselman C: The Grid 2: Blueprint for a new Computing Infrastructure. Houston-USA: Elsevier; 2003.
8. Owens JD, Houston M, Luebke D, Green S, Stone JE, Phillips JC: GPU computing. Proc IEEE 2008, 96(5):879-899.
9. Satish N, Harris M, Garland M: Designing efficient sorting algorithms for manycore GPUs. In Parallel & Distributed Processing, 2009 IPDPS 2009 IEEE International Symposium on: 2009, IEEE; 2009:1-10.
10. He B, Fang W, Luo Q, Govindaraju NK, Wang T: Mars: a MapReduce framework on graphics processors. In Proceedings of the 17th International Conference on Parallel Architectures and Compilation Techniques: 2008, ACM; 2008:260-269.
11. Dean J, Ghemawat S: MapReduce: simplified data processing on large clusters. Commun ACM 2008, 51(1):107-113.
12. Peyton Jones SL: The Implementation of Functional Programming Languages (Prentice-Hall International Series in Computer Science). New Jersey-USA: Prentice-Hall, Inc; 1987.
13. Bryant RE: Data-intensive supercomputing: The case for DISC. Pittsburgh, PA-USA: School of Computer Science, Carnegie Mellon University; 2007:1-20.
14. White T: Hadoop: The Definitive Guide. Sebastopol-USA: " O'Reilly Media, Inc."; 2012.
15. Shvachko K, Kuang H, Radia S, Chansler R: The hadoop distributed file system. In Mass Storage Systems and Technologies (MSST), 2010 IEEE 26th Symposium on: 2010, IEEE; 2010:1-10.
16. The Apache Software Foundation. [http://apache.org/]
17. Olson M: Hadoop: Scalable, flexible data storage and analysis. IQT Quart 2010, 1(3):14-18.
18. Xiaojing J: Google Cloud Computing Platform Technology Architecture and the Impact of Its Cost. In 2010 Second WRI World Congress on Software Engineering: 2010; 2010:17-20.
19. Thusoo A, Sarma JS, Jain N, Shao Z, Chakka P, Anthony S, Liu H, Wyckoff P, Murthy R: Hive: a warehousing solution over a map-reduce framework. Proc VLDB Endowment 2009, 2(2):1626-1629.
20. Olston C, Reed B, Srivastava U, Kumar R, Tomkins A: Pig latin: a not-so-foreign language for data processing. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data: 2008, ACM; 2008:1099-1110.
21. The Platform for Big Data and the Leading Solution for Apache Hadoop in the Enterprise-Cloudera. [http://www.cloudera.com/content/cloudera/en/home.html]
22. DataStax. [http://www.datastax.com/]
23. Hortonworks. [http://hortonworks.com/]
24. MAPR. [http://www.mapr.com/products/m3]
25. Top 14 Hadoop Technology Companies. [http://www.technavio.com/blog/top-14-hadoop-technology-companies]
26. Taylor RC: An overview of the Hadoop/MapReduce/HBase framework and its current applications in bioinformatics. BMC Bioinformatics 2010, 11(Suppl 12):S1.
27. Dai L, Gao X, Guo Y, Xiao J, Zhang Z: Bioinformatics clouds for big data manipulation. Biol Direct 2012, 7(1):43.
28. Microsoft Excel 2013: Spreadsheet software. [http://office.microsoft.com/en-ca/excel/]
29. Jonas M, Solangasenathirajan S, Hett D: Patient Identification, A Review of the Use of Biometrics in the ICU. In Annual Update in Intensive Care and Emergency Medicine 2014. New York-USA: Springer; 2014:679-688.
30. Wang W, Haerian K, Salmasian H, Harpaz R, Chase H, Friedman C: A drug-adverse event extraction algorithm to support pharmacovigilance knowledge mining from PubMed citations. In AMIA Annual Symposium Proceedings: 2011. Bethesda, Maryland-USA: American Medical Informatics Association; 2011:1464.
31. Aphinyanaphongs Y, Fu LD, Aliferis CF: Identifying unproven cancer treatments on the health web: addressing accuracy, generalizability and scalability. Stud Health Technol Inform 2012, 192:667-671.
32. Yaramakala S, Margaritis D: Speculative Markov blanket discovery for optimal feature selection. In Data Mining, Fifth IEEE International Conference on: 2005, IEEE; 2005:4.
33. Horiguchi H, Yasunaga H, Hashimoto H, Ohe K: A user-friendly tool to transform large scale administrative data into wide table format using a mapreduce program with a pig latin based script. BMC Med Inform Decis Mak 2012, 12(1):151.
34. Kohlwey E, Sussman A, Trost J, Maurer A: Leveraging the cloud for big data biometrics: Meeting the performance requirements of the next generation biometric systems. In Services (SERVICES), 2011 IEEE World Congress on: 2011, IEEE; 2011:597-601.
35. Raghava N: Iris recognition on hadoop: A biometrics system implementation on cloud computing. In Cloud Computing and Intelligence Systems (CCIS), 2011 IEEE International Conference on: 2011, IEEE; 2011:482-485.
36. Omri F, Hamila R, Foufou S, Jarraya M: Cloud-Ready Biometric System for Mobile Security Access. In Networked Digital Technologies. New York-USA: Springer; 2012:192-200.
37. Chen W-P, Hung C-L, Tsai S-JJ, Lin Y-L: Novel and efficient tag SNPs selection algorithms. Biomed Mater Eng 2014, 24(1):1383-1389.
38. Zhang K, Sun F, Waterman MS, Chen T: Dynamic programming algorithms for haplotype block partitioning: applications to human chromosome 21 haplotype data. In Proceedings of the Seventh Annual International Conference on Research in Computational Molecular Biology: 2003, ACM; 2003:332-340.
39. Nguyen AV, Wynden R, Sun Y: HBase, MapReduce, and Integrated Data Visualization for Processing Clinical Signal Data. In AAAI Spring Symposium: Computational Physiology: 2011; 2011.
40. Nordberg H, Bhatia K, Wang K, Wang Z: BioPig: a Hadoop-based analytic toolkit for large-scale sequence data. Bioinformatics 2013, 29(23):3014-3019.
41. Cloud Computing at NERSC. [http://www.nersc.gov/research-and-development/cloud-computing/]
42. AWS | Amazon Elastic Compute Cloud (EC2)-Scalable Cloud Hosting. [http://aws.amazon.com/ec2/]
43. Chang Y-J, Chen C-C, Ho J-M, Chen C-L: De Novo Assembly of High-Throughput Sequencing Data with Cloud Computing and New Operations on String Graphs. In Cloud Computing (CLOUD), 2012 IEEE 5th International Conference on: 2012, IEEE; 2012:155-161.
44. McKenna A, Hanna M, Banks E, Sivachenko A, Cibulskis K, Kernytsky A, Garimella K, Altshuler D, Gabriel S, Daly M: The Genome Analysis Toolkit: a MapReduce framework for analyzing next-generation DNA sequencing data. Genome Res 2010, 20(9):1297-1303.
45. MacLean B, Eng JK, Beavis RC, McIntosh M: General framework for developing and evaluating database scoring algorithms using the TANDEM search engine. Bioinformatics 2006, 22(22):2830-2832.
46. Lin Y-L: Implementation of a parallel protein structure alignment service on cloud. Int J Genomics 2013, 2013:1-8.
47. Huang H, Tata S, Prill RJ: BlueSNP: R package for highly scalable genome-wide association studies using Hadoop clusters. Bioinformatics 2013, 29(1):135-136.
48. Xu B, Gao J, Li C: An efficient algorithm for DNA fragment assembly in MapReduce. Biochem Biophys Res Commun 2012, 426(3):395-398.
49. Bean DR: Recursive Euler and Hamilton paths. Proc Am Math Soc 1976, 55(2):385-394.
50. Schatz MC: CloudBurst: highly sensitive read mapping with MapReduce. Bioinformatics 2009, 25(11):1363-1369.
51. Gropp W, Lusk E, Skjellum A: Using MPI: Portable Parallel Programming With the Message-Passing Interface, Volume 1. Cambridge, Massachusetts-USA: MIT press; 1999.
52. Isard M, Budiu M, Yu Y, Birrell A, Fetterly D: Dryad: distributed data-parallel programs from sequential building blocks. ACM SIGOPS Oper Syst Rev 2007, 41(3):59-72.
53. Qiu X, Ekanayake J, Beason S, Gunarathne T, Fox G, Barga R, Gannon D: Cloud technologies for bioinformatics applications. In Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers: 2009, ACM; 2009:6.
54. Gaggero M, Leo S, Manca S, Santoni F, Schiaratura O, Zanetti G, CRS E, Ricerche S: Parallelizing bioinformatics applications with MapReduce. In Cloud Computing and Its Applications; 2008.
55. Matsunaga A, Tsugawa M, Fortes J: Cloudblast: Combining mapreduce and virtualization on distributed resources for bioinformatics applications. In eScience, 2008 eScience'08 IEEE Fourth International Conference on: 2008, IEEE; 2008:222-229.
56. Tatusova TA, Madden TL: BLAST 2 Sequences, a new tool for comparing protein and nucleotide sequences. FEMS Microbiol Lett 1999, 174(2):247-250.
57. Darling A, Carey L, Feng W-c: The design, implementation, and evaluation of mpiBLAST. Proc Cluster World 2003, 2003:1-14.
58. Sadasivam GS, Baktavatchalam G: A novel approach to multiple sequence alignment using hadoop data grids. In Proceedings of the 2010 Workshop on Massive Data Analytics on the Cloud: 2010, ACM; 2010:2.
59. Schönherr S, Forer L, Weißensteiner H, Kronenberg F, Specht G, Kloss-Brandstätter A: Cloudgene: A graphical execution platform for MapReduce programs on private and public clouds. BMC Bioinformatics 2012, 13(1):200.
60. Lewis S, Csordas A, Killcoyne S, Hermjakob H, Hoopmann MR, Moritz RL, Deutsch EW, Boyle J: Hydra: a scalable proteomic search engine which utilizes the Hadoop distributed computing framework. BMC Bioinformatics 2012, 13(1):324.
61. Díaz-Uriarte R, De Andres SA: Gene selection and classification of microarray data using random forest. BMC Bioinformatics 2006, 7(1):3.
62. Wang Y, Goh W, Wong L, Montana G: Random forests on Hadoop for genome-wide association studies of multivariate neuroimaging phenotypes. BMC Bioinformatics 2013, 14(16):1-15.
63. Almeida JS, Grüneberg A, Maass W, Vinga S: Fractal MapReduce decomposition of sequence alignment. Algorithms Mol Biol 2012, 7(1):12.
64. Colosimo ME, Peterson MW, Mardis SA, Hirschman L: Nephele: genotyping via complete composition vectors and MapReduce. Source Code Biol Med 2011, 6:13.
65. Gao L, Qi J: Whole genome molecular phylogeny of large dsDNA viruses using composition vector method. BMC Evol Biol 2007, 7(1):41.
66. Lee W-P, Hsiao Y-T, Hwang W-C: Designing a parallel evolutionary algorithm for inferring gene networks on the cloud computing environment. BMC Syst Biol 2014, 8(1):5.
67. Juang C-F: A hybrid of genetic algorithm and particle swarm optimization for recurrent network design. Syst Man Cybern B Cybern IEEE Trans on 2004, 34(2):997-1006.
68. Zhang B, Yehdego DT, Johnson KL, Leung M-Y, Taufer M: Enhancement of accuracy and efficiency for RNA secondary structure prediction by sequence segmentation and MapReduce. BMC Struct Biol 2013, 13(Suppl 1):S3.
69. Zhao S, Prenger K, Smith L, Messina T, Fan H, Jaeger E, Stephens S: Rainbow: a tool for large-scale whole-genome sequencing data analysis using cloud computing. BMC Genomics 2013, 14(1):425.
70. Gurtowski J, Schatz MC, Langmead B: Genotyping in the cloud with crossbow. Curr Protoc Bioinformatics 2012, 15.13:11-15.
71. Reid JG, Carroll A, Veeraraghavan N, Dahdouli M, Sundquist A, English A, Bainbridge M, White S, Salerno W, Buhay C: Launching genomics into the cloud: deployment of Mercury, a next generation sequence analysis pipeline. BMC Bioinformatics 2014, 15(1):30.
72. Wu Z, Huang NE: Ensemble empirical mode decomposition: a noise-assisted data analysis method. Adv Adapt Data Anal 2009, 1(01):1-41.
73. Wang L, Chen D, Ranjan R, Khan SU, KolOdziej J, Wang J: Parallel Processing of Massive EEG Data with MapReduce. In ICPADS: 2012; 2012:164-171.
74. Wang F, Lee R, Liu Q, Aji A, Zhang X, Saltz J: Hadoop-gis: A high performance query system for analytical medical imaging with mapreduce. In Atlanta-USA: Technical report, Emory University; 2011:1-13.
75. Markonis D, Schaer R, Eggel I, Müller H, Depeursinge A: Using MapReduce for Large-Scale Medical Image Analysis. In HISB: 2012; 2012:1.
76. Meng B, Pratx G, Xing L: Ultrafast and scalable cone-beam CT reconstruction using MapReduce in a cloud computing environment. Med Phys 2011, 38(12):6603-6609.
77. Feldkamp L, Davis L, Kress J: Practical cone-beam algorithm. JOSA A 1984, 1(6):612-619.
78. Kaplan RS, Porter ME: How to solve the cost crisis in health care. Harv Bus Rev 2011, 89(9):46-52.
79. Musen MA, Middleton B, Greenes RA: Clinical decision-support systems. In Biomedical Informatics. New York-USA: Springer; 2014:643-674.
80. Devaraj S, Ow TT, Kohli R: Examining the impact of information technology and patient flow on healthcare performance: A Theory of Swift and Even Flow (TSEF) perspective. J Oper Manage 2013, 31(4):181-192.
81. Friedman AB: Preparing for responsible sharing of clinical trial data. N Engl J Med 2014, 370(5):484-484.
82. Mazurek M: Applying NoSQL Databases for Operationalizing Clinical Data Mining Models. In Beyond Databases, Architectures, and Structures. New York-USA: Springer; 2014:527-536.
83. Chawla NV, Davis DA: Bringing big data to personalized healthcare: A patient-centered framework. J Gen Intern Med 2013, 28(3):660-665.
84. Cusack CM, Hripcsak G, Bloomrosen M, Rosenbloom ST, Weaver CA, Wright A, Vawdrey DK, Walker J, Mamykina L: The future state of clinical data capture and documentation: a report from AMIA's 2011 Policy Meeting. J Am Med Inform Assoc 2013, 20(1):134-140.
85. Brodie MJ, Schachter SC, Kwan PKL: Fast Facts: Epilepsy. Albuquerque, New Mexico-USA: Health Press; 2012.
86. Fabene PF, Bramanti P, Constantin G: The emerging role for chemokines in epilepsy. J Neuroimmunol 2010, 224(1):22-27.
87. Shepherd GM, Mirsky JS, Healy MD, Singer MS, Skoufos E, Hines MS, Nadkarni PM, Miller PL: The Human Brain Project: neuroinformatics tools for integrating, searching and modeling multidisciplinary neuroscience data. Trends Neurosci 1998, 21(11):460-468.
88. Purves D: Body and Brain: A Trophic Theory of Neural Connections. Cambridge, Massachusetts-USA: Harvard University Press; 1990.
89. Hämäläinen M, Hari R, Ilmoniemi RJ, Knuutila J, Lounasmaa OV: Magnetoencephalography - theory, instrumentation, and applications to noninvasive studies of the working human brain. Rev Mod Phys 1993, 65(2):413.
90. Braak H, Braak E: Neuropathological stageing of Alzheimer-related changes. Acta Neuropathol 1991, 82(4):239-259.
91. Herculano-Houzel S: The human brain in numbers: a linearly scaled-up primate brain. Front Hum Neurosci 2009, 3:1-11.
92. Kumar G, Taneja A, Majumdar T, Jacobs ER, Whittle J, Nanchal R: The association of lacking insurance with outcomes of severe sepsis: retrospective analysis of an administrative database∗. Crit Care Med 2014, 42(3):583-591.
93. Youssef AE: A framework for secure healthcare systems based on Big data analytics in mobile cloud computing environments. Int J Ambient Syst Appl 2014, 2(2):1-11.