Big data analytics in the cloud: Spark on Hadoop vs MPI/OpenMP on Beowulf

Procedia Computer Science

Volumn 53, Issue 1, 2015, Pages 121-130

  Reyes Ortiz, Jorge L ^a   Oneto, Luca ^a   Anguita, Davide ^a  

^a UNIVERSITY OF GENOA   (Italy)

Author keywords

Beowulf; Big data; Cloud; Hadoop; MPI; OpenMP; Parallel computing; Spark; Supervised learning

Indexed keywords

ALGORITHMS; APPLICATION PROGRAMMING INTERFACES (API); ARTIFICIAL INTELLIGENCE; CLOUDS; CLUSTER COMPUTING; DISTRIBUTED COMPUTER SYSTEMS; ELECTRIC SPARKS; INFORMATION MANAGEMENT; ITERATIVE METHODS; LEARNING ALGORITHMS; LEARNING SYSTEMS; PARALLEL PROCESSING SYSTEMS; SUPERVISED LEARNING;

BEOWULF; CONSISTENT PERFORMANCE; DISTRIBUTED COMPUTING FRAMEWORKS; HADOOP; MANAGEMENT INFRASTRUCTURE; MPI; OPENMP; SUPERVISED MACHINE LEARNING;

BIG DATA;

EID: 84939199851     PISSN: None     EISSN: 18770509     Source Type: Conference Proceeding    
DOI: 10.1016/j.procs.2015.07.286     Document Type: Conference Paper

References (48)

1. A. Agarwal, O. Chapelle, M. Dudík, and J. Langford. A reliable effective terascale linear learning system. The Journal of Machine Learning Research, 15(1):1111-1133, 2014.
2. Divyakant Agrawal, Sudipto Das, and Amr El Abbadi. Big data and cloud computing: current state and future opportunities. In International Conference on Extending Database Technology, pages 530-533, 2011.
3. D. W. Aha. Lazy learning. Kluwer academic publishers, 1997.
4. D. Anguita, A. Ghio, L. Oneto, and S. Ridella. In-sample and out-of-sample model selection and error estimation for support vector machines. IEEE Transactions on Neural Networks and Learning Systems, 23(9):1390-1406, 2012.
5. P. Baldi, P. Sadowski, and D. Whiteson. Searching for exotic particles in high-energy physics with deep learning. Nature communications, 5, 2014.
6. A. Basumallik, S. J. Min, and R. Eigenmann. Programming distributed memory sytems using openmp. In IEEE International Parallel and Distributed Processing Symposium, pages 1-8, 2007.
7. A. Beygelzimer, S. Kakade, and J. Langford. Cover trees for nearest neighbor. In ACM International conference on Machine learning, pages 97-104, 2006.
8. C. M. Bishop. Neural networks for pattern recognition. Clarendon press Oxford, 1995.
9. O. Bousquet and A. Elisseeff. Stability and generalization. The Journal of Machine Learning Research, 2:499-526, 2002.
10. L. J. Cao, S. S. Keerthi, C. J. Ong, J. Q. Zhang, U. Periyathamby, X. J. Fu, and H. P. Lee. Parallel sequential minimal optimization for the training of support vector machines. IEEE Transactions on Neural Networks, 17(4):1039-1049, 2006.
11. F. Cappello and D. Etiemble. Mpi versus mpi+ openmp on the ibm sp for the nas benchmarks. In ACM/IEEE Conference on Supercomputing, pages 12-12, 2000.
12. A. G. Carlyle, S. L. Harrell, and P. M. Smith. Cost-effective hpc: The community or the cloud? In IEEE International Conference on Cloud Computing Technology and Science, pages 169-176, 2010.
13. R. Caruana, S. Lawrence, and G. Lee. Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping. In Advances in Neural Information Processing Systems, pages 402-410, 2001.
14. B. Chapman, G. Jost, and R. Van Der Pas. Using OpenMP: portable shared memory parallel programming. MIT press, 2008.
15. T. Cover and P. Hart. Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13(1):21-27, 1967.
16. L. Dagum and R. Menon. Openmp: an industry standard api for shared-memory programming. IEEE Computational Science & Engineering, 5(1):46-55, 1998.
17. A. Fanfarillo, T. Burnus, V. Cardellini, S. Filippone, D. Nagle, and D. Rouson. Opencoarrays: open-source transport layers supporting coarray fortran compilers. In International Conference on Partitioned Global Address Space Programming Models, pages 4-14, 2014.
18. M. Fernández-Delgado, E. Cernadas, S. Barro, and D. Amorim. Do we need hundreds of classifiers to solve real world classification problems? The Journal of Machine Learning Research, 15(1):3133-3181, 2014.
19. H. Furuta, T. Kameda, Y. Fukuda, and D. M. Frangopol. Life-cycle cost analysis for infrastructure systems: Life cycle cost vs. safety level vs. service life. Life-cycle performance of deteriorating structures: Assessment, design and management, pages 19-25, 2004.
20. V. Garcia, E. Debreuve, and M. Barlaud. Fast k nearest neighbor search using gpu. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pages 1-6, 2008.
21. Google. Google cloud platform - google compute engine. https://cloud.google.com, 2015.
22. W. Gropp, E. Lusk, N. Doss, and A. Skjellum. A high-performance, portable implementation of the mpi message passing interface standard. Parallel computing, 22(6):789-828, 1996.
23. I. Hendrickx and A. Van Den Bosch. Hybrid algorithms with instance-based classification. In Machine Learning: European Conference on Machine Learning, pages 158-169, 2005.
24. Intel. Intel parallel studio xe 2015 sp2. https://software.intel.com/en-us/intel-parallel-studio-xe, 2015.
25. Holden Karau, Andy Konwinski, Patrick Wendell, and Matei Zaharia. Learning Spark. O'Reilly Media, 2015.
26. M. Lichman. UCI machine learning repository, 2013.
27. E. Lusk and A. Chan. Early experiments with the openmp/mpi hybrid programming model. In OpenMP in a New Era of Parallelism, pages 36-47, 2008.
28. S. Mills, S. Lucas, L. Irakliotis, M. Rappa, T. Carlson, and B. Perlowitz. Demystifying big data: a practical guide to transforming the business of government. In Technical report. http://www.ibm.com/software/data/demystifying-big-data, pages 1-100, 2012.
29. T. M. Mitchell. Machine learning. WCB. McGraw-Hill Boston, 1997.
30. R. W. Numrich and J. Reid. Co-array fortran for parallel programming. In ACM Sigplan Fortran Forum, pages 1-31, 1998.
31. K. Olukotun. Beyond parallel programming with domain specific languages. In Symposium on Principles and practice of parallel programming, pages 179-180, 2014.
32. R. Rabenseifner, G. Hager, and G. Jost. Hybrid mpi/openmp parallel programming on clusters of multi-core smp nodes. In Euromicro International Conference on Parallel, Distributed and Network-based Processing, pages 427-436, 2009.
33. K. Sato, A. Moody, K. Mohror, T. Gamblin, B. R. De Supinski, N. Maruyama, and S. Matsuoka. Fmi: Fault tolerant messaging interface for fast and transparent recovery. In IEEE International Parallel and Distributed Processing Symposium, pages 1225-1234, 2014.
34. D. Schnitzer and A. Flexer. Choosing the metric in high-dimensional spaces based on hub analysis. In European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, pages 1-6, 2014.
35. S. Shalev-Shwartz, Y. Singer, N. Srebro, and A. Cotter. Pegasos: Primal estimated sub-gradient solver for svm. Mathematical programming, 127(1):3-30, 2011.
36. J. Shawe-Taylor and N. Cristianini. Kernel methods for pattern analysis. Cambridge university press, 2004.
37. J. Shawe-Taylor and S. Sun. A review of optimization methodologies in support vector machines. Neurocomputing, 74(17):3609-3618, 2011.
38. S. Sur, M. J. Koop, and D. K. Panda. High-performance and scalable mpi over infiniband with reduced memory usage: an in-depth performance analysis. In ACM/IEEE conference on Supercomputing, page 105, 2006.
39. V. N. Vapnik. Statistical learning theory. Wiley-Interscience, 1998.
40. K. Q. Weinberger and L. K. Saul. Distance metric learning for large margin nearest neighbor classification. The Journal of Machine Learning Research, 10:207-244, 2009.
41. Tom White. Hadoop: The definitive guide. "O'Reilly Media, Inc.", 2012.
42. Daniel Whiteson. Higgs data set. https://archive.ics.uci.edu/ml/datasets/HIGGS, 2014.
43. X. Wu, X. Zhu, G. Q. Wu, and W. Ding. Data mining with big data. IEEE Transactions on Knowledge and Data Engineering, 26(1):97-107, 2014.
44. Reynold S Xin, Josh Rosen, Matei Zaharia, Michael J Franklin, Scott Shenker, and Ion Stoica. Shark: Sql and rich analytics at scale. In ACM SIGMOD International Conference on Management of data, pages 13-24, 2013.
45. Y. You, S. L. Song, H. Fu, A. Marquez, M. M. Dehnavi, K. Barker, K. W. Cameron, A. P. Randles, and G. Yang. Mic-svm: Designing a highly efficient support vector machine for advanced modern multi-core and many-core architectures. In IEEE International Parallel and Distributed Processing Symposium, pages 809-818, 2014.
46. Matei Zaharia, Mosharaf Chowdhury, Tathagata Das, Ankur Dave, Justin Ma, Murphy McCauley, Michael J Franklin, Scott Shenker, and Ion Stoica. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. In USENIX Conference on Networked Systems Design and Implementation, pages 2-2, 2012.
47. Matei Zaharia, Mosharaf Chowdhury, Michael J Franklin, Scott Shenker, and Ion Stoica. Spark: cluster computing with working sets. In USENIX conference on Hot topics in cloud computing, pages 10-10, 2010.
48. Yiteng Zhai, Y. Ong, and I. Tsang. The emerging big dimensionality. IEEE Computational Intelligence Magazine, 9(3):14-26, 2014.