Big data and HPC convergence: The cutting edge and outlook

Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST

Volumn 224, Issue , 2018, Pages 11-26

  Usman, Sardar ^a   Mehmood, Rashid ^a   Katib, Iyad ^a  

^a KING ABDULAZIZ UNIVERSITY   (Saudi Arabia)

Author keywords

Big data; Cognitive computing; Design patterns; Hadoop; HPC; HPDA; IoT; Smart cities

Indexed keywords

DATA INTEGRATION; SMART CITY;

COGNITIVE COMPUTING; DESIGN PATTERNS; DIFFERENT ARCHITECTURES; HADOOP; HPDA; PERFORMANCE SENSITIVITY; SUBOPTIMAL SOLUTION; TECHNICAL LEVELS;

BIG DATA;

EID: 85051115890     PISSN: 18678211     EISSN: None     Source Type: Book Series    
DOI: 10.1007/978-3-319-94180-6_4     Document Type: Conference Paper

References (32)

1. Singh, K., Kaur, R.: Hadoop: addressing challenges of big data. In: 2014 IEEE International Advance Computing Conference (IACC), pp. 686–689. IEEE (2014)
2. Charl, S.: IBM-HPC and HPDA for the Cognitive Journey with OpenPOWER. https://www-03.ibm.com/systems/power/solutions/bigdata-analytics/smartpaper/high-value-insights.html
3. Keable, C.: The convergence of High Performance Computing and Big Data – Ascent. https://ascent.atos.net/convergence-high-performance-computing-big-data/
4. Joseph, E., Sorensen, B.: IDC Update on How Big Data Is Redefining High Performance Computing. https://www.tacc.utexas.edu/documents/1084364/1136739/IDC+HPDA+Briefing+ slides+10.21.2014_2.pdf
5. Geist, A., Lucas, R.: Whitepaper on the Major Computer Science Challenges at Exascale (2009)
6. Krishnan, S., Tatineni, M., Baru, C.: myHadoop-Hadoop-on-Demand on Traditional HPC Resources (2011)
7. Xuan, P., Denton, J., Ge, R., Srimani, P.K., Luo, F.: Big data analytics on traditional HPC infrastructure using two-level storage (2015)
8. Is Hadoop the New HPC.http://www.admin-magazine.com/HPC/Articles/Is-Hadoop-the-New-HPC
9. Katal, A., Wazid, M., Goudar, R.H.: Big data: issues, challenges, tools and good practices. In: 2013 Sixth International Conference on Contemporary Computing (IC3), pp. 404–409. IEEE (2013)
10. Hess, K.: Hadoop vs. Spark: The New Age of Big Data. http://www.datamation.com/data-center/hadoop-vs.-spark-the-new-age-of-big-data.html
11. Muhammad, J.: Is Apache Spark going to replace Hadoop? http://aptuz.com/blog/is-apache-spark-going-to-replace-hadoop/
12. OLCF Staff Writer: OLCF Group to Offer Spark On-Demand Data Analysis. https://www. olcf.ornl.gov/2016/03/29/olcf-group-to-offer-spark-on-demand-data-analysis/
13. Islam, N.S., Lu, X., Wasi-ur-Rahman, M., Shankar, D., Panda, D.K.: Triple-H: a hybrid approach to accelerate HDFS on HPC clusters with heterogeneous storage architecture. In: 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 101–110. IEEE (2015)
14. Ranger, C., Raghuraman, R., Penmetsa, A., Bradski, G., Kozyrakis, C.: Evaluating MapReduce for multi-core and multiprocessor systems. In: 2007 IEEE 13th International Symposium on High Performance Computer Architecture, pp. 13–24. IEEE (2007)
15. Tiwari, N., Sarkar, S., Bellur, U., Indrawan, M.: An empirical study of Hadoop’s energy efficiency on a HPC cluster. Procedia Comput. Sci. 29, 62–72 (2014)
16. Woodie, A.: Does InfiniBand Have a Future on Hadoop? http://www.datanami.com/2015/08/04/does-infiniband-have-a-future-on-hadoop/
17. Veiga, J., Exp, R.R., Taboada, G.L., Touri, J.: Analysis and Evaluation of Big Data Computing Solutions in an HPC Environment (2015)
18. Wang, Y., et al.: Assessing the performance impact of high-speed interconnects on MapReduce. In: Rabl, T., Poess, M., Baru, C., Jacobsen, H.-A. (eds.) WBDB-2012. LNCS, vol. 8163, pp. 148–163. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-53974-9_13
19. Islam, N.S., Lu, X., Wasi-ur-Rahman, M., Panda, D.K.: Can parallel replication benefit Hadoop distributed file system for high performance interconnects? In: 2013 IEEE 21st Annual Symposium on High-Performance Interconnects, pp. 75–78. IEEE (2013)
20. Moore, J., Chase, J., Ranganathan, P., Sharma, R.: Making scheduling cool: temperature-aware workload placement in data centers (2005)
21. Reed, D.A., Dongarra, J.: Exascale computing and big data. Commun. ACM 58, 56–68 (2015)
22. Rajovic, N., Puzovic, N., Vilanova, L., Villavieja, C., Ramirez, A.: The low-power architecture approach towards exascale computing. In: Proceedings of the Second Workshop on Scalable Algorithms for Large-Scale Systems-ScalA 2011, p. 1. ACM Press, New York (2011)
23. Cappello, F.: Fault tolerance in petascale/exascale systems: current knowledge, challenges and research opportunities. Int. J. High Perform. Comput. Appl. 23, 212–226 (2009)
24. Gutierrez, D.: The Convergence of Big Data and HPC – insideBIGDATA. https://insidebigdata.com/2016/10/25/the-convergence-of-big-data-and-hpc/
25. High Performance Data Analytics (HPDA) Market-Forecast 2022. https://www. marketresearchfuture.com/reports/high-performance-data-analytics-hpda-market
26. Willard, C.G., Snell, A., Segervall, L., Feldman, M.: Top Six Predictions for HPC in 2015 (2015)
27. Egham: Gartner Says 8.4 Billion Connected “Things”; Will Be in Use in 2017, Up 31 Percent From 2016. http://www.gartner.com/newsroom/id/3598917
28. El Baz, D.: IoT and the need for high performance computing. In: 2014 International Conference on Identification, Information and Knowledge in the Internet of Things, pp. 1–6. IEEE (2014)
29. Conway, S.: High Performance Data Analysis (HPDA): HPC-Big Data Convergence-insideHPC (2017)
30. Keutzer, K., Tim, M.: Our Pattern Language_Our Pattern Language (2016). Keutzer—EECS UC Berkeley, Tim—Intel. file:///Users/abdulmanan/Desktop/Our Pattern Language_Our Pattern Language.htm
31. Bodkin, R., Bodkin, R.: Big Data Patterns, pp. 1–23 (2017)
32. Mysore, D., Khupat, S., Jain, S.: Big data architecture and patterns, Part 1: Introduction to big data classification and architecture. https://www.ibm.com/developerworks/library/bd-archpatterns1/index.html