Parallel shortest path graph computations of United States road network data on apache spark

Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST

Volumn 224, Issue , 2018, Pages 323-336

  Arfat, Yasir ^a   Mehmood, Rashid ^a   Albeshri, Aiiad ^a  

^a KING ABDULAZIZ UNIVERSITY   (Saudi Arabia)

Author keywords

Apache GraphX; Apache hadoop; Apache spark; Big data; Graph analytics; Road networks; Shortest paths

Indexed keywords

COMPUTER SOFTWARE; GRAPH THEORY; INTERNET OF THINGS; MOTOR TRANSPORTATION; PROBLEM SOLVING; ROADS AND STREETS; SUPERCOMPUTERS;

APACHE GRAPHX; APACHE HADOOP; GRAPH ANALYTICS; ROAD NETWORK; SHORTEST PATH;

BIG DATA;

EID: 85051145190     PISSN: 18678211     EISSN: None     Source Type: Book Series    
DOI: 10.1007/978-3-319-94180-6_30     Document Type: Conference Paper

References (43)

1. Lu, Y., Cheng, J., Yan, D., Wu, H.: Large-scale distributed graph computing systems. Proc. VLDB Endow. 8, 281–292 (2014)
2. Sanfeliu, A., Alquézar, R., Andrade, J., Climent, J., Serratosa, F., Vergés, J.: Graph-based representations and techniques for image processing and image analysis. Pattern Recognit. 35, 639–650 (2002)
3. Ding, Y., Yan, S., Zhang, Y.B., Dai, W., Dong, L.: Predicting the attributes of social network users using a graph-based machine learning method. Comput. Commun. 73, 3–11 (2016)
4. Khan, A., Uddin, S., Srinivasan, U.: Adapting graph theory and social network measures on healthcare data. In: Proceedings of the Australasian Computer Science Week Multiconfer-ence on-ACSW 2016, pp. 1–7. ACM Press, New York (2016)
5. Mehmood, R., Graham, G.: Big data logistics: a health-care transport capacity sharing model. Procedia Comput. Sci. 64, 1107–1114 (2015)
6. Mehmood, R., Meriton, R., Graham, G., Hennelly, P., Kumar, M.: Exploring the influence of big data on city transport operations: a Markovian approach. Int. J. Oper. Prod. Manag. (2016). Forthcoming
7. Hendrickson, B., Kolda, T.G.: Graph partitioning models for parallel computing. Parallel Comput. 26, 1519–1534 (2000)
8. Mehmood, R., Crowcroft, J.: Parallel iterative solution method for large sparse linear equation systems. Comput. Lab. Univ. (2005)
9. Kwiatkowska, M., Parker, D., Zhang, Y., Mehmood, R.: Dual-processor parallelisation of symbolic probabilistic model checking. In: DeGroot, D., Harrison, P. (eds.) Proceedings-IEEE Computer Society’s Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, MASCOTS, pp. 123–130. IEEE, Volendam (2004)
10. Alazawi, Z., Abdljabar, M.B., Altowaijri, S., Vegni, A.M., Mehmood, R.: ICDMS: an intelligent cloud based disaster management system for vehicular networks. In: Vinel, A., Mehmood, R., Berbineau, M., Garcia, C.R., Huang, C.-M., Chilamkurti, N. (eds.) Nets4Cars/Nets4Trains 2012. LNCS, vol. 7266, pp. 40–56. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29667-3_4
11. Junghanns, M., Petermann, A., Neumann, M., Rahm, E.: Management and analysis of big graph data: current systems and open challenges. In: Zomaya, A., Sakr, S. (eds.) Handbook of Big Data Technologies, pp. 457–505. Springer International Publishing, Cham (2017). https://doi.org/10.1007/978-3-319-49340-4_14
12. Oh, S., Ha, J., Lee, K., Oh, S.: DegoViz: an interactive visualization tool for a differentially expressed genes heatmap and gene ontology graph. Appl. Sci. 7, 543 (2017)
13. Mehmood, R., Faisal, M.A., Altowaijri, S.: Future networked healthcare systems: a review and case study. In: Big Data: Concepts, Methodologies, Tools, and Applications, pp. 2429– 2457. IGI Global (2016)
14. Arfat, Y., Aqib, M., Mehmood, R., Albeshri, A., Katib, I., Albogami, N., Alzahrani, A.: Enabling smarter societies through mobile big data fogs and clouds. Procedia Comput. Sci. 109, 1128–1133 (2017)
15. GraphX | Apache Spark
16. Apache Spark: Apache Spark™-Lightning-Fast Cluster Computing
17. Welcome to Apache™ Hadoop®!-index.pdf. https://hadoop.apache.org/index.pdf
18. Kajdanowicz, T., Kazienko, P., Indyk, W.: Parallel processing of large graphs. Futur. Gener. Comput. Syst. 32, 324–337 (2014)
19. Liu, X., Zhou, Y., Guan, X., Sun, X.: A feasible graph partition framework for random walks implemented by parallel computing in big graph. In: Chinese Control Conference, CCC 2015, pp. 4986–4991, September 2015
20. Wang, Z., Chen, Q., Hou, B., Suo, B., Li, Z., Pan, W., Ives, Z.G.: Parallelizing maximal clique and k-plex enumeration over graph data. J. Parallel Distrib. Comput. 106, 79–91 (2017)
21. Braun, P., Cuzzocrea, A., Leung, C.K., Pazdor, A.G.M., Tran, K.: Knowledge Discovery from Social Graph Data. Procedia Comput. Sci. 96, 682–691 (2016)
22. Laboshin, L.U., Lukashin, A.A., Zaborovsky, V.S.: The big data approach to collecting and analyzing traffic data in large scale networks. Procedia Comput. Sci. 103, 536–542 (2017)
23. Liu, R., Li, X., Du, L., Zhi, S., Wei, M.: Parallel implementation of density peaks clustering algorithm based on spark. Procedia Comput. Sci. 107, 442–447 (2017)
24. Aridhi, S., Mephu Nguifo, E.: Big graph mining: frameworks and techniques. Big Data Res. 6, 1–10 (2016)
25. Drosou, A., Kalamaras, I., Papadopoulos, S., Tzovaras, D.: An enhanced Graph Analytics Platform (GAP) providing insight in Big Network Data. J. Innov. Digit. Ecosyst. 3, 83–97 (2016)
26. Zhao, Y., Yoshigoe, K., Xie, M., Zhou, S., Seker, R., Bian, J.: Evaluation and analysis of distributed graph-parallel processing frameworks. J. Cyber Secur. Mobil. 3, 289–316 (2014)
27. Mohan, A., Remya, R.: A review on large scale graph processing using big data based parallel programming models. Int. J. Intell. Syst. Appl. 9, 49–57 (2017)
28. Pollard, S., Norris, B.: A Comparison of Parallel Graph Processing Benchmarks (2017)
29. Suma, S., Mehmood, R., Albugami, N., Katib, I., Albeshri, A.: Enabling next generation logistics and planning for smarter societies. Procedia Comput. Sci. 109, 1122–1127 (2017)
30. Miller, J.A., Ramaswamy, L., Kochut, K.J., Fard, A.: Research directions for big data graph analytics. In: Proceedings of the 2015 IEEE International Congress on Big Data, BigData Congress 2015, pp. 785–794 (2015)
31. Chakaravarthy, V.T., Checconi, F., Petrini, F., Sabharwal, Y.: Scalable single source shortest path algorithms for massively parallel systems. In: Proceedings of the 28th International Parallel and Distributed Processing Symposium, IPDPS, pp. 889–901 (2014)
32. Xia, Y., Tanase, I.G., Nai, L., Tan, W., Liu, Y., Crawford, J., Lin, C.: Explore efficient data organization for large scale graph analytics and storage. In: Proceedings of the 2014 IEEE BigData Conference, pp. 942–951 (2014)
33. Zhang, B., Liu, X., Lang, B.: Fast Graph Similarity Search via Locality Sensitive Hashing. In: Ho, Y.-S., Sang, J., Ro, Y.M., Kim, J., Wu, F. (eds.) PCM 2015. LNCS, vol. 9314, pp. 623–633. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24075-6_60
34. Shao, Y., Cui, B., Ma, L.: PAGE: A partition aware engine for parallel graph computation. IEEE Trans. Knowl. Data Eng. 27, 518–530 (2015)
35. Chen, R., Yang, M., Weng, X., Choi, B., He, B., Li, X.: Improving large graph processing on partitioned graphs in the cloud. In: Proceedings of the Third ACM Symposium on Cloud Computing, SoCC 2012, pp. 1–13 (2012)
36. Zeng, Z., Wu, B., Wang, H.: A parallel graph partitioning algorithm to speed up the large-scale distributed graph mining. In: Proceedings of the 1st International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications-BigMine 2012, pp. 61–68 (2012)
37. Lee, K., Liu, L.: Efficient data partitioning model for heterogeneous graphs in the cloud. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, pp. 1–12 (2013)
38. Xu, N., Chen, L., Cui, B.: LogGP: a log-based dynamic graph partitioning method. Proc. VLDB Endow. 7, 1917–1928 (2014)
39. Yang, S., Yan, X., Zong, B., Khan, A.: Towards effective partition management for large graphs. In: Proceedings of the 2012 International Conference on Management Data-SIGMOD 2012, pp. 517–528 (2012)
40. Spark Data Locality Documentation. https://spark.apache.org/docs/latest/tuning.html#data-locality
41. GraphX Partitioning Scheme Documentation. https://spark.apache.org/docs/latest/graphx-programming-guide.html#optimized-representation
42. DIMACS Implementation Challenge. http://www.dis.uniroma1.it/challenge9/download. shtml
43. Gephi-The Open Graph Viz Platform. https://gephi.org/