How well do graph-processing platforms perform? An empirical performance evaluation and analysis

Proceedings of the International Parallel and Distributed Processing Symposium, IPDPS

Volumn , Issue , 2014, Pages 395-404

  Guo, Yong ^a   Biczak, Marcin ^a   Varbanescu, Ana Lucia ^b   Iosup, Alexandru ^a   Martella, Claudio ^c   Willke, Theodore L ^d  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

^b UNIVERSITY OF AMSTERDAM   (Netherlands)

^c VU UNIVERSITY AMSTERDAM   (Netherlands)

^d INTEL CORPORATION   (United States)

Author keywords

benchmark; graph processing; performance evaluation

Indexed keywords

DISTRIBUTED PARAMETER NETWORKS;

COMPREHENSIVE COMPARISONS; EMPIRICAL METHOD; EMPIRICAL PERFORMANCE; EVALUATION AND ANALYSIS; GRAPH PROCESSING; PERFORMANCE EVALUATION; RESOURCE UTILIZATIONS;

BENCHMARKING;

EID: 84906684323     PISSN: 15302075     EISSN: 23321237     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPS.2014.49     Document Type: Conference Paper

References (44)

1. J. Leskovec, J. Kleinberg, and C. Faloutsos, "Graphs over Time: Densification Laws, Shrinking Diameters and Possible Explanations," in SIGKDD, 2005.
2. Y. Guo, S. Shen, O. Visser, and A. Iosup, "An Analysis of Online Match- Based Games," in MMVE, 2012.
3. D. Dominguez-Sal, P. Urbón-Bayes, A. Giménez-Vanó, S. Gómez-Villamor, N. Martínez-Bazán, and J. Larriba-Pey, "Survey of Graph Database Performance on the HPC Scalable Graph Analysis Benchmark," Web-Age Information Management, pp. 37-48, 2010.
4. G. Malewicz, M. H. Austern, A. J. Bik, J. C. Dehnert, I. Horn, N. Leiser, and G. Czajkowski, "Pregel: A System for Large-scale Graph Processing," in SIGMOD, 2010, pp. 135-146.
5. A. Lugowski, D. M. Alber, A. Buluc,, J. R. Gilbert, S. Reinhardt, Y. Teng, and A.Waranis, "A Flexible Open-Source Toolbox for Scalable Complex Graph Analysis," in SDM, 2012.
6. Ü. V. Catalyürek, M. Deveci, K. Kaya, and B. Ucar, "Multithreaded Clustering for Multi-level Hypergraph Partitioning," in IPDPS, 2012.
7. K. D. Devine, E. G. Boman, R. T. Heaphy, R. H. Bisseling, and Ü. V. Catalyürek, "Parallel hypergraph partitioning for scientific computing," in IPDPS, 2006.
8. E. Solomonik, A. Buluc,, and J. Demmel, "Minimizing Communication in All-Pairs Shortest Paths," in IPDPS, 2013.
9. B. Wu and Y. Du, "Cloud-Based Connected Component Algorithm," in ICAICI, 2010, pp. 122-126.
10. D. Merrill, M. Garland, and A. S. Grimshaw, "Scalable GPU graph traversal," in PPOPP, 2012.
11. E. J. Riedy, D. A. Bader, and H. Meyerhenke, "Scalable Multi-threaded Community Detection in Social Networks," in IPDPS Workshops, 2012.
12. I. X. Leung, P. Hui, P. Liò, and J. Crowcroft, "Towards Real-Time Community Detection in Large Networks," Physical Review E, 2009.
13. Ü. V. C, atalyürek, K. Kaya, A. E. Sariyüce, and E. Saule, "Shattering and Compressing Networks for Betweenness Centrality," in SDM, 2013.
14. M. Redekopp, Y. Simmhan, and V. K. Prasanna, "Optimizations and Analysis of BSP Graph Processing Models on Public Clouds," in IPDPS, 2013.
15. T. G. Kolda, A. Pinar, and C. Seshadhri, "Triadic Measures on Graphs: The Power of Wedge Sampling," in SDM, 2013.
16. K. Dempsey, K. Duraisamy, H. Ali, and S. Bhowmick, "A parallel graph sampling algorithm for analyzing gene correlation networks," Procedia Computer Science, 2011.
17. "Neo4j." [Online]. Available: http://www.neo4j.org/
18. B. Iordanov, "HyperGraphDB: a Generalized Graph Database," Web-Age Information Management, pp. 25-36, 2010.
19. A. Kyrola, G. Blelloch, and C. Guestrin, "GraphChi: Large-scale Graph Computation on Just a PC," in OSDI, 2012.
20. T. White, Hadoop: The definitive guide. O'Reilly Media, Inc., 2012.
21. "YARN." [Online]. Available: http://hadoop.apache.org/docs/ current/ hadoop-yarn/hadoop-yarn-site/YARN.html
22. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly, "Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks," ACM SIGOPS Operating Systems Review, 2007.
23. D. Warneke and O. Kao, "Nephele: Efficient Parallel Data Processing in the Cloud," in MTAGS, 2009.
24. Y. Bu, B. Howe, M. Balazinska, and M. D. Ernst, "HaLoop: Efficient Iterative Data Processing on Large Clusters," VLDB, pp. 285-296, 2010.
25. "Giraph." [Online]. Available: http://giraph.apache.org/
26. U. Kang, C. E. Tsourakakis, and C. Faloutsos, "PEGASUS: A Peta-Scale Graph Mining System Implementation and Observations," in ICDM. IEEE, 2009, pp. 229-238.
27. Y. Low, D. Bickson, J. Gonzalez, C. Guestrin, A. Kyrola, and J. M. Hellerstein, "Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud," in VLDB, 2012, pp. 716-727.
28. B. Shao, H. Wang, and Y. Li, "The Trinity Graph Engine," Technical Report 161291, Microsoft Research, Tech. Rep., 2012.
29. Graph500. [Online]. Available: http://www.graph500.org/
30. T. Suzumura, K. Ueno, H. Sato, K. Fujisawa, and S. Matsuoka, "Performance Characteristics of Graph500 on Large-Scale Distributed Environment," in ISWC, 2011, pp. 149-158.
31. Y. Guo, et al., "How Well do Graph-Processing Platforms Perform? An Empirical Performance Evaluation and Analysis: Extended Report," Delft University of Technology, Tech. Rep. PDS-2013-004, 2013, http: //www.pds.ewi.tudelft.nl/research-publications/technical-reports/2013/.
32. R. J. Wilson, Introduction to Graph Theory. Academic Press, 1972.
33. A. Rowstron, D. Narayanan, A. Donnelly, G. O'Shea, and A. Douglas, "Nobody ever got fired for using hadoop on a cluster," in HotCDP, 2012.
34. "SNAP." [Online]. Available: http://snap.stanford.edu/index. html
35. Y. Guo and A. Iosup, "The Game Trace Archive," in NetGames, 2012.
36. "DAS4." [Online]. Available: http://www.cs.vu.nl/das4/
37. B. Ghit, N. Yigitbasi, and D. Epema, "Resource Management for Dynamic MapReduce Clusters in Multicluster Systems," in MTAGS, 2012.
38. M. Ferdman, et al., "Clearing the Clouds: A Study of Emerging Scaleout Workloads on Modern Hardware," in ASPLOS, 2012.
39. C. Vicknair, M. Macias, Z. Zhao, X. Nan, Y. Chen, and D. Wilkins, "A Comparison of a Graph Database and a Relational Database: A Data Provenance Perspective," in Southeast regional conference, 2010.
40. S. Salihoglu and J. Widom, "GPS: A Graph Processing System," Tech. Rep., 2012.
41. A. Chan, F. Dehne, and R. Taylor, "CGMgraph/CGMlib: Implementing and Testing CGM Graph Algorithms on PC Clusters and Shared Memory Machines," IJHPCA, vol. 19, no. 1, pp. 81-97, 2005.
42. B. Elser and A. Montresor, "An Evaluation Study of BigData Frameworks for Graph Processing," in IEEE BigData, 2013.
43. D. Gregor and A. Lumsdaine, "The Parallel BGL: A Generic Library for Distributed Graph Computations," POOSC, 2005.
44. K. Kambatla, G. Kollias, and A. Grama, "Efficient Large-Scale Graph Analysis in MapReduce," in PMAA, 2012.