Scalable performance of ScaleGraph for large scale graph analysis

2012 19th International Conference on High Performance Computing, HiPC 2012

Volumn , Issue , 2012, Pages

  Dayarathna, Miyuru ^a   Houngkaew, Charuwat ^a   Ogata, Hidefumi ^a   Suzumura, Toyotaro ^a,b  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^b IBM RESEARCH   (United States)

Author keywords

distributed computing; HPCS; large graph analytics; PGAS; programming techniques; reusable libraries; X10

Indexed keywords

HPCS; LARGE GRAPHS; PGAS; PROGRAMMING TECHNIQUE; REUSABLE LIBRARY; X10;

COMPUTER SCIENCE; DISTRIBUTED COMPUTER SYSTEMS;

CLUSTERING ALGORITHMS;

EID: 84880388376     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/HiPC.2012.6507498     Document Type: Conference Paper

References (61)

1. J. Shalf, S. Dosanjh, and J. Morrison, "Exascale computing technology challenges," in Proceedings of the 9th international conference on High performance computing for computational science, ser. VECPAR'10. Berlin, Heidelberg: Springer-Verlag, 2011, pp. 1-25.
2. V. Agarwal, F. Petrini, D. Pasetto, and D. A. Bader, "Scalable graph exploration on multicore processors," in Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, ser. SC '10. Washington, DC, USA: IEEE Computer Society, 2010, pp. 1-11.
3. Facebook, "Facebook's latest news, announcements and media resources," URL: http://newsroom.fb.com/, May 2012.
4. A. Yonezawa, T. Watanabe, M. Yokokawa, M. Sato, and K. Hirao, "Advanced institute for computational science (aics): Japanese national high-performance computing research institute and its 10-petaflops supercomputer "k"," in State of the Practice Reports, ser. SC '11. New York, NY, USA: ACM, 2011, pp. 13:1-13:8.
5. V. Sarkar, W. Harrod, and A. E. Snavely, "Software challenges in extreme scale systems," Journal of Physics: Conference Series, vol. 180, no. 1, p. 012045, 2009.
6. K. Madduri, B. Hendrickson, J. Berry, D. Bader, and J. Crobak, Multithreaded Algorithms for Processing Massive Graphs, 2008.
7. A. Vajda, Programming Many-Core Chips. Springer-Verlag, 2011.
8. U. Consortium, "Upc language specifications v1.2," May 2005.
9. IBM, "X10 language specification (version 2.2)," Jan 2012.
10. B. Chamberlain, D. Callahan, and H. Zima, "Parallel programmability and the chapel language," Int. J. High Perform. Comput. Appl., vol. 21, no. 3, pp. 291-312, Aug. 2007.
11. M. Dayarathna, C. Houngkaew, and T. Suzumura, "Introducing scalegraph: an x10 library for billion scale graph analytics," in Proceedings of the ACM SIGPLAN 2012 X10 Workshop, ser. X10 '12. New York, NY, USA: ACM, 2012, pp. 6:1-6:9.
12. G. Csardi and T. Nepusz, "The igraph software package for complex network research," InterJournal, vol. Complex Systems, p. 1695, 2006. [Online]. Available: http://igraph.sf.net
13. L.-Q. Lee, J. G. Siek, and A. Lumsdaine, "The generic graph component library," SIGPLAN Not., vol. 34, pp. 399-414, October 1999.
14. J. Leskovec, "Snap: Stanford network analysis project," URL: http: //snap.stanford.edu/, Jan. 2012.
15. R. Garcia, J. Jarvi, A. Lumsdaine, J. G. Siek, and J. Willcock, "A comparative study of language support for generic programming," ser. OOPSLA '03. New York, NY, USA: ACM, 2003, pp. 115-134.
16. D. Batenkov, "Boosting productivity with the boost graph library," XRDS, vol. 17, pp. 31-32, Mar. 2011.
17. D. Gregor and A. Lumsdaine, "Lifting sequential graph algorithms for distributed-memory parallel computation," SIGPLAN Not., vol. 40, pp. 423-437, October 2005.
18. J. Law, "Review of "the boost graph library: user guide and reference manual by jeremy g. siek, lie-quan lee, and andrew lumsdaine." addison-wesley 2002," ACM SIGSOFT Software Engineering Notes, vol. 28, no. 2, pp. 35-36, 2003.
19. V. S. Kemal Ebcioglu, Vijay Saraswat, "X10: Programming for hierarchical parallelism and non-uniform data access," in 3rd International Workshop on Language Runtimes, Impact of Next Generation Processor Architectures on Virtual Machine Technologies, 2004.
20. F. Hielscher and P. Gottschling, "Pargraph," URL: http://pargraph. sourceforge.net/, Jan. 2012.
21. P. An, A. Jula, S. Rus, S. Saunders, T. Smith, G. Tanase, N. Thomas, N. Amato, and L. Rauchwerger, "Stapl: an adaptive, generic parallel c++ library," in Proceedings of the 14th international conference on Languages and compilers for parallel computing, ser. LCPC'01. Berlin, Heidelberg: Springer-Verlag, 2003, pp. 193-208.
22. K. Pingali, D. Nguyen, M. Kulkarni, M. Burtscher, M. A. Hassaan, R. Kaleem, T.-H. Lee, A. Lenharth, R. Manevich, M. Méndez-Lojo, D. Prountzos, and X. Sui, "The tao of parallelism in algorithms," in Proceedings of the 32nd ACM SIGPLAN conference on Programming language design and implementation, ser. PLDI '11. New York, NY, USA: ACM, 2011, pp. 12-25.
23. Sourceforge, "Jung - java universal network/graph framework," URL: http://jung.sourceforge.net/index.html, Jan. 2012.
24. J. O'Madadhain, D. Fisher, S. White, and Y. Boey, "The JUNG (Java Universal Network/Graph) Framework," UCI-ICS, Tech. Rep., Oct. 2003.
25. A. Buluc and J. R. Gilbert, "The combinatorial blas: design, implementation, and applications," International Journal of High Performance Computing Applications, vol. 25, no. 4, pp. 496-509, 2011.
26. A. Lugowski, D. Alber, A. Buluc, J. Gilbert, S. Reinhardt, Y. Teng, and A. Waranis, "A flexible open-source toolbox for scalable complex graph analysis," in SIAM Conference on Data Mining (SDM), 2012 (accepted).
27. D. Bader, G. Cong, and J. Feo, "On the architectural requirements for efficient execution of graph algorithms," in Parallel Processing, 2005. ICPP 2005. International Conference on, june 2005, pp. 547 -556.
28. D. Ediger, K. Jiang, J. Riedy, D. A. Bader, and C. Corley, "Massive social network analysis: Mining twitter for social good," in Proceedings of the 2010 39th International Conference on Parallel Processing, ser. ICPP '10. Washington, DC, USA: IEEE Computer Society, 2010, pp. 583-593.
29. J. Berry, B. Hendrickson, S. Kahan, and P. Konecny, "Software and algorithms for graph queries on multithreaded architectures," in Parallel and Distributed Processing Symposium, 2007. IPDPS 2007. IEEE International, march 2007, pp. 1-14.
30. G. Malewicz, M. H. Austern, A. J. Bik, J. C. Dehnert, I. Horn, N. Leiser, and G. Czajkowski, "Pregel: a system for large-scale graph processing," in Proceedings of the 2010 international conference on Management of data, ser. SIGMOD '10. New York, NY, USA: ACM, 2010, pp. 135-146.
31. B. Barrett, J. Berry, R. Murphy, and K. Wheeler, "Implementing a portable multi-threaded graph library: The mtgl on qthreads," in Parallel Distributed Processing, 2009. IPDPS 2009. IEEE International Symposium on, may 2009, pp. 1-8.
32. G. Cong, G. Almasi, and V. Saraswat, "Fast pgas implementation of distributed graph algorithms," ser. SC '10. Washington, DC, USA: IEEE Computer Society, 2010, pp. 1-11.
33. -, "Fast pgas connected components algorithms," ser. PGAS '09. New York, NY, USA: ACM, 2009, pp. 13:1-13:6.
34. M. Kulkarni, K. Pingali, B. Walter, G. Ramanarayanan, K. Bala, and L. P. Chew, "Optimistic parallelism requires abstractions," in Proceedings of the 2007 ACM SIGPLAN conference on Programming language design and implementation, ser. PLDI '07. New York, NY, USA: ACM, 2007, pp. 211-222.
35. IBM, "X10: Performance and productivity at scale," URL: http://x10-lang.org/, Jan. 2012.
36. P. Charles, C. Grothoff, V. Saraswat, C. Donawa, A. Kielstra, K. Ebcioglu, C. von Praun, and V. Sarkar, "X10: an object-oriented approach to non-uniform cluster computing," in Proceedings of the 20th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, ser. OOPSLA '05. New York, NY, USA: ACM, 2005, pp. 519-538.
37. J. Dongarra and et al., "The international exascale software project roadmap," International Journal of High Performance Computing Applications, vol. 25, no. 1, pp. 3-60, 2011.
38. S. Agarwal, R. Barik, V. Sarkar, and R. K. Shyamasundar, "Mayhappen- in-parallel analysis of x10 programs," ser. PPoPP '07, 2007, pp. 183-193.
39. D. Grove, O. Tardieu, D. Cunningham, B. Herta, I. Peshansky, and V. Saraswat, "A performance model for x10 applications: What's going on under the hood?" 2011.
40. D. Cunningham, R. Bordawekar, and V. Saraswat, "Gpu programming in a high level language: Compiling x10 to cuda," 2011.
41. J. Milthorpe, V. Ganesh, A. Rendell, and D. Grove, "X10 as a parallel language for scientific computation: Practice and experience," in Parallel Distributed Processing Symposium (IPDPS), 2011 IEEE International, may 2011, pp. 1080-1088.
42. A. X. Project, "Xerces-c++ xml parser," URL: http://xerces.apache.org/xerces-c/, Jan. 2012.
43. L. S. Blackford, J. Choi, A. Cleary, E. D'Azevedo, J. Demmel, I. Dhillon, J. Dongarra, S. Hammarling, G. Henry, A. Petitet, K. Stanley, D. Walker, and R. C. Whaley, ScaLAPACK Users' Guide. Philadelphia, PA: Society for Industrial and Applied Mathematics, 1997.
44. S. S. Skiena, The Algorithm Design Manual, 2nd ed. Springer, 2008.
45. D. Chakrabarti, Y. Zhan, and C. Faloutsos, "R-MAT: A Recursive Model for Graph Mining," in Fourth SIAM International Conference on Data Mining, Apr. 2004.
46. M. Hay, C. Li, G. Miklau, and D. Jensen, "Accurate estimation of the degree distribution of private networks," in Data Mining, 2009. ICDM '09. Ninth IEEE International Conference on, dec. 2009, pp. 169-178.
47. M. Newmann, A.-L. Barabasi, and D. J. Watts, The Structure and Dynamics of Networks. Princeton University Press, 2006.
48. J. Anthonisse, "The rush in a directed graph," Technical Report BN, vol. 9/71, 1971.
49. L. C. Freeman, "A Set of Measures of Centrality Based on Betweenness," Sociometry, vol. 40, no. 1, pp. 35-41, Mar. 1977.
50. M. Newmann, Networks: An Introduction. Oxford University Press, 2010.
51. U. Brandes, "A Faster Algorithm for Betweenness Centrality," Journal of Mathematical Sociology, vol. 25, pp. 163-177, 2001.
52. S. E. Schaeffer, "Graph clustering," Computer Science Review, vol. 1, no. 1, pp. 27 -64, 2007.
53. C. C. Aggarwal and H. Wang, "A survey of clustering algorithms for graph data," in Managing and Mining Graph Data, ser. The Kluwer International Series on Advances in Database Systems, C. C. Aggarwal, H. Wang, and A. K. Elmagarmid, Eds. Springer US, 2010, vol. 40, pp. 275-301.
54. S. Fortunato, "Community detection in graphs," CoRR, vol. abs/0906.0612, 2009.
55. S. Marsland, Machine Learning : An Algorithmic Perspective. Chapman & Hall/CRC, 2009.
56. T. Endo, A. Nukada, S. Matsuoka, and N. Maruyama, "Linpack evaluation on a supercomputer with heterogeneous accelerators," in IPDPS, 2010, pp. 1-8.
57. N.-B. C. Laboratory, "Mvapich: Mpi over infiniband, 10gige/iwarp and roce," URL: http://mvapich.cse.ohio-state.edu/, May 2012.
58. U. Kang, "Dataset," URL: http://www.cs.cmu.edu/-ukang/dataset/, May 2012.
59. L. Backstrom, D. Huttenlocher, J. Kleinberg, and X. Lan, "Group formation in large social networks: membership, growth, and evolution," in Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, ser. KDD '06. New York, NY, USA: ACM, 2006, pp. 44-54.
60. J. Leskovec, K. J. Lang, A. Dasgupta, and M. W. Mahoney, "Community structure in large networks: Natural cluster sizes and the absence of large well-defined clusters," Internet Mathematics, vol. 6, no. 1, pp. 29-123, 2009.
61. J. Leskovec, "Snap: Network datasets: Livejournal social network," URL: http://snap.stanford.edu/data/soc-LiveJournal1.html, May 2012.