MapReduce in MPI for Large-scale graph algorithms

Parallel Computing

Volumn 37, Issue 9, 2011, Pages 610-632

  Plimpton, Steven J ^a   Devine, Karen D ^a  

^a SANDIA NATIONAL LABORATORIES   (United States)

Author keywords

Graph algorithms; MapReduce; Message passing; MPI; R MAT matrices

Indexed keywords

CONNECTED COMPONENT; DATA MOVEMENTS; DATA SETS; DISTRIBUTED MEMORY CLUSTERS; GRAPH ALGORITHMS; INDEPENDENT SET; MAPREDUCE; MPI; OPEN-SOURCE LIBRARIES; OUT-OF-CORE; PAGERANK; PARALLEL CODE; PARALLEL LIBRARY; PARALLEL MACHINE; PARALLEL PLATFORMS; R-MAT MATRICES; SCRIPTING LANGUAGES; SHORTEST-PATH; VERTEX-RANKING;

COMPUTER PROGRAMMING; MESSAGE PASSING; PARALLEL PROCESSING SYSTEMS; PROGRAM PROCESSORS;

CLUSTERING ALGORITHMS;

EID: 80052031568     PISSN: 01678191     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.parco.2011.02.004     Document Type: Article

References (22)

1. Apache Hadoop WWW site: .
2. Titan Informatics Toolkit WWW site: .
3. S. Balay, W.D. Gropp, L.C. McInnes, and B.F. Smith Efficient management of parallelism in object oriented numerical software libraries E. Arge, A.M. Bruaset, H.P. Langtangen, Modern Software Tools in Scientific Computing 1997 Birkhäuser Press 163 202
4. R. Bellman On a routing problem Quart. Appl. Math. 16 1 1958 87 90
5. J.W. Berry, B. Hendrickson, S. Kahan, P. Konecny, Software and algorithms for graph queries on multithreaded architectures, in: Proceedings of the 21st International Parallel and Distributed Processing Symposium, 2007.
6. C. Bisciglia, A. Kimball, S. Michels-Slettvet, Lecture 5: Graph algorithms and PageRank, 2007. .
7. D. Chakrabarti, Y. Zhan, C. Faloutsos, R-MAT: a recursive model for graph mining, in: SIAM Data Mining, 2004.
8. J. Cohen Graph twiddling in a mapreduce world Comput. Sci. Eng. 11 2009 29 41
9. J. Dean, S. Ghemawat, MapReduce: simplified data processing on large clusters. in: OSDI04: sixth Symposium on Operating System Design and Implementation, 2004.
10. J. Dean, and S. Ghemawat Mapreduce: simplified data processing on large clusters Commun. ACM 51 1 2008 107 113
11. J. Ekanayake, G. Fox, High performance parallel computing with clouds and cloud technologies, in: First International Conference on Cloud Computing (CloudComp09), 2009.
12. L.R. Ford, and D.R. Fulkerson Flows in Networks 1962 Princeton University Press
13. D. Gregor, A. Lumsdaine, The parallel BGL: a generic library for distributed graph computations, in: Proceedings of the Fourth Workshop on Parallel Object-Oriented Scientific Computing, 2005.
14. M. Heroux, R. Bartlett, V.H.R. Hoekstra, J. Hu, T. Kolda, R. Lehoucq, K. Long, R. Pawlowski, E. Phipps, A. Salinger, H. Thornquist, R. Tuminaro, J. Willenbring, A. Williams, An overview of Trilinos, Technical Report SAND2003-2927, Sandia National Laboratories, 2003.
15. T. Hoefler, A. Lumsdaine, and J. Dongarra Towards efficient mapreduce using mpi M. Ropo, J. Westerholm, J. Dongarra, volume 5759 of Lecture Notes in Computer Science 2009 Springer 240 249
16. U. Kang, C. Tsourakakis, C. Faloutsos, PEGASUS: a peta-scale graph mining system - implementation and observations, in: ICDM: IEEE International Conference on Data Mining, 2009.
17. A.N. Langville, and C.D. Meyer A survey of eigenvector methods for web information retrieval SIAM Rev. 47 1 2005 135 161 (Pubitemid 40535974)
18. J. Lin, and C. Dyer Data-Intensive Text Processing with Mapreduce 2010 Morgan & Claypool
19. M. Luby A simple parallel algorithm for the maximal independent set problem SIAM J. Comput. 15 4 1986 1036 1055
20. K. Madduri, D.A. Bader, J.W. Berry, and J.R. Crobak An experimental study of a parallel shortest path algorithm for solving large-scale graph instances ALENEX 2007 SIAM
21. U. Meyer, and P. Sanders Delta-stepping: a parallel single source shortest path algorithm ESA '98: Proceedings of the 6th Annual European Symposium on Algorithms 1998 Springer-Verlag 393 404 (Pubitemid 128117071)
22. T. Tu, C.A. Rendleman, D.W. Borhani, R.O. Dror, J. Gullingsrud, M.O. Jensen, J.L. Klepeis, P. Maragakis, P. Miller, K.A. Stafford, and D.E. Shaw A scalable parallel framework for analyzing terascale molecular dynamics simulation trajectories SC '08: Proceedings of the 2008 ACM/IEEE Conference on Supercomputing 2008 IEEE Press Piscataway, NJ, USA 1 12