Large scale complex network analysis using the hybrid combination of a mapreduce cluster and a highly multithreaded system

Proceedings of the 2010 IEEE International Symposium on Parallel and Distributed Processing, Workshops and Phd Forum, IPDPSW 2010

Volumn , Issue , 2010, Pages

  Kang, Seunghwa ^a   Bader, David A ^a  

^a GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Cloud computing; Parallel algorithms; Power law graph

Indexed keywords

APPLICATION AREA; CLOUD COMPUTING; COMPLEX NETWORK ANALYSIS; COMPLEX NETWORKS; GRAPH REPRESENTATION; IRREGULAR STRUCTURES; MULTI-THREADED SYSTEM; ORDERS OF MAGNITUDE; POWER-LAW GRAPH; SUBGRAPHS;

COMPUTATION THEORY; DISTRIBUTED COMPUTER SYSTEMS; DISTRIBUTED PARAMETER NETWORKS; ELECTRIC GROUNDING; ELECTRIC NETWORK ANALYSIS; HYBRID COMPUTERS; HYBRID SYSTEMS; PARALLEL ALGORITHMS; PARALLEL ARCHITECTURES;

PROBLEM SOLVING;

EID: 77954061771     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPSW.2010.5470691     Document Type: Conference Paper

References (18)

1. Welcome to Apache Hadoop!, 2009. http://hadoop.apache.org/core.
2. R. Albert, H. Jeong, and A.-L. Barabasi. The diameter of the world wide web. Nature, 401:130-131, 1999.
3. R. Albert, H. Jeong, and A.-L. Barabasi. Error and attack tolerance of complex networks. Nature, 406:378-382, 2000.
4. D. A. Bader, G. Cong, and J. Feo. On the architectural requirements for efficient execution of graph algorithms. In Proc. Int'l Conf. on Parallel Processing (ICPP), Oslo, Norway, Jun. 2005.
5. D. A. Bader, V. Kanade, and K. Madduri. SWARM: A parallel programming framework for multi-core processors. In Proc. Workshop on Multithreaded Architectures and Applications (MTAAP), Long Beach, CA, Mar. 2007.
6. D. A. Bader and K. Madduri. Designing multithreaded algorithms for breadth-first search and st-connectivity on the Cray MTA-2. In Proc. Int'l Conf. on Parallel Processing (ICPP), Columbus, OH, Aug. 2006.
7. D. A. Bader and K. Madduri. A graph-theoretic analysis of the human protein-interaction network using multicore parallel algorithms. Parallel Computing, 34(11):627-639, 2008.
8. D. A. Bader and K. Madduri. SNAP, small-world network analysis and partitioning: an open-source parallel graph framework for the exploration of large-scale networks. In Proc. Int'l Parallel and Distributed Processing Symp. (IPDPS), Miami, FL, Apr. 2008.
9. C. Bisciglia, A. Kimball, and S. Michels-Slettvet. Lecture 5: Graph Algorithms & PageRank, 2007. http://code.google.com/edu/submissions/ mapreduce-minilecture/lec5-pagerank.ppt.
10. D. Chakrabarti, Y. Zhan, and C. Faloutsos. R-MAT: A recursive model for graph mining. In Proc. SIAM Int'l Conf. on Data Mining (SDM), Lake Buena Vista, FL, Apr. 2004.
11. J. Cohen. Graph twiddling in a MapReduce world. Computing in Science and Engineering, 11(4):29-41, 2009.
12. L. F. Costa, F. A. Rodrigues, G. Travieso, and P. R. Villas Boas. Characterization of complex networks: A survey of measurements. Advances In Physics, 56(1):167-242, 2007.
13. J. Dean and S. Ghemawat. MapReduce: simplified data processing on large clusters. In Proc. 6th USENIX Symp. on Operating System Design and Implementation (OSDI), San Francisco, CA, Dec. 2004.
14. M. Faloutsos, P. Faloutsos, and C. Faloutsos. On power-law relationships of the internet topology. In Proc. ACM Conf. on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM), Cambridge, MA, Aug. 1999.
15. D. R. Helman and J. JáJá. Prefix computations on symmetric multiprocessors. J. of Parallel and Distributed Computing, 61(2):265-278, 2001.
16. J. JáJá. An Introduction to Parallel Algorithms. Addison- Wesley Publishing Company, 1992.
17. K. Madduri, D. Ediger, K. Jiang, D. A. Bader, and D. Chavarría- Miranda. A faster parallel algorithm and efficient multithreaded implementations for evaluating betweenness centrality on massive datasets. Technical Report LBNL- 1703E, Lawrence Berkeley National Laboratory, Apr. 2009.
18. D. J. Watts and S. H. Strogatz. Collective dynamics of 'smallworld' networks. Nature, 393:440-442, 1998.