PEGASUS: A peta-scale graph mining system - Implementation and observations

Proceedings - IEEE International Conference on Data Mining, ICDM

Volumn , Issue , 2009, Pages 229-238

  Kang, U ^a   Tsourakakis, Charalampos E ^a   Faloutsos, Christos ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Graph mining; HADOOP; PEGASUS

Indexed keywords

CONNECTED COMPONENT; DIAMETER ESTIMATION; GRAPH MINING; MATRIX VECTOR MULTIPLICATION; OPEN SOURCES; PAGERANK; PEGASUS; RUNNING TIME; SCALE-UP; SPECTRAL CLUSTERING; WEB GRAPHS;

MINING; SUPERCOMPUTERS;

MULTIPROCESSING SYSTEMS;

EID: 77951152705     PISSN: 15504786     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDM.2009.14     Document Type: Conference Paper

References (43)

1. A. Broder, R. Kumar, F. Maghoul, P. Raghavan, S. Rajagopalan, R. Stata, A. Tomkins, and J. Wiener, "Graph structure in the web," Computer Networks 33, 2000.
2. J. Dean and S. Ghemawat, "Mapreduce: Simplified data processing on large clusters," OSDI, 2004.
3. J. Chen, O. R. Zaiane, and R. Goebel, "Detecting communities in social networks using max-min modularity," SDM, 2009.
4. T. Falkowski, A. Barth, and M. Spiliopoulou, "Dengraph: A density-based community detection algorithm," Web Intelligence, 2007.
5. G. Karypis and V. Kumar, "Parallel multilevel kway partitioning for irregular graphs," SIAM Review, vol. 41, no. 2, 1999.
6. S. Ranu and A. K. Singh, "Graphsig: A scalable approach to mining significant subgraphs in large graph databases," ICDE, 2009.
7. Y. Ke, J. Cheng, and J. X. Yu, "Top-k correlative graph mining," SDM, 2009.
8. P. Hintsanen and H. Toivonen, "Finding reliable subgraphs from large probabilistic graphs," PKDD, 2008.
9. J. Cheng, J. X. Yu, B. Ding, P. S. Yu, and H. Wang, "Fast graph pattern matching," ICDE, 2008.
10. F. Zhu, X. Yan, J. Han, and P. S. Yu, "gprune: A constraint pushing framework for graph pattern mining," PAKDD, 2007.
11. C. Chen, X. Yan, F. Zhu, and J. Han, "gapprox: Mining frequent approximate patterns from a massive network," ICDM, 2007.
12. X. Yan and J. Han, "gspan: Graph-based substructure pattern mining," ICDM, 2002.
13. N. S. Ketkar, L. B. Holder, and D. J. Cook, "Subdue: Compression-based frequent pattern discovery in graph data," OSDM, August 2005.
14. M. Kuramochi and G. Karypis, "Finding frequent patterns in a large sparse graph," SIAM Data Mining Conference, 2004.
15. C. Wang, W. Wang, J. Pei, Y. Zhu, and B. Shi, "Scalable mining of large disk-based graph databases," KDD, 2004.
16. N. Wang, S. Parthasarathy, K.-L. Tan, and A. K. H. Tung, "Csv: Visualizing and mining cohesive subgraph," SIGMOD, 2008.
17. S. Brin and L. Page, "The anatomy of a large-scale hypertextual (web) search engine." in WWW, 1998.
18. J. Kleinberg, "Authoritative sources in a hyperlinked environment," in Proc. 9th ACM-SIAM SODA, 1998.
19. C. E. Tsourakakis, U. Kang, G. L. Miller, and C. Faloutsos, "Doulion: Counting triangles in massive graphs with a coin," KDD, 2009.
20. C. E. Tsourakakis, M. N. Kolountzakis, and G. L. Miller, "Approximate triangle counting," Apr 2009. [Online]. Available: http://arxiv.org/abs/0904.3761.
21. U. Kang, C. Tsourakakis, A. Appel, C. Faloutsos, and J. Leskovec, "Hadi: Fast diameter estimation and mining in massive graphs with hadoop," CMU-ML-08-117, 2008.
22. T. Qian, J. Srivastava, Z. Peng, and P. C. Sheu, "Simultaneouly finding fundamental articles and new topics using a community tracking method," PAKDD, 2009.
23. N. Shrivastava, A. Majumder, and R. Rastogi, "Mining (social) network graphs to detect random link attacks," ICDE, 2008.
24. Y. Shiloach and U. Vishkin, "An o(logn) parallel connectivity algorithm," Journal of Algorithms, pp. 57-67,
25. B. Awerbuch and Y. Shiloach, "New connectivity and msf algorithms for ultracomputer and pram," ICPP, 1983.
26. D. Hirschberg, A. Chandra, and D. Sarwate, "Computing connected components on parallel computers," Communications of the ACM, vol. 22, no. 8, pp. 461- 464, 1979.
27. J. Greiner, "A comparison of parallel algorithms for connected components," Proceedings of the 6th ACM Symposium on Parallel Algorithms and Architectures, June 1994.
28. G. Aggarwal, M. Data, S. Rajagopalan, and M. Ruhl, "On the streaming model augmented with a sorting primitive," Proceedings of FOCS, 2004.
29. R. Lämmel, "Google's mapreduce programming model - revisited," Science of Computer Programming, vol. 70, pp. 1-30, 2008.
30. "Hadoop information," http://hadoop.apache.org/.
31. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins, "Pig latin: a not-so-foreign language for data processing," in SIGMOD '08, 2008, pp. 1099-1110.
32. S. Papadimitriou and J. Sun, "Disco: Distributed coclustering with map-reduce," ICDM, 2008.
33. R. Chaiken, B. Jenkins, P.-A. Larson, B. Ramsey, D. Shakib, S. Weaver, and J. Zhou, "Scope: easy and efficient parallel processing of massive data sets," VLDB, 2008.
34. R. Pike, S. Dorward, R. Griesemer, and S. Quinlan, "Interpreting the data: Parallel analysis with sawzall," Scientific Programming Journal, 2005.
35. R. L. Grossman and Y. Gu, "Data mining using high performance data clouds: experimental studies using sector and sphere," KDD, 2008.
36. J.-Y. Pan, H.-J. Yang, C. Faloutsos, and P. Duygulu, "Automatic multimedia cross-modal correlation discovery," ACM SIGKDD, Aug. 2004.
37. J. Leskovec, D. Chakrabarti, J. M. Kleinberg, and C. Faloutsos, "Realistic, mathematically tractable graph generation and evolution, using kronecker multiplication," PKDD, 2005.
38. M. E. J. Newman, "Power laws, pareto distributions and zipf's law," Contemporary Physics, no. 46, pp. 323-351, 2005.
39. M. Mcglohon, L. Akoglu, and C. Faloutsos, "Weighted graphs and disconnected components: patterns and a generator," KDD, pp. 524-532, 2008.
40. R. Dunbar, "Grooming, gossip, and the evolution of language," Harvard Univ Press, October 1998.
41. G. Pandurangan, P. Raghavan, and E. Upfal, "Using pagerank to characterize web structure," COCOON, August 2002.
42. "Hama website," http://incubator.apache.org/hama/.
43. T. G. Kolda and J. Sun, "Scalable tensor decompsitions for multi-aspect data mining," ICDM, 2008.