Graphchi: Large-scale graph computation on just a PC

Proceedings of the 10th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2012

Volumn , Issue , 2012, Pages 31-46

  Kyrola, Aapo ^a   Blelloch, Guy ^a   Guestrin, Carlos ^b  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b UNIVERSITY OF WASHINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLUSTER COMPUTING; DIGITAL STORAGE; GRAPHIC METHODS; LEARNING ALGORITHMS; MACHINE LEARNING; SOCIAL SCIENCES COMPUTING; SYSTEMS ANALYSIS;

DISK-BASED SYSTEMS; DISTRIBUTED COMPUTATIONAL RESOURCES; DISTRIBUTED GRAPH ALGORITHMS; DISTRIBUTED SYSTEMS; REAL-WORLD PROBLEM; SINGLE COMPUTER; SLIDING WINDOW; SUPPORT GRAPHS;

DATA MINING;

EID: 85042632297     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (47)

1. A. Aggarwal, J. Vitter, et al. The input/output complexity of sorting and related problems. Communications of the ACM, 31(9):1116-1127, 1988.
2. L. Arge. The buffer tree: A new technique for optimal i/o-algorithms. Algorithms and Data Structures, pages 334-345, 1995.
3. L. Backstrom, D. Huttenlocher, J. Kleinberg, and X. Lan. Group formation in large social networks: membership, growth, and evolution. The 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD'06. ACM, 2006.
4. A. Badam and V. S. Pai. Ssdalloc: hybrid ssd/ram memory management made easy. In Proc. of the 8th USENIX conference on Networked systems design and implementation, NSDI'11, pages 16-16, Boston, MA, 2011. USENIX Association.
5. M. Bender, G. Brodal, R. Fagerberg, R. Jacob, and E. Vicari. Optimal sparse matrix dense vector multiplication in the i/o-model. Theory of Computing Systems, 47(4):934-962, 2010.
6. J. Bennett and S. Lanning. The netflix prize. In Proc. of the KDD Cup Workshop 2007, pages 3-6, San Jose, CA, Aug. 2007. ACM.
7. D. P. Bertsekas and J. N. Tsitsiklis. Parallel and distributed computation: numerical methods. Prentice-Hall, Inc., 1989.
8. D. K. Blandford, G. E. Blelloch, and I. A. Kash. Compact representations of separable graphs. In In Proc. of the Annual ACM-SIAM Symposium on Discrete Algorithms, pages 679-688, 2003.
9. G. Blelloch, H. Simhadri, and K. Tangwongsan. Parallel and i/o efficient set covering algorithms. In Proc. of the 24th ACM symposium on Parallelism in algorithms and architectures, pages 82-90, 2012.
10. G. E. Blelloch. Prefix sums and their applications. Synthesis of Parallel Algorithms, 1990.
11. P. Boldi, M. Santini, and S. Vigna. A large time-aware graph. SIGIR Forum, 42(2):33-38, 2008.
12. P. Boldi and S. Vigna. The webgraph framework i: compression techniques. In Proc. of the 13th international conference on World Wide Web, pages 595-602. ACM, 2004.
13. R. Chen, X. Weng, B. He, and M. Yang. Large graph processing in the cloud. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of data, SIGMOD'10, pages 1123-1126, Indianapolis, Indiana, USA, 2010. ACM.
14. Y. Chen, Q. Gan, and T. Suel. I/O-efficient techniques for computing pagerank. In Proc. of the eleventh international conference on Information and knowledge management, pages 549-557, McLean, Virginia, USA, 2002. ACM.
15. R. Cheng, J. Hong, A. Kyrola, Y. Miao, X. Weng, M. Wu, F. Yang, L. Zhou, F. Zhao, and E. Chen. Kineograph: taking the pulse of a fast-changing and connected world. In Proc. of the 7th ACM european conference on Computer Systems, EuroSys'12, pages 85-98, Bern, Switzerland, 2012. ACM.
16. Y.-J. Chiang, M. T. Goodrich, E. F. Grove, R. Tamassia, D. E. Vengroff, and J. S. Vitter. External-memory graph algorithms. In Proc. of the sixth annual ACM-SIAM symposium on Discrete algorithms, SODA'95, pages 139-149, Philadelphia, PA, 1995. Society for Industrial and Applied Mathematics.
17. F. Chierichetti, R. Kumar, S. Lattanzi, M. Mitzenmacher, A. Panconesi, and P. Raghavan. On compressing social networks. In Proc. of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 219-228, Paris, France, April 2009. ACM.
18. S. Chu and J. Cheng. Triangle listing in massive networks and its applications. In In Proc. of the 17th ACM SIGKDD international conf. on Knowledge discovery and data mining, pages 672-680, 2011.
19. J. Dean and S. Ghemawat. Mapreduce: simplified data processing on large clusters. In Proc. of the 6th USENIX conference on Operating systems design and implementation, OSDI'04, pages 10-10, San Francisco, CA, 2004. USENIX.
20. J. Gonzalez, Y. Low, H. Gu, D. Bickson, and C. Guestrin. PowerGraph: Distributed graph-parallel computation on natural graphs. In Proc. of the 10th USENIX conference on Operating systems design and implementation, OSDI'12, Hollywood, CA, 2012.
21. T. Haveliwala. Efficient computation of pagerank. Technical report, Stanford University, 1999.
22. U. Kang, D. Chau, and C. Faloutsos. Inference of beliefs on billion-scale graphs. In The 2nd Workshop on Large-scale Data Mining: Theory and Applications, Washington, D.C., 2010.
23. U. Kang and C. Faloutsos. Beyond'caveman communities': Hubs and spokes for graph compression and mining. In 11th International Conference on Data Mining (ICDM'11), pages 300-309, Vancouver, Canada, 2011.
24. U. Kang, C. E. Tsourakakis, and C. Faloutsos. Pegasus: A peta-scale graph mining system implementation and observations. ICDM'09. IEEE Computer Society, 2009.
25. G. Karypis and V. Kumar. Multilevel k-way partitioning scheme for irregular graphs. J. Parallel Distrib. Comput., 48(1):96-129, 1998.
26. H. Kwak, C. Lee, H. Park, and S. Moon. What is Twitter, a social network or a news media? In Proc. of the 19th international conference on World wide web, pages 591-600. ACM, 2010.
27. J. Leskovec, K. Lang, A. Dasgupta, and M. Mahoney. Community structure in large networks: Natural cluster sizes and the absence of large well-defined clusters. Internet Mathematics, 6(1):29-123, 2009.
28. X. Liu and T. Murata. Advanced modularity-specialized label propagation algorithm for detecting communities in networks. Physica A: Stat. Mechanics and its Applications, 389(7):1493-1500, 2010.
29. Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin, and J. M. Hellerstein. GraphLab: A new parallel framework for machine learning. In Conference on Uncertainty in Artificial Intelligence (UAI), Catalina Island, CA, July 2010.
30. Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin, and J. M. Hellerstein. Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud. PVLDB, 2012.
31. G. Malewicz, M. H. Austern, A. J. Bik, J. Dehnert, I. Horn, N. Leiser, and G. Czajkowski. Pregel: a system for large-scale graph processing. SIGMOD 10: Proc. of the 2010 international conference on Management of data, Indianapolis, IN, 2010.
32. L. Page, S. Brin, R. Motwani, and T. Winograd. The pagerank citation ranking: Bringing order to the web. Technical Report 1999-66, Stanford InfoLab, 1999.
33. D. A. Patterson, G. Gibson, and R. H. Katz. A case for redundant arrays of inexpensive disks (RAID). In Proc. of the 1988 ACM SIGMOD international conference on Management of data, SIGMOD'88, pages 109-116, Chicago, IL, 1988.
34. R. Pearce, M. Gokhale, and N. Amato. Multithreaded Asynchronous Graph Traversal for In-Memory and Semi-External Memory. In SuperComputing, 2010.
35. J. Pearl. Reverend Bayes on inference engines: A distributed hierarchical approach. Cognitive Systems Laboratory, School of Engineering and Applied Science, University of California, Los Angeles, 1982.
36. R. Power and J. Li. Piccolo: building fast, distributed programs with partitioned tables. In Proc. of the 9th USENIX conference on Operating systems design and implementation, OSDI'10, pages 1-14, 2010.
37. S. Salihoglu and J. Widom. GPS: a graph processing system. Technical report, Stanford University, 2012.
38. I. Stanton and G. Kliot. Streaming graph partitioning for large distributed graphs. Technical report, Microsoft Research, 2012.
39. S. Suri and S. Vassilvitskii. Counting triangles and the curse of the last reducer. In In Proc. of the 20th international conference on World wide web, pages 607-614, Lyon, France, 2011. ACM.
40. S. Toledo. A survey of out-of-core algorithms in numerical linear algebra. External Memory Algorithms and Visualization, 50:161-179, 1999.
41. L. Valiant. A bridging model for parallel computation. Communications of the ACM, 33(8):103-111, 1990.
42. J. Vitter. External Memory Algorithms. ESA, 1998.
43. D. Watts and S. Strogatz. Collective dynamics of small-world networks. Nature, 393(6684):440-442, 1998.
44. Yahoo WebScope. Yahoo! altavista web page hyperlink connectivity graph, circa 2002, 2012. http://webscope.sandbox.yahoo.com/.
45. M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica. Spark: Cluster computing with working sets. In HotCloud, 2010.
46. Y. Zhou, D. Wilkinson, R. Schreiber, and R. Pan. Large-scale parallel collaborative filtering for the netflix prize. In In Proc. of the 4th international conference on Algorithmic Aspects in Information and Management, AAIM'08, pages 337-348, Berlin, Heidelberg, 2008. Springer-Verlag.
47. X. Zhu and Z. Ghahramani. Learning from labeled and unlabeled data with label propagation. Technical report, Carnegie Mellon University, 2002.