Mizan: A system for dynamic load balancing in large-scale graph processing

Proceedings of the 8th ACM European Conference on Computer Systems, EuroSys 2013

Volumn , Issue , 2013, Pages 169-182

  Khayyat, Zuhair ^a   Awara, Karim ^a   Alonazi, Amani ^a   Jamjoom, Hani ^b   Williams, Dan ^b   Kalnis, Panos ^a  

^a KING ABDULLAH UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Saudi Arabia)

^b IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE APPROACH; END-TO-END COMPUTATIONS; GRAPH PARTITIONING; GRAPH PROCESSING; PERFORMANCE IMPROVEMENTS; PRE-PROCESSING STEP; PRIORI KNOWLEDGE; RUNTIME BEHAVIORS;

ALGORITHMS; GRAPH THEORY;

NETWORK MANAGEMENT;

EID: 84877705020     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2465351.2465369     Document Type: Conference Paper

References (31)

1. H. Balakrishnan, M. F. Kaashoek, D. Karger, R. Morris, and I. Stoica. Looking Up Data in P2P Systems. Communications of the ACM, 46(2):43-48, 2003.
2. P. Boldi, B. Codenotti, M. Santini, and S. Vigna. UbiCrawler: A Scalable Fully Distributed Web Crawler. Software: Practice & Experience, 34(8):711-726, 2004.
3. D. Buntinas. Designing a Common Communication Subsystem. In Proceedings of the 12th European Parallel Virtual Machine and Message Passing Interface Conference (Euro PVM MPI), pages 156-166, Sorrento, Italy, 2005.
4. R. Chen, M. Yang, X. Weng, B. Choi, B. He, and X. Li. Improving Large Graph Processing on Partitioned Graphs in the Cloud. In Proceedings of the 3rd ACM Symposium on Cloud Computing (SOCC), pages 3:1-3:13, San Jose, California, USA, 2012.
5. R. Cheng, J. Hong, A. Kyrola, Y. Miao, X. Weng, M. Wu, F. Yang, L. Zhou, F. Zhao, and E. Chen. Kineograph: Taking the Pulse of a Fast-Changing and Connected World. In Proceedings of the 7th ACM european conference on Computer Systems (EuroSys), pages 85-98, Bern, Switzerland, 2012.
6. J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In Proceedings of the 6th USENIX Symposium on Opearting Systems Design and Implementation (OSDI), pages 137-150, San Francisco, California, USA, 2004.
7. R. G. Gallager, P. A. Humblet, and P. M. Spira. A Distributed Algorithm for Minimum-Weight Spanning Trees. ACM Transactions on Programming Languages and Systems (TOPLAS), 5(1):66-77, 1983.
8. Giraph. Apache Incubator Giraph. http://incubator.apache.org/giraph, 2012.
9. GoldenOrb. A Cloud-based Open Source Project for Massive-Scale Graph Analysis. http://goldenorbos.org/, 2012.
10. J. E. Gonzalez, Y. Low, H. Gu, D. Bickson, and C. Guestrin. PowerGraph: Distributed Graph-Parallel Computation on Natural Graphs. In Proceedings of the 10th USENIX Symposium on Operating Systems Design and Implementation (OSDI), pages 17-30, Hollywood, California, USA, 2012.
11. B. Hendrickson and K. Devine. Dynamic Load Balancing in Computational Mechanics. Computer Methods in Applied Mechanics and Engineering, 184(2-4):485-500, 2000. (Pubitemid 30590642)
12. L.-Y. Ho, J.-J.Wu, and P. Liu. Distributed Graph Database for Large-Scale Social Computing. In Proceedings of the IEEE 5th International Conference in Cloud Computing (CLOUD), pages 455-462, Honolulu, Hawaii, USA, 2012.
13. U. Kang, C. E. Tsourakakis, and C. Faloutsos. PEGASUS: A Peta-Scale Graph Mining System Implementation and Observations. In Proceedings of the IEEE 9th International Conference on Data Mining (ICDM), pages 229-238, Miami, Florida, USA, 2009.
14. U. Kang, C. Tsourakakis, A. P. Appel, C. Faloutsos, and J. Leskovec. HADI: Fast Diameter Estimation and Mining in Massive Graphs with Hadoop. ACM Trasactions on Knowledge Discovery from Data (TKDD), 5(2):8:1-8:24, 2011.
15. G. Karypis and V. Kumar. Parallel Multilevel K-Way Partitioning Scheme for Irregular Graphs. In Proceedings of the ACM/IEEE Conference on Supercomputing (CDROM), pages 278-300, Pittsburgh, Pennsylvania, USA.
16. G. Karypis and V. Kumar. Multilevel k-way Partitioning Scheme for Irregular Graphs. Journal of Parallel and Distributed Computing, 48(1):96-129, 1998. (Pubitemid 128347296)
17. Z. Khayyat, K. Awara, H. Jamjoom, and P. Kalnis. Mizan: Optimizing Graph Mining in Large Parallel Systems. Technical report, King Abdullah University of Science and Technology, 2012.
18. E. Krepska, T. Kielmann, W. Fokkink, and H. Bal. HipG: Parallel Processing of Large-Scale Graphs. ACM SIGOPS Operating Systems Review, 45(2):3-13, 2011.
19. A. Kyrola, G. Blelloch, and C. Guestrin. GraphChi: Large-Scale Graph computation on Just a PC. In Proceedings of the 10th USENIX Symposium on Operating Systems Design and Implementation (OSDI), pages 31-46, Hollywood, California, USA, 2012.
20. LAW. The Laboratory for Web Algorithmics. http://law.di.unimi.it/index. php, 2012.
21. J. Leskovec, D. Chakrabarti, J. Kleinberg, C. Faloutsos, and Z. Ghahramani. Kronecker Graphs: An Approach to Modeling Networks. Journal of Machine Learning Research, 11:985-1042, 2010.
22. Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin, and J. Hellerstein. Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud. Proceedings of the VLDB Endowment (PVLDB), 5(8):716-727, 2012.
23. G. Malewicz, M. H. Austern, A. J. Bik, J. C. Dehnert, I. Horn, N. Leiser, and G. Czajkowski. Pregel: A System for Large-Scale Graph Processing. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, pages 135-146, Indianapolis, Indiana, USA.
24. L. Page, S. Brin, R. Motwani, and T. Winograd. The PageRank Citation Ranking: Bringing Order to the Web. http://ilpubs.stanford.edu:8090/422/, 2001.
25. J. M. Pujol, V. Erramilli, G. Siganos, X. Yang, N. Laoutaris, P. Chhabra, and P. Rodriguez. The Little Engine(s) That Could: Scaling Online Social Networks. IEEE/ACM Transactions on Networking, 20(4):1162-1175, 2012.
26. D. Rees. Foundations of Statistics. Chapman and Hall, London New York, 1987. ISBN 0412285606.
27. S. Salihoglu and J.Widom. GPS: A Graph Processing System. Technical report, Stanford University, 2012.
28. S. Seo, E. Yoon, J. Kim, S. Jin, J.-S. Kim, and S. Maeng. HAMA: An Efficient Matrix Computation with the MapReduce Framework. In Proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom), pages 721-726, Athens, Greece.
29. B. Shao, H. Wang, and Y. Li. Trinity: A Distributed Graph Engine on a Memory Cloud. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, New York, New York, USA.
30. L. G. Valiant. A Bridging Model for Parallel Computation. Communications of the ACM, 33:103-111, 1990.
31. W. Xue, J. Shi, and B. Yang. X-RIME: Cloud-Based Large Scale Social Network Analysis. In Proceedings of the 2010 IEEE International Conference on Services Computing (SCC), pages 506-513, Shanghai, China.