iMapReduce: A distributed computing framework for iterative computation

IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum

Volumn , Issue , 2011, Pages 1112-1121

  Zhang, Yanfeng ^a   Gao, Qinxin ^a   Gao, Lixin ^b   Wang, Cuirong ^a  

^a NORTHEASTERN UNIVERSITY   (China)

^b UNIVERSITY OF MASSACHUSETTS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ITERATIVE ALGORITHM; ITERATIVE COMPUTATION; ITERATIVE OPERATION; ITERATIVE PROCESS; ITERATIVE PROCESSING; LARGE CLUSTERS; MAP-REDUCE; RELATIONAL DATA; SOCIAL NETWORK ANALYSIS;

ALGORITHMS; CLUSTER COMPUTING; COMPUTER SOFTWARE; DATA HANDLING; DISTRIBUTED PARAMETER NETWORKS; SOCIAL NETWORKING (ONLINE); SOCIAL SCIENCES COMPUTING;

ITERATIVE METHODS;

EID: 83455229790     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPS.2011.260     Document Type: Conference Paper

References (23)

1. J. Dean and S. Ghemawat, "Mapreduce: simplified data processing on large clusters," in OSDI'04: Proceedings of the 6th conference on Symposium on Opearting Systems Design and Implementation, 2004.
2. H. H. Song, T. W. Cho, V. Dave, Y. Zhang, and L. Qiu, "Scalable proximity estimation and link prediction in online social networks," in IMC '09: Proceedings of the 9th ACM SIGCOMM conference on Internet measurement conference, 2009, pp. 322-335.
3. S. Baluja, R. Seth, D. Sivakumar, Y. Jing, J. Yagnik, S. Kumar, D. Ravichandran, and M. Aly, "Video suggestion and discovery for youtube: taking random walks through the view graph," in WWW '08: Proceeding of the 17th international conference on World Wide Web, 2008, pp. 895-904.
4. M. Zaharia, A. Konwinski, A. D. Joseph, R. Katz, and I. Stoica, "Improving mapreduce performance in heterogeneous environments," in OSDI '08: Proceedings of the 8th USENIX conference on Operating systems design and implementation, 2008, pp. 29-42.
5. Hadoop mapreduce. [Online]. Available: http://hadoop.apache.org/
6. Amazon ec2. [Online]. Available: http://aws.amazon.com/ec2/
7. C. Wilson, B. Boe, A. Sala, K. P. Puttaswamy, and B. Y. Zhao, "User interactions in social networks and their implications," in EuroSys '09: Proceedings of the 4th ACM European conference on Computer systems, 2009, pp. 205-218.
8. A. Clauset, C. R. Shalizi, and M. E. J. Newman, "Power-law distributions in empirical data," SIAM Rev., vol. 51, no. 4, pp. 661-703, 2009.
9. J. Leskovec, K. J. Lang, A. Dasgupta, and M. W. Mahoney, "Community structure in large networks: Natural cluster sizes and the absence of large well-defined clusters," Tech. Rep., Oct 2008. [Online]. Available: http://arxiv.org/abs/0810.1355
10. Last.fm web services. [Online]. Available: http://www.last.fm/api/
11. T. Condie, N. Conway, P. Alvaro, J. M. Hellerstein, K. Elmeleegy, and R. Sears, "Mapreduce online," in Proceedings of the 7th USENIX conference on Networked systems design and implementation, ser. NSDI'10, 2010, pp. 21-21.
12. U. Kang, C. Tsourakakis, and C. Faloutsos, "Pegasus: A peta-scale graph mining system implementation and observations," in ICDM '09: Proceedings of the 9th IEEE International Conference on Data Mining, 2009, pp. 229-238.
13. D. G. Murray and S. Hand, "Scripting the cloud with skywriting," in HotCloud '10: Proceedings of the 2nd USENIX conference on Hot topics in cloud computing, 2010, pp. 12-12.
14. M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica, "Spark: Cluster Computing with Working Sets," in HotCloud '10: Proceedings of the 2nd USENIX conference on Hot topics in cloud computing, 2010.
15. G. Malewicz, M. H. Austern, A. J. Bik, J. C. Dehnert, I. Horn, N. Leiser, and G. Czajkowski, "Pregel: a system for large-scale graph processing," in PODC '09: Proceedings of the 28th ACM symposium on Principles of distributed computing, pp. 6-6.
16. P. Russell and L. Jinyang, "Piccolo: Building fast, distributed programs with partitioned tables," in OSDI'10: Proceedings of the 9th conference on Symposium on Opearting Systems Design and Implementation, 2010.
17. P. Daniel and D. Frank, "Large-scale incremental processing using distributed transactions and notifications," in OSDI'10: Proceedings of the 9th conference on Symposium on Opearting Systems Design and Implementation, 2010.
18. K. Karthik, R. Naresh, J. Suresh, and G. Ananth, "Asynchronous algorithms in mapreduce," in IEEE Cluster'10: Proceedings of the 2010 IEEE International Conference on Cluster Computing, 2010.
19. J. Lin and M. Schatz, "Design patterns for efficient graph algorithms in mapreduce," in MLG '10: Proceedings of the Eighth Workshop on Mining and Learning with Graphs, 2010, pp. 78-85.
20. J. Ekanayake, H. Li, B. Zhang, T. Gunarathne, S.-H. Bae, J. Qiu, and G. Fox, "Twister: a runtime for iterative mapreduce," in MAPREDUCE '10: Proceedings of the 1st International Workshop on MapReduce and its Applications, 2010, pp. 810-818.
21. B. Yingyi, H. Bill, B. Magdalena, and D. E. Michael, "Haloop: Efficient iterative data processing on large clusters," in VLDB '2010: Proceedings of the 36th international conference on Very Large Data Bases. VLDB Endowment, 2010.
22. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly, "Dryad: distributed data-parallel programs from sequential building blocks," in EuroSys '07: Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007, 2007, pp. 59-72.
23. Y. Yu, M. Isard, D. Fetterly, M. Budiu, lfar Erlingsson, P. K. Gunda, and J. Currey, "Dryadlinq: A system for general-purpose distributed data-parallel computing using a high-level language," in OSDI '08: Proceedings of the 8th conference on Symposium on Opearting Systems Design and Implementation, pp. 1-14.