Accelerate large-scale iterative computation through asynchronous accumulative updates

ScienceCloud '12 - 3rd Workshop on Scientific Cloud Computing

Volumn , Issue , 2012, Pages 13-22

  Zhang, Yanfeng ^a,b   Gao, Qixin ^a   Gao, Lixin ^b   Wang, Cuirong ^a  

^a NORTHEASTERN UNIVERSITY   (China)

^b UNIVERSITY OF MASSACHUSETTS   (United States)

Author keywords

Asynchronous accumulative update; Iterative computation; Maiter

Indexed keywords

AMAZON EC2; ASYNCHRONOUS ACCUMULATIVE UPDATE; COMPUTATION MODEL; DATA MINING ALGORITHM; DATA SETS; DISTRIBUTED ENVIRONMENTS; DISTRIBUTED FRAMEWORK; ITERATIVE ALGORITHM; ITERATIVE COMPUTATION; ITERATIVE UPDATE; MAITER; MASSIVE DATA SETS;

ALGORITHMS; CLOUD COMPUTING; DATA MINING;

ITERATIVE METHODS;

EID: 84863903790     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2287036.2287041     Document Type: Conference Paper

References (28)

1. Amazon ec2. http://aws.amazon.com/ec2/.
2. Hadoop. http://hadoop.apache.org/.
3. Maiter project. http://code.google.com/p/maiter/.
4. Open mpi. http://www.open-mpi.org/.
5. Stanford dataset. http://snap.stanford.edu/data/.
6. S. Baluja, R. Seth, D. Sivakumar, Y. Jing, J. Yagnik, S. Kumar, D. Ravichandran, and M. Aly. Video suggestion and discovery for youtube: taking random walks through the view graph. In WWW '08, pages 895-904, 2008.
7. G. M. Baudet. Asynchronous iterative methods for multiprocessors. J. ACM, 25:226-244, April 1978.
8. D. P. Bertsekas. Distributed asynchronous computation of fixed points. Math. Programming, 27:107-120, 1983.
9. S. Brin and L. Page. The anatomy of a large-scale hypertextual web search engine. Comput. Netw. ISDN Syst., 30:107-117, April 1998.
10. Y. Bu, B. Howe, M. Balazinska, and D. M. Ernst. Haloop: Efficient iterative data processing on large clusters. In VLDB '10, 2010.
11. D. Chazan and W. Miranker. Chaotic relaxation. Linear Algebra and its Applications, 2(2):199 - 222, 1969.
12. J. Dean and S. Ghemawat. Mapreduce: simplified data processing on large clusters. In OSDI'04, pages 10-10, 2004.
13. J. Ekanayake, H. Li, B. Zhang, T. Gunarathne, S.-H. Bae, J. Qiu, and G. Fox. Twister: a runtime for iterative mapreduce. In MAPREDUCE '10, pages 810-818, 2010.
14. A. Frommer and D. B. Szyld. On asynchronous iterations. J. Comput. Appl. Math., 123:201-216, November 2000.
15. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: distributed data-parallel programs from sequential building blocks. In EuroSys '07, pages 59-72, 2007.
16. K. Kambatla, N. Rapolu, S. Jagannathan, and A. Grama. Asynchronous algorithms in mapreduce. In CLUSTER' 10, pages 245-254, 2010.
17. U. Kang, C. Tsourakakis, and C. Faloutsos. Pegasus: A peta-scale graph mining system implementation and observations. In ICDM '09, pages 229-238, 2009.
18. L. Katz. A new status index derived from sociometric analysis. Psychometrika, 18:39-43, 1953.
19. J. M. Kleinberg. Authoritative sources in a hyperlinked environment. J. ACM, 46:604-632, September 1999.
20. D. Liben-Nowell and J. Kleinberg. The link-prediction problem for social networks. J. Am. Soc. Inf. Sci. Technol., 58:1019-1031, May 2007.
21. Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin, and J. M. Hellerstein. Graphlab: A new framework for parallel machine learning. In UAI '10, 2010.
22. G. Malewicz, M. H. Austern, A. J. Bik, J. C. Dehnert, I. Horn, N. Leiser, and G. Czajkowski. Pregel: a system for large-scale graph processing. In SIGMOD '10, pages 135-146, 2010.
23. R. Power and J. Li. Piccolo: Building fast, distributed programs with partitioned tables. In OSDI'10, pages 1-14, 2010.
24. H. H. Song, T. W. Cho, V. Dave, Y. Zhang, and L. Qiu. Scalable proximity estimation and link prediction in online social networks. In IMC '09, pages 322-335, 2009.
25. M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica. Spark: cluster computing with working sets. In HotCloud'10, 2010.
26. Y. Zhang, Q. Gao, L. Gao, and C. Wang. imapreduce: A distributed computing framework for iterative computation. In DataCloud '11, pages 1112-1121, 2011.
27. Y. Zhang, Q. Gao, L. Gao, and C. Wang. Priter: A distributed framework for prioritized iterative computations. In SOCC '11, 2011.
28. Y. Zhang, Q. Gao, L. Gao, and C. Wang. Maiter: A message-passing distributed framework for accumulative iterative computation, http://rio.ecs.umass.edu/~yzhang/maiter-full.pdf. Technical report, 2012.