Spinning fast iterative data flows

Proceedings of the VLDB Endowment

Volumn 5, Issue 11, 2012, Pages 1268-1279

  Ewen, Stephan ^a   Tzoumas, Kostas ^a   Kaufmann, Moritz ^a   Mark, Volker ^a  

^a TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

BIG DATA; DATA FLOW ANALYSIS; LEARNING ALGORITHMS; LEARNING SYSTEMS; MESSAGE PASSING;

ITERATIVE ALGORITHM; OPTIMIZERS; ORDERS OF MAGNITUDE; PARALLEL DATAFLOW; PROGRAMMING MODELS; PROTOTYPICAL IMPLEMENTATION; SHARED MEMORY; SPECIALIZED SYSTEMS;

ITERATIVE METHODS;

EID: 84870501280     PISSN: None     EISSN: 21508097     Source Type: Conference Proceeding    
DOI: 10.14778/2350229.2350245     Document Type: Article

References (37)

1. L. Afanasiev, T. Grust, M. Marx, J. Rittinger, and J. Teubner. Recursion in XQuery: Put your distributivity safety belt on. In EDBT, pp. 345-356, 2009.
2. F. N. Afrati, V. R. Borkar, M. J. Carey, N. Polyzotis, and J. D. Ullman. Map-reduce extensions and recursive queries. In EDBT, pp. 1-8, 2011.
3. Apache Giraph. http://incubator.apache.org/giraph/.
4. Apache Hadoop. http://hadoop.apache.org.
5. Apache Mahout. http://mahout.apache.org.
6. F. Bancilhon, R. Ramakrishnan. An amateur's introduction to recursive query processing strategies. In SIGMOD, pp. 16-52, 1986.
7. D. Battré, S. Ewen, F. Hueske, O. Kao, V. Markl, and D. Warneke. Nephele/PACTs: A programming model and execution framework for web-scale analytical processing. In SoCC, pp. 119-130, 2010.
8. C. Beeri and R. Ramakrishnan. On the power of Magic. In PODS, pp. 269-284, 1987.
9. A. Behm, V. R. Borkar, M. J. Carey, R. Grover, C. Li, N. Onose, R. Vernica, A. Deutsch, Y. Papakonstantinou, and V. J. Tsotras. ASTERIX: Towards a scalable, semistructured data platform for evolving-world models. Distributed and Parallel Databases, 29(3):185-216, 2011.
10. P. Boldi and S. Vigna. The WebGraph framework I: Compression techniques. In WWW, pp. 595-601, 2004.
11. V. R. Borkar, M. J. Carey, R. Grover, N. Onose, and R. Vernica. Hyracks: A flexible and extensible foundation for data-intensive computing. In ICDE, pp. 1151-1162, 2011.
12. Y. Bu, V. R. Borkar, M. J. Carey, J. Rosen, N. Polyzotis, T. Condie, M. Weimer, R. Ramakrishnan. Scaling Datalog for machine learning on Big Data. In CoRR, abs/1203.0160, 2012.
13. Y. Bu, B. Howe, M. Balazinska, and M. D. Ernst. HaLoop: Efficient iterative data processing on large clusters. PVLDB, 3(1):285-296, 2010.
14. J. Cai and R. Paige. Program derivation by fixed point computation. Sci. Comput. Program., 11(3):197-261, 1989.
15. R. Chaiken, B. Jenkins, P.-Å. Larson, B. Ramsey, D. Shakib, S. Weaver, and J. Zhou. SCOPE: Easy and efficient parallel processing of massive data sets. PVLDB, 1(2):1265-1276, 2008.
16. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. In OSDI, pp. 137-150, 2004.
17. D. J. DeWitt, R. H. Gerber, G. Graefe, M. L. Heytens, K. B. Kumar, and M. Muralikrishna. GAMMA - A high performance dataflow database machine. In VLDB, pp. 228-237, 1986.
18. J. Ekanayake, H. Li, B. Zhang, T. Gunarathne, S.-H. Bae, J. Qiu, and G. Fox. Twister: A runtime for iterative MapReduce. In HPDC, pp. 810-818, 2010.
19. S. Fushimi, M. Kitsuregawa, and H. Tanaka. An overview of the system software of a parallel relational database machine GRACE. In VLDB, pp. 209-219, 1986.
20. G. Graefe and W. J. McKenna. The Volcano optimizer generator: Extensibility and efficient search. In ICDE, pp. 209-218, 1993.
21. U. Güntzer, W. Kießling, and R. Bayer. On the evaluation of recursion in (deductive) database systems by efficient differential fixpoint iteration. In ICDE, pp. 120-129, 1987.
22. F. Hueske, M. Peters, M. J. Sax, A. Rheinländer, R. Bergmann, A. Krettek, K. Tzoumas. Opening the black boxes in data flow optimization. PVLDB, Vol. 5, 2012 (to appear).
23. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. In EuroSys, pp. 59-72, 2007.
24. J. B. Kam and J. D. Ullman. Monotone data flow analysis frameworks. Acta Inf., Vol. 7, No. 3: pp. 305-317, 1977.
25. S. Kamvar, T. Haveliwala, and G. Golub. Adaptive methods for the computation of PageRank. Technical Report 2003-26, Stanford InfoLab, 2003.
26. U. Kang, C. E. Tsourakakis, and C. Faloutsos. Pegasus: A peta-scale graph mining system. In ICDM, pp. 229-238, 2009.
27. T-H. Lai, Y-C. Tseng, X. Dong. A more efficient message-optimal algorithm for distributed termination detection. In IPPS, pp. 646-649, 1992
28. Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin, and J. M. Hellerstein. GraphLab: A new framework for parallel machine learning. In UAI, pp. 340-349, 2010.
29. G. Malewicz, M. H. Austern, A. J. C. Bik, J. C. Dehnert, I. Horn, N. Leiser, and G. Czajkowski. Pregel: A system for large-scale graph processing. In SIGMOD, pp. 135-146, 2010.
30. D. G. Murray, M. Schwarzkopf, C. Smowton, S. Smith, A. Madhavapeddy, and S. Hand. Ciel: A universal execution engine for distributed data-flow computing. In NSDI, 2011.
31. L. Page, S. Brin, R. Motwani, and T. Winograd. The PageRank citation ranking: Bringing order to the web. Technical Report No. 1999-66, Stanford InfoLab, 1999.
32. D. A. Schneider and D. J. DeWitt. A performance evaluation of four parallel join algorithms in a shared-nothing multiprocessor environment. In SIGMOD, pp. 110-121, 1989.
33. P. G. Selinger, M. M. Astrahan, D. D. Chamberlin, R. A. Lorie, and T. G. Price. Access path selection in a relational database management system. In SIGMOD, pp. 23-34, 1979.
34. L. G. Valiant. General purpose parallel architectures. In Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity (A), pp. 943-972, 1990.
35. M. Weimer, T. Condie, and R. Ramakrishnan. Machine learning in ScalOps, a higher order cloud computing language. In NIPS BigLearn, Vol. 9, pp. 389-396, 2011.
36. M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica. Spark: Cluster computing with working sets. In HotCloud, pp. 1-7, 2010.
37. M. Zinkevich, M. Weimer, A. J. Smola, and L. Li. Parallelized stochastic gradient descent. In NIPS, pp. 2595-2603, 2010