Peeking into the optimization of data flow programs with MapReduce-style UDFs

Proceedings - International Conference on Data Engineering

Volumn , Issue , 2013, Pages 1292-1295

  Hueske, Fabian ^a   Peters, Mathias ^b   Krettek, Aljoscha ^a   Ringwald, Matthias ^a   Tzoumas, Kostas ^a   Markl, Volker ^a   Freytag, Johann Christoph ^b  

^a TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

^b HUMBOLDT UNIVERSITY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

DATA PROCESSING SYSTEMS; EXECUTION STRATEGIES; EXTRACT INFORMATIONS; IMPERATIVE LANGUAGES; ORDERS OF MAGNITUDE; RELATIONAL OPERATOR; STATIC CODE ANALYSIS; USER DEFINED FUNCTIONS;

DATA TRANSFER; OPTIMIZATION;

DATA FLOW ANALYSIS;

EID: 84881319590     PISSN: 10844627     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDE.2013.6544927     Document Type: Conference Paper

References (18)

1. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins, "Pig latin: a not-so-foreign language for data processing," in SIGMOD Conference, 2008, pp. 1099-1110.
2. K. S. Beyer, V. Ercegovac, R. Gemulla, A. Balmin, M. Y. Eltabakh, C.-C. Kanne, F. Özcan, and E. J. Shekita, "Jaql: A scripting language for large scale semistructured data analysis," PVLDB, vol. 4, no. 12, pp. 1272-1283, 2011.
3. R. Chaiken, B. Jenkins, P.-Å. Larson, B. Ramsey, D. Shakib, S. Weaver, and J. Zhou, "Scope: easy and efficient parallel processing of massive data sets," PVLDB, vol. 1, no. 2, pp. 1265-1276, 2008.
4. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wyckoff, and R. Murthy, "Hive - a warehousing solution over a mapreduce framework," PVLDB, vol. 2, no. 2, pp. 1626-1629, 2009.
5. B. Chattopadhyay, L. Lin, W. Liu, S. Mittal, P. Aragonda, V. Lychagina, Y. Kwon, and M. Wong, "Tenzing a sql implementation on the mapreduce framework," PVLDB, vol. 4, no. 12, pp. 1318-1327, 2011.
6. E. Friedman, P. M. Pawlowski, and J. Cieslewicz, "Sql/mapreduce: A practical approach to self-describing, polymorphic, and parallelizable user-defined functions," PVLDB, vol. 2, no. 2, pp. 1402-1413, 2009.
7. http://www.greenplum.com/technology/mapreduce.
8. J. M. Hellerstein, "Optimization techniques for queries with expensive methods," ACM Trans. Database Syst., vol. 23, no. 2, pp. 113-157, 1998.
9. S. Chaudhuri and K. Shim, "Optimization of queries with user-defined predicates," ACM Trans. Database Syst., vol. 24, no. 2, pp. 177-228, 1999.
10. F. Hueske, M. Peters, M. Sax, A. Rheinländer, R. Bergmann, A. Krettek, and K. Tzoumas, "Opening the black boxes in data flow optimization," PVLDB, vol. 5, no. 11, pp. 1256-1267, 2012.
11. F. Hueske, A. Krettek, and K. Tzoumas, "Enabling rewrite optimizations of data flow programs through static code analysis," in XLDI Workshop, affiliated with ICFP, 2012.
12. D. Battré, S. Ewen, F. Hueske, O. Kao, V. Markl, and D. Warneke, "Nephele/pacts: a programming model and execution framework for web-scale analytical processing," in SoCC, 2010, pp. 119-130.
13. http://stratosphere.eu.
14. S. Chaudhuri and K. Shim, "Including group-by in query optimization," in VLDB, 1994, pp. 354-366.
15. J. Zhou, P.-Å. Larson, and R. Chaiken, "Incorporating partitioning and parallel plans into the scope optimizer," in ICDE, 2010, pp. 1060-1071.
16. J. Dean and S. Ghemawat, "Mapreduce: Simplified data processing on large clusters," in OSDI, 2004, pp. 137-150.
17. D. Warneke and O. Kao, "Nephele: efficient parallel data processing in the cloud," in SC-MTAGS, 2009.
18. A. V. Aho, M. S. Lam, R. Sethi, and J. D. Ullman, Compilers: Principles, Techniques and Tools. Pearson, 2006.