YSmart: Yet another SQL-to-MapReduce translator

Proceedings - International Conference on Distributed Computing Systems

Volumn , Issue , 2011, Pages 25-36

  Lee, Rubao ^a   Luo, Tian ^a   Huai, Yin ^a   Wang, Fusheng ^b   He, Yongqiang ^c   Zhang, Xiaodong ^a  

^a Ohio State University   (United States)

^b EMORY UNIVERSITY   (United States)

^c FACEBOOK   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AMAZON EC2; COMPLEX QUERIES; DATA ANALYTICS; FACEBOOK; I/O OPERATIONS; LARGE CLUSTERS; LOW SPEED; MAP-REDUCE; NETWORK TRANSFERS; QUERY EXECUTION; REDUNDANT COMPUTATION; SET OF RULES;

CLUSTER COMPUTING; SOCIAL NETWORKING (ONLINE);

PROGRAM TRANSLATORS;

EID: 80051874596     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDCS.2011.26     Document Type: Conference Paper

References (32)

1. L. Guo, E. Tan, S. Chen, X. Zhang, and Y. E. Zhao, "Analyzing patterns of user content generation in online social networks, " in KDD, 2009.
2. E. Friedman, P. M. Pawlowski, and J. Cieslewicz, "SQL/MapReduce: A practical approach to self-describing, polymorphic, and parallelizable user-defined functions, " PVLDB, vol. 2, no. 2, pp. 1402-1413, 2009.
3. D. J. DeWitt, E. Paulson, E. Robinson, J. F. Naughton, J. Royalty, S. Shankar, and A. Krioukov, "Clustera: an integrated computation and data management system, " PVLDB, vol. 1, no. 1, pp. 28-41, 2008.
4. J. Dean and S. Ghemawat, "MapReduce: Simplified data processing on large clusters, " in OSDI, 2004.
5. J. Tan, S. Kavulya, R. Gandhi, and P. Narasimhan, "Visual, log-based causal tracing for performance debugging of mapreduce systems, " in ICDCS, 2010, pp. 795-806.
6. M. Stonebraker, D. Abadi, D. J. DeWitt, S. Madden, E. Paulson, A. Pavlo, and A. Rasin, "MapReduce and parallel DBMSs: friends or foes?" Commun. ACM, vol. 53, no. 1, pp. 64-71, 2010.
7. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins, "Pig latin: a not-so-foreign language for data processing, " in SIGMOD, 2008.
8. A. Gates, O. Natkovich, S. Chopra, P. Kamath, S. Narayanam, C. Olston, B. Reed, S. Srinivasan, and U. Srivastava, "Building a highlevel dataflow system on top of MapReduce: The Pig experience, " PVLDB, 2009.
9. R. Chaiken, B. Jenkins, P.-A. Larson, B. Ramsey, D. Shakib, S. Weaver, and J. Zhou, "SCOPE: easy and efficient parallel processing of massive data sets, " PVLDB, vol. 1, no. 2, pp. 1265-1276, 2008.
10. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wyckoff, and R. Murthy, "Hive - a warehousing solution over a Map- Reduce framework, " PVLDB, vol. 2, no. 2, pp. 1626-1629, 2009.
11. A. Abouzeid, K. Bajda-Pawlikowski, D. J. Abadi, A. Rasin, and A. Silberschatz, "HadoopDB: An architectural hybrid of MapReduce and DBMS technologies for analytical workloads, " PVLDB, 2009.
12. Y. Cao, G. C. Das, C. Y. Chan, and K.-L. Tan, "Optimizing complex queries with multiple relation instances, " in SIGMOD Conference, 2008.
13. S. Shekhar and S. Chawla, Spatial Databases: A Tour. Prentice Hall, 2003.
14. Q. Zou, H. Wang, R. Soulé, M. Hirzel, H. Andrade, B. Gedik, and K.-L. Wu, "From a stream of relational queries to distributed stream processing, " PVLDB, vol. 3, no. 2, pp. 1394-1405, 2010.
15. H.-C. Yang, A. Dasdan, R.-L. Hsiao, and D. S. Parker, "Map-reducemerge: simplified relational data processing on large clusters, " in SIGMOD Conference, 2007, pp. 1029-1040. (Pubitemid 47630879)
16. A. Pavlo, E. Paulson, A. Rasin, D. J. Abadi, D. J. DeWitt, S. Madden, and M. Stonebraker, "A comparison of approaches to large-scale data analysis, " in SIGMOD Conference, 2009, pp. 165-178.
17. S. Chaudhuri, "An overview of query optimization in relational systems, " in PODS, 1998, pp. 34-43.
18. G. Graefe, "Query evaluation techniques for large databases, " ACM Comput. Surv., vol. 25, no. 2, pp. 73-170, 1993. (Pubitemid 23712749)
19. B. Hindman, A. Konwinski, M. Zaharia, and I. Stoica, "A common substrate for cluster computing, " in HotCloud, 2009.
20. W. Kim, "On optimizing an SQL-like nested query, " ACM Trans. Database Syst., vol. 7, no. 3, pp. 443-469, 1982.
21. S. Harizopoulos, V. Shkapenyuk, and A. Ailamaki, "QPipe: A simultaneously pipelined relational query engine, " in SIGMOD, 2005.
22. M. Zukowski, S. Héman, N. Nes, and P. A. Boncz, "Cooperative scans: Dynamic bandwidth sharing in a dbms, " in VLDB, 2007, pp. 723-734.
23. T. K. Sellis, "Multiple-query optimization, " ACM Trans. Database Syst., vol. 13, no. 1, pp. 23-52, 1988.
24. T. Condie, N. Conway, P. Alvaro, J. M. Hellerstein, K. Elmeleegy, and R. Sears, "MapReduce Online, " in NSDI, 2010.
25. D. Battré, S. Ewen, F. Hueske, O. Kao, V. Markl, and D. Warneke, "Nephele/pacts: a programming model and execution framework for web-scale analytical processing, " in ACM SoCC, 2010, pp. 119-130.
26. Y. Bu, B. Howe, M. Balazinska, and M. D. Ernst, "Haloop: Efficient iterative data processing on large clusters, " in VLDB, 2010.
27. J. Dittrich, J.-A. Quiané-Ruiz, A. Jindal, Y. Kargin, V. Setty, and J. Schad, "Hadoop++: Making a yellow elephant run like a cheetah (without it even noticing), " PVLDB, vol. 3, no. 1, pp. 518-529, 2010.
28. Y. He, R. Lee, Y. Huai, Z. Shao, N. Jain, X. Zhang, and Z. Xu, "Rcfile: A fast and space-efficient data placement structure in mapreduce-based warehouse systems, " in ICDE, 2011.
29. P. Agrawal, D. Kifer, and C. Olston, "Scheduling shared scans of large data files, " PVLDB, vol. 1, no. 1, pp. 958-969, 2008.
30. T. Nykiel, M. Potamias, C. Mishra, G. Kollios, and N. Koudas, "Mrshare: Sharing across multiple queries in mapreduce, " in VLDB, 2010.
31. F. N. Afrati and J. D. Ullman, "Optimizing joins in a Map-Reduce environment, " in EDBT, 2010.
32. J. Zhou, P.-Å. Larson, and R. Chaiken, "Incorporating partitioning and parallel plans into the scope optimizer, " in ICDE, 2010, pp. 1060-1071.