Optimizing analytic data flows for multiple execution engines

Proceedings of the ACM SIGMOD International Conference on Management of Data

Volumn , Issue , 2012, Pages 829-840

  Simitsis, Alkis ^a   Wilkinson, Kevin ^a   Castellanos, Malu ^a   Dayal, Umeshwar ^a  

^a HEWLETT PACKARD LABORATORIES   (United States)

Author keywords

bi; data flows; databases; map reduce; optimization

Indexed keywords

AD HOC PROCESS; DATA FLOW; EXECUTION ENGINE; FLOW OPTIMIZATION; FLOW TRANSFORMATIONS; MAP-REDUCE; MODEL DATA; MULTIPLE OBJECTIVES; SCRIPTING LANGUAGES; SINGLE OBJECTIVE; STRUCTURED DATA; UNSTRUCTURED DATA;

BISMUTH; DATA TRANSFER; DATABASE SYSTEMS; MARINE ENGINES;

OPTIMIZATION;

EID: 84862683313     PISSN: 07308078     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2213836.2213963     Document Type: Conference Paper

References (23)

1. A. Abouzeid et al. Hadoopdb: An architectural hybrid of mapreduce and dbms technologies for analytical workloads. PVLDB, 2(1):922-933, 2009.
2. D. Battré et al. Nephele/pacts: a programming model and execution framework for web-scale analytical processing. In SoCC, pages 119-130, 2010.
3. K. S. Beyer et al. Jaql: A scripting language for large scale semistructured data analysis. PVLDB, 4(12):1272-1283, 2011.
4. U. Dayal. Processing queries over generalization hierarchies in a multidatabase system. In VLDB, pages 342-353, 1983.
5. J. Dittrich et al. Hadoop++: Making a yellow elephant run like a cheetah. PVLDB, 3(1):518-529, 2010.
6. W. Du, R. Krishnamurthy, and M.-C. Shan. Query optimization in a heterogeneous dbms. In VLDB, pages 277-291, 1992.
7. C. Gupta et al. PQR: Predicting query execution times for autonomous workload management. In ICAC, pages 13-22, 2008.
8. W.-S. Han, W. Kwak, J. Lee, G. M. Lohman, and V. Markl. Parallelizing query optimization. PVLDB, 1(1):188-200, 2008.
9. Informatica. PowerCenter Pushdown Optimization Option Datasheet, 2011.
10. M. Isard et al. Dryad: distributed data-parallel programs from sequential building blocks. In EuroSys, pages 59-72, 2007.
11. D. Jiang, B. C. Ooi, L. Shi, and S. Wu. The performance of mapreduce: An in-depth study. PVLDB, 3(1):472-483, 2010.
12. H. Kllapi, E. Sitaridi, M. M. Tsangaris, and Y. E. Ioannidis. Schedule optimization for data processing ows on the cloud. In SIGMOD Conference, pages 289-300, 2011.
13. G. M. Lohman et al. Query Processing in R*. In Query Processing in Database Systems, pages 31-47. 1985.
14. D. G. Murray et al. CIEL: A universal execution engine for distributed data-ow computing. In USENIX NSDI, 2011.
15. C. Olston et al. Pig latin: a not-so-foreign language for data processing. In SIGMOD Conference, pages 1099-1110, 2008.
16. M. T. Roth et al. The Garlic project. In SIGMOD Conference, page 557, 1996.
17. T. K. Sellis. Multiple-query optimization. ACM Trans. Database Syst., 13(1):23-52, 1988.
18. A. Simitsis, P. Vassiliadis, and T. K. Sellis. Optimizing ETL processes in data warehouses. In ICDE, pages 564-575, 2005.
19. A. Simitsis, K. Wilkinson, M. Castellanos, and U. Dayal. QoX-driven ETL design: reducing the cost of ETL consulting engagements. In SIGMOD Conference, pages 953-960, 2009.
20. A. Simitsis, K. Wilkinson, U. Dayal, and M. Castellanos. Optimizing ETL work ows for fault-tolerance. In ICDE, 2010.
21. A. Thusoo et al. Hive - a petabyte scale data warehouse using hadoop. In ICDE, pages 996-1005, 2010.
22. A. Verma et al. ARIA: automatic resource inference and allocation for mapreduce environments. In ICAC, 2011.
23. A. Verma et al. Resource provisioning framework for mapreduce jobs with performance goals. In Middleware, 2011.