Shark: SQL and rich analytics at scale

Proceedings of the ACM SIGMOD International Conference on Management of Data

Volumn , Issue , 2013, Pages 13-24

  Xin, Reynold S ^a   Rosen, Josh ^a   Zaharia, Matei ^a   Franklin, Michael J ^a   Shenker, Scott ^a   Stoica, Ion ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Data Warehouse; Databases; Hadoop; Machine Learning; Shark; Spark

Indexed keywords

COLUMN-ORIENTED; DATA ANALYSIS SYSTEM; DISTRIBUTED MEMORY; EXECUTION ENGINE; FAULT TOLERANCE PROPERTY; HADOOP; LEARNING PROGRAMS; SHARK;

DATA WAREHOUSES; DATABASE SYSTEMS; DIGITAL STORAGE; ELECTRIC SPARKS; FAULT TOLERANCE; LEARNING SYSTEMS; QUERY LANGUAGES;

ENGINES;

EID: 84880533620     PISSN: 07308078     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2463676.2465288     Document Type: Conference Paper

References (39)

1. https://github.com/cloudera/impala.
2. http://hadoop.apache.org/.
3. http://aws.amazon.com/elasticmapreduce/.
4. A. Abouzeid et al. Hadoopdb: an architectural hybrid of mapreduce and dbms technologies for analytical workloads. VLDB, 2009.
5. S. Agarwal et al. Re-optimizing data-parallel computing. In NSDI'12.
6. G. Ananthanarayanan et al. Pacman: Coordinated memory caching for parallel jobs. In NSDI, 2012.
7. R. Avnur and J. M. Hellerstein. Eddies: continuously adaptive query processing. In SIGMOD, 2000.
8. S. Babu. Towards automatic optimization of mapreduce programs. In SoCC'10.
9. A. Behm et al. Asterix: towards a scalable, semistructured data platform for evolving-world models. Distributed and Parallel Databases, 29(3):185-216, 2011.
10. V. Borkar et al. Hyracks: A flexible and extensible foundation for data-intensive computing. In ICDE'11.
11. Y. Bu et al. HaLoop: efficient iterative data processing on large clusters. Proc. VLDB Endow., 2010.
12. R. Chaiken et al. Scope: easy and efficient parallel processing of massive data sets. VLDB, 2008.
13. B. Chattopadhyay, et al Tenzing a sql implementation on the mapreduce framework. PVLDB, 4(12):1318-1327, 2011.
14. S. Chen. Cheetah: a high performance, custom data warehouse on top of mapreduce. VLDB, 2010.
15. C. Chu et al. Map-reduce for machine learning on multicore. Advances in neural information processing systems, 19:281, 2007.
16. J. Cohen, B. Dolan, M. Dunlap, J. Hellerstein, and C. Welton. Mad skills: new analysis practices for big data. VLDB, 2009.
17. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. In OSDI, 2004.
18. X. Feng et al. Towards a unified architecture for in-rdbms analytics. In SIGMOD, 2012.
19. B. Guffler et al. Handling data skew in mapreduce. In CLOSER'11.
20. A. Hall et al. Processing a trillion cells per mouse click. VLDB.
21. B. Hindman et al. Mesos: A platform for fine-grained resource sharing in the data center. In NSDI'11.
22. M. Isard et al. Dryad: distributed data-parallel programs from sequential building blocks. SIGOPS, 2007.
23. M. Isard et al. Quincy: Fair scheduling for distributed computing clusters. In SOSP '09, 2009.
24. M. Isard and Y. Yu. Distributed data-parallel computing using a high-level programming language. In SIGMOD, 2009.
25. N. Kabra and D. J. DeWitt. Efficient mid-query re-optimization of sub-optimal query execution plans. In SIGMOD, 1998.
26. Y. Kwon et al. Skewtune: mitigating skew in mapreduce applications. In SIGMOD '12, 2012.
27. Y. Low et al. Distributed graphlab: a framework for machine learning and data mining in the cloud. VLDB, 2012.
28. G. Malewicz et al. Pregel: a system for large-scale graph processing. In SIGMOD, 2010.
29. S. Melnik et al. Dremel: interactive analysis of web-scale datasets. Proc. VLDB Endow., 3:330-339, Sept 2010.
30. K. Ousterhout et al. The case for tiny tasks in compute clusters. In HotOS'13.
31. A. Pavlo et al. A comparison of approaches to large-scale data analysis. In SIGMOD, 2009.
32. M. Stonebraker et al. C-store: a column-oriented dbms. In VLDB'05.
33. M. Stonebraker et al. Mapreduce and parallel dbmss: friends or foes? Commun. ACM.
34. A. Thusoo et al. Hive - a petabyte scale data warehouse using hadoop. In ICDE, 2010.
35. Transaction Processing Performance Council. TPC BENCHMARK H.
36. T. Urhan, M. J. Franklin, and L. Amsaleg. Cost-based query scrambling for initial delays. In SIGMOD, 1998.
37. C. Yang et al. Osprey: Implementing mapreduce-style fault tolerance in a shared-nothing distributed database. In ICDE, 2010.
38. M. Zaharia et al. Delay scheduling: A simple technique for achieving locality and fairness in cluster scheduling. In EuroSys 10, 2010.
39. M. Zaharia et al. Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. NSDI, 2012.