Benchmarking performance for migrating a relational application to a parallel implementation

Future Generation Computer Systems

Volumn 63, Issue , 2016, Pages 148-156

  Gadiraju, Krishna Karthik ^a   Verma, Manik ^a   Davis, Karen C ^a   Talaga, Paul G ^a  

^a 705 Engineering Research Center   (United States)

Author keywords

Benchmarking; Big data; Hadoop; Hive; Queries; SQL

Indexed keywords

BIG DATA; DATA WAREHOUSES; QUERY LANGUAGES; QUERY PROCESSING; RELATIONAL DATABASE SYSTEMS;

DATA ENVIRONMENT; HADOOP; HIVE; PARALLEL IMPLEMENTATIONS; QUERIES; QUERY EXECUTION; RELATIONAL DATABASE; SERIAL EXECUTION;

BENCHMARKING;

EID: 84961310427     PISSN: 0167739X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.future.2015.12.015     Document Type: Article

References (28)

1. [1] Baru, Chaitanya, Bhandarkar, Milind, Nambiar, Raghunath, Poess, Meikel, Rabl, Tilmann, Setting the direction for big data benchmark standards. Selected Topics in Performance Evaluation and Benchmarking, 2013, Springer, Berlin, Heidelberg, 197–208.
2. [2] Apache Hive vs MySQL—What are the key differences? Matthew Rathbone. http://blog.matthewrathbone.com/2015/12/08/hive-vs-mysql.html (posted 12.8.15, accessed 12.14.15).
3. [3] Hive vs. RDMS, Siva, http://hadooptutorial.info/hive-vs-rdbms/ (posted 8.1.14, accessed 12.14.15).
4. [4] Schema-on-Read vs Schema-on-Write, Joe Pasqua, posted 2.17.15, http://www.marklogic.com/blog/schema-on-read-vs-schema-on-write/ (accessed 12.14.15).
5. [5] TPC-DS benchmarking standard. http://www.tpc.org/tpcds/spec/tpcds_1.1.0.pdf.
6. [6] Using MySQL as a Hive backend database, http://mapredit.blogspot.com/2011/08/using-mysql-as-hive-backend-database.html (accessed 12.14.15).
7. [7] AdminManual MetastoreAdmin. https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin (modified 11.11.15, accessed 12.14.15).
8. [8] Dean, Jeffrey, Ghemawat, Sanjay, MapReduce: simplified data processing on large clusters. Commun. ACM 51:1 (2008), 107–113.
9. [9] Thusoo, Ashish, Sarma, Joydeep Sen, Jain, Namit, Shao, Zheng, Chakka, Prasad, Anthony, Suresh, Liu, Hao, Wyckoff, Pete, Murthy, Raghotham, Hive: a warehousing solution over a map-reduce framework. Proc. VLDB Endow. 2:2 (2009), 1626–1629.
10. [10] Hortonworks HDP 1.3.2, http://hortonworks.com/products/hdp/hdp-1-3/#overview.
11. [11] Sort program Available in Hadoop source distribution: https://github.com/facebookarchive/hadoop-20/blob/master/src/examples/org/apache/hadoop/examples/Sort.java.
12. [12] Hadoop TeraSort program. Available in Hadoop source distribution since 0.19 version: src/examples/org/apache/hadoop/examples/terasort.
13. [13] GridMix program. Available in Hadoop source distribution: src/benchmarks/gridmix.
14. [14] Huang, Shengsheng, Huang, Jie, Dai, Jinquan, Xie, Tao, Huang, Bo, The HiBench benchmark suite: Characterization of the MapReduce-based data analysis. Proceedings of the Data Engineering Workshops (ICDEW), 2010, IEEE, 41–51.
15. [15] Ghazal, Ahmad, Rabl, Tilmann, Hu, Minqing, Raab, Francois, Poess, Meikel, Crolotte, Alain, Jacobsen, Hans-Arno, BigBench: towards an industry standard benchmark for big data analytics. Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, 2013, ACM, 1197–1208.
16. [16] Hive Performance Benchmark. Available at: https://issues.apache.org/jira/browse/hive-396.
17. [17] Hortonworks Stinger Initiative, http://hortonworks.com/labs/stinger/.
18. [18] DSGen v1.1.0, data generation tool for TPC-DS, http://www.tpc.org/tpcds/.
19. [19] Tom. White, Hadoop: the Definitive Guide. O'Reilly, 2012.
20. [20] Nambiar, Raghunath Othayoth, Poess, Meikel, The making of TPC-DS. Proceedings of the 32nd International Conference on Very Large Data Bases, 2006, VLDB Endowment, 1049–1058.
21. [21] Shi, Yingjie, Meng, Xiaofeng, Zhao, Jing, Hu, Xiangmei, Liu, Bingbing, Wang, Haiping, Benchmarking cloud-based data management systems. Proceedings of the Second International Workshop on Cloud Data Management, 2010, ACM, 47–54.
22. [22] Pavlo, Andrew, Paulson, Erik, Rasin, Alexander, Abadi, Daniel J., DeWitt, David J., Madden, Samuel, Stonebraker, Michael, A comparison of approaches to large-scale data analysis. Proceedings of the 2009 ACM SIGMOD International Conference on Management of Data, 2009, ACM, 165–178.
23. [23] Yuntao Jia, Running the TPC-H benchmark on Hive, https://issues.apache.org/jira/secure/attachment/12416257/TPC-H_on_Hive_2009-08-11.pdf.
24. [24] Niketan Pansare, Zhuhua Cai, Using Hive to perform medium-scale data analysis, 2010. http://www.cs.rice.edu/~np6/Papers/HiveProjectReport.pdf.
25. [25] K.K. Gadiraju, P.G. Talaga, K.C. Davis, Benchmarking performance for migrating a relational application to a parallel implementation, in: Proceedings of the Modeling and Management of Big Data Workshop, MoBiD, 33nd International Conference on Conceptual Modeling, ER, 2014, pp. 55–64.
26. [26] Hortonworks Stinger Initiative, Apache Hive 0.13 Performance Benchmarks Query Times in Hive 0.13 v. Hive 0.10, June 2014.
27. [27] Gruenheid, Anja, Omiecinski, Edward, Mark, Leo, Query optimization using column statistics in hive. Proceedings of the 15th Symposium on International Database Engineering & Applications, 2011, ACM, 97–105.
28. [28] Herodotos Herodotou, Harold Lim, Gang Luo, Nedyalko Borisov, Liang Dong, Fatma Bilgen Cetin, Shivnath Babu, Starfish: a self-tuning system for big data analytics, in: Proceedings of the Fifth Biennial Conference on Innovative Data Systems Research, CIDR, 2011, pp. 261–272.