The HiBench benchmark suite: Characterization of the MapReduce-based data analysis

Proceedings - International Conference on Data Engineering

Volumn , Issue , 2010, Pages 41-51

  Huang, Shengsheng ^a   Huang, Jie ^a   Dai, Jinquan ^a   Xie, Tao ^a   Huang, Bo ^a  

^a INTEL CORPORATION   (China)

Author keywords

[No Author keywords available]

Indexed keywords

BENCHMARK SUITES; DATA ACCESS PATTERNS; DATA ANALYSIS; JOB RUNNING TIME; OPEN SOURCE IMPLEMENTATION; REAL-WORLD; SYSTEM RESOURCES;

BENCHMARKING;

DATA HANDLING;

EID: 77952721751     PISSN: 10844627     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDEW.2010.5452747     Document Type: Conference Paper

References (33)

1. J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," USENIX OSDI, December, 2004.
2. Hadoop homepage. http://hadoop.apache.org/
3. Pig homepage. http://hadoop.apache.org/pig/
4. Hive homepage. http://hadoop.apache.org/hive/
5. Mahout homepage. http://lucene.apache.org/mahout/
6. HBase homepage. http://hadoop.apache.org/hbase/
7. GridMix program. Available in Hadoop source distribution: src/benchmarks/gridmix.
8. A. Pavlo, A. Rasin, S. Madden, M. Stonebraker, D. DeWitt, E. Paulson, L. Shrinivas, and D. J. Abadi. "A Comparison of Approaches to Large-Scale Data Analysis", SIGMOD, June, 2009
9. Y. Jia and Z. Shao. "A Benchmark for Hive, PIG and Hadoop", Available: http://issues.apache.org/jira/browse/HIVE-396
10. Sort program. Available in Hadoop source distribution: src/examples/org/apache/hadoop/examples/sort
11. Matei Zaharia, Andy Konwinski, Anthony D. Joseph, Randy Katz, and Ion Stoica. "Improving MapReduce Performance in Heterogeneous Environments", OSDI'08, December, 2008.
12. HDFS homepage. http://hadoop.apache.org/hdfs/
13. th ACM Symposium on Operating Systems Principles, October, 2003.
14. TeraSort. http://sortbenchmark.org/
15. Hadoop TeraSort program. Available in Hadoop source distribution since 0.19 version: src/examples/org/apache/hadoop/examples/terasort
16. TeraGen program. Available in Hadoop source distribution since 0.19 version: src/examples/org/apache/hadoop/examples/terasort/TeraGen
17. O. O'Malley and A. C. Murthy, "Winning a 60 Second Dash with a Yellow Elephant". Available: http://sortbenchmark.org/Yahoo2009.pdf
18. "Sorting 1PB with MapReduce", Available: http://googleblog. blogspot.com/2008/11/sorting-1pb-with-mapreduce.html
19. DFSIO program. Available in Hadoop source distribution: src/test/org/apache/hadoop/fs/TestDFSIO
20. Nutch homepage. http://lucene.apache.org/nutch/
21. WordCount program. Available in Hadoop source distribution: src/examples/org/apache/hadoop/ examples/WordCount
22. Lucene homepage. http://lucene.apache.org
23. SmarFrog project homepage. http://www.smartfrog.org
24. P. Castagna, "Having fun with PageRank and MapReduce," Hadoop User Group UK talk. Available: http://static.last.fm/johan/huguk-20090414/paolo- castagna-pagerank.pdf
25. H. Haselgrove, "Using the Wikipedia page-to-page link database," Available: http://users.on.net/~henry/home/wikipedia.htm
26. Mahout Naïve Bayesian. http://cwiki.apache.org/MAHOUT/naivebayes. html
27. N-Gram. http://en.wikipedia.org/wiki/N-gram
28. Tf-Idf. http://en.wikipedia.org/wiki/Tf%E2%80%93idf
29. Wikipedia Dump. http://en.wikipedia.org/wiki/index.php?curid=68321
30. Mahout K-means. http://cwiki.apache.org/MAHOUT/k-means.html
31. Yahoo! distribution of Hadoop. Avai lable: http://developer.yahoo.com/ hadoop/distribution/
32. Nurcan Coskun, "Optimizing Hadoop Deployments", Hadoop World 2009 Presentation.
33. Hadoop-5191. Available: http://issues.apache.org/jira/browse/HADOOP-5191.