Benchmarking MapReduce implementations for application usage scenarios

Proceedings - 2011 12th IEEE/ACM International Conference on Grid Computing, Grid 2011

Volumn , Issue , 2011, Pages 90-97

  Fadika, Zacharia ^a   Dede, Elif ^a   Govindaraju, Madhusudhan ^a   Ramakrishnan, Lavanya ^b  

^a State University of New York   (United States)

^b LAWRENCE BERKELEY NATIONAL LABORATORY   (United States)

Author keywords

Benchmarking; Hadoop; LEMO MR; MapReduce

Indexed keywords

BENCHMARK SUITES; CURRENT PERFORMANCE; DATA PRODUCTION; DATA-INTENSIVE COMPUTING; DESIGN DECISIONS; HADOOP; LEMO-MR; MAP-REDUCE; MIDDLEWARE DEVELOPERS; PERFORMANCE ANALYSIS; SCALABLE MODEL; USAGE SCENARIOS;

FAULT TOLERANCE; GRID COMPUTING; MIDDLEWARE;

BENCHMARKING;

EID: 83455264036     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/Grid.2011.21     Document Type: Conference Paper

References (25)

1. M. P. I. Forum, "Mpi: A message-passing interface standard," 1994.
2. Apache Hadoop. [Online]. Available: http://hadoop.apache.org
3. J. Ekanayake, H. Li, B. Zhang, T. Gunarathne, S.-H. Bae, J. Qiu, and G. Fox, "Twister: a runtime for iterative mapreduce," in HPDC, 2010, pp. 810-818.
4. Z. Fadika and M. Govindaraju, "Lemo-mr: Low overhead and elastic mapreduce implementation optimized for memory and cpu-intensive applications," Cloud Computing Technology and Science, IEEE International Conference on, vol. 0, pp. 1-8, 2010.
5. Octopy, Easy MapReduce for Python. [Online]. Available: http: //code.google.com/p/octopy/
6. Skynet. [Online]. Available: http://rubyforge.org/projects/skynet
7. AMAZON, "Amazon Elastic MapReduce." [Online]. Available: http: //aws.amazon.com/ec2
8. Z. Fadika and M. Govindaraju, "Delma: Dynamically elastic mapreduce framework for cpu-intensive applications," in CCGRID, 2011, pp. 454-463.
9. B. He, W. Fang, Q. Luo, N. K. Govindaraju, and T. Wang, "Mars: a map-reduce framework on graphics processors," in PACT '08: Proceedings of the 17th international conference on Parallel architectures and compilation techniques. New York, NY, USA: ACM, 2008, pp. 260-269.
10. J. Ekanayake, S. Pallickara, and G. Fox, "MapReduce for Data Intensive Scientific Analyses," in IEEE Fourth International Conference on eScience, December 2008, pp. 277-284.
11. Y. Shan, B. Wang, J. Yan, Y. Wang, N. Xu, and H. Yang, "Fpmr: Mapreduce framework on fpga," in FPGA '10: Proceedings of the 18th annual ACM/SIGDA international symposium on Field programmable gate arrays. New York, NY, USA: ACM, 2010, pp. 93-102.
12. D. Borthakur, "The hadoop distributed file system: Architecture and design," Hadoop Project, 2007.
13. S. Ghemawat, H. Gobioff, and S.-T. Leung, "The google file system," in Proceedings of the nineteenth ACM symposium on Operating systems principles, ser. SOSP '03. New York, NY, USA: ACM, 2003, pp. 29-43. [Online]. Available: http://doi.acm.org/10.1145/945445.945450
14. G. Mackey, S. Sehrish, J. Lopez, J. Bent, S. Habib, and J. Wang, "Introducing mapreduce to high end computing in Petascale Data," in Storage Workshop Held in conjunction with SC08.
15. J. Xie, S. Yin, X. Ruan, Z. Ding, Y. Tian, J. Majors, A. Manzanares, and X. Qin, "Improving mapreduce performance through data placement in heterogeneous hadoop clusters," in IPDPS Workshops, 2010, pp. 1-9.
16. M. Zaharia, A. Konwinski, A. D. Joseph, R. H. Katz, and I. Stoica, "Improving mapreduce performance in heterogeneous environments," in OSDI, 2008, pp. 29-42.
17. C. Tian, H. Zhou, Y. He, and L. Zha, "A dynamic mapreduce scheduler for heterogeneous workloads," in GCC '09: Proceedings of the 2009 Eighth International Conference on Grid and Cooperative Computing. Washington, DC, USA: IEEE Computer Society, 2009, pp. 218-224. [Online]. Available: http://dx.doi.org/10.1109/GCC.2009.19
18. Jeffrey Dean and Sanjay Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," Communications of the ACM, vol. 51, no. 1, pp. 107-113, 2008.
19. S. V. Valvag, D. Johansen, and A. Kvalnes, "Cogset vs. hadoop: Measurements and analysis," Cloud Computing Technology and Science, IEEE International Conference on, vol. 0, pp. 768-775, 2010.
20. Z. Fadika, M. R. Head, and M. Govindaraju, "Parallel and Distributed Approach for Processing Large-Scale XML Datasets," in 10th IEEE/ACM International Conference on Grid Computing, October 2009, pp. 5-7.
21. S. Pallickara and G. Fox, "Naradabrokering: A distributed middleware framework and architecture for enabling durable peer-to-peer grids," in in Proceedings of ACM/IFIP/USENIX International Middleware Conference Middleware-2003, Rio Janeiro, 2003, pp. 41-61.
22. Apache ActiveMQ. [Online]. Available: http://activemq.apache.org/
23. K. Kim, K. Jeon, H. Han, S.-g. Kim, H. Jung, and H. Y. Yeom, "Mrbench: A benchmark for mapreduce framework," in Proceedings of the 2008 14th IEEE International Conference on Parallel and Distributed Systems. Washington, DC, USA: IEEE Computer Society, 2008, pp. 11-18. [Online]. Available: http://portal.acm.org/citation.cfm?id=1491261.1491574
24. A. Pavlo, E. Paulson, A. Rasin, D. J. Abadi, D. J. DeWitt, S. Madden, and M. Stonebraker, "A comparison of approaches to large-scale data analysis," in SIGMOD '09: Proceedings of the 35th SIGMOD international conference on Management of data. New York, NY, USA: ACM, 2009, pp. 165-178. [Online]. Available: http://database.cs.brown.edu/sigmod09/benchmarks-sigmod09. pdf
25. S. V. Valvag and D. Johansen, "Cogset: A unified engine for reliable storage and parallel processing," in Proceedings of the 2009 Sixth IFIP International Conference on Network and Parallel Computing, ser. NPC '09. Washington, DC, USA: IEEE Computer Society, 2009, pp. 174-181. [Online]. Available: http://dx.doi.org/10.1109/NPC.2009.23