On the performance and energy efficiency of Hadoop deployment models

Proceedings - 2013 IEEE International Conference on Big Data, Big Data 2013

Volumn , Issue , 2013, Pages 131-136

  Feller, Eugen ^a,b   Ramakrishnan, Lavanya ^b   Morin, Christine ^a,b  

^a INRIA   (France)

^b LAWRENCE BERKELEY NATIONAL LABORATORY   (United States)

Author keywords

Cloud Computing; Energy Efficiency; Hadoop MapReduce; Performance; Virtualization

Indexed keywords

BIG DATA; CLOUD COMPUTING; ELECTRIC POWER UTILIZATION; ENERGY EFFICIENCY; PARALLEL PROGRAMMING; SEPARATION; VIRTUALIZATION;

APPLICATION SPECIFIC; ENERGY EFFICIENCY EVALUATION; HADOOP MAPREDUCE; LARGE-SCALE DATA ANALYSIS; PARALLEL PROGRAMMING MODEL; PERFORMANCE; PERFORMANCE DEGRADATION; VIRTUALIZED ENVIRONMENT;

GREEN COMPUTING;

EID: 84893229786     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/BigData.2013.6691564     Document Type: Conference Paper

References (23)

1. J. Ekanayake, S. Pallickara, and G. Fox, "Mapreduce for data intensive scientific analyses," in Proceedings of the 2008 Fourth IEEE International Conference on eScience, ser. ESCIENCE '08, 2008, pp. 277-284.
2. A. Menon, "Big data @ facebook," in Proceedings of the 2012 workshop on Management of big data systems, ser. MBDS '12, 2012, pp. 31-32.
3. J. Dean and S. Ghemawat, "Mapreduce: simplified data processing on large clusters," in Proceedings of the 6th conference on Symposium on Opearting Systems Design &Implementation, 2004, pp. 10-10.
4. The Apache Hadoop Framework," http://hadoop.apache.org, 2013.
5. Amazon Elastic MapReduce," http://aws.amazon.com/ elasticmapreduce/, 2013.
6. R. McDougall, "Towards an Elastic Elephant: Enabling Hadoop for the Cloud," http://cto.vmware.com/ towards-an-elastic-elephant-enabling-hadoop- for-The-cloud/, 2012.
7. J. Leverich and C. Kozyrakis, "On the energy (in)efficiency of hadoop clusters," SIGOPS Oper. Syst. Rev., vol. 44, no. 1, pp. 61-65, Mar. 2010.
8. W. Lang and J. M. Patel, "Energy management for mapreduce clusters," Proc. VLDB Endow., vol. 3, no. 1-2, pp. 129-139, Sep. 2010.
9. T. Wirtz and R. Ge, "Improving mapreduce energy efficiency for computation intensive workloads," in Proceedings of the 2011 International Green Computing Conference and Workshops, ser. IGCC '11, 2011.
10. R. Bolze et al., "Grid'5000: A large scale and highly reconfigurable experimental grid testbed," Int. J. High Perform. Comput. Appl., vol. 20, no. 4, pp. 481-494, Nov. 2006.
11. S. Weil et al., "Ceph: A scalable, high-performance distributed file system," in Proceedings of the 7th symposium on Operating systems design and implementation, ser. OSDI '06, 2006, pp. 307-320.
12. The Gluster Filesystem," http://www.gluster.org, 2013.
13. G. Ananthanarayanan and et. al, "Disk-locality in datacenter computing considered irrelevant," in Proceedings of the 13th USENIX conference on Hot topics in operating systems, ser. HotOS'13, 2011.
14. E. Le Sueur and et. al., "Dynamic voltage and frequency scaling: the laws of diminishing returns," in Proceedings of the 2010 international conference on Power aware computing and systems, 2010, pp. 1-8.
15. Z. Fadika, M. Govindaraju, R. Canon, and L. Ramakrishnan, "Evaluating hadoop for data-intensive scientific operations," in Proceedings of the 2012 IEEE Conference on Cloud Computing, 2012, pp. 67-74.
16. E. Feller, L. Rilling, and C. Morin, "Snooze: A scalable and autonomic virtual machine management framework for private clouds," in Proceedings of the 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, ser. CCGRID '12, 2012, pp. 482-489.
17. D. Ghoshal, R. S. Canon, and L. Ramakrishnan, "I/o performance of virtualized cloud environments," in Proceedings of workshop on Data intensive computing in the clouds, 2011, pp. 71-80.
18. D. Moise and A. Carpen-Amarie, "Mapreduce applications in the cloud: A cost evaluation of computation and storage," in Data Management in Cloud, Grid and P2P Systems, vol. 7450, 2012, pp. 37-48.
19. G. Dhiman, G. Marchetti, and T. Rosing, "vgreen: a system for energy efficient computing in virtualized environments," in Proceedings of the 14th ACM/IEEE international symposium on Low power electronics and design, ser. ISLPED '09, 2009, pp. 243-248.
20. S. Ibrahim et al., "Evaluating mapreduce on virtual machines: The hadoop case," in Proceedings of the 1st International Conference on Cloud Computing, ser. CloudCom '09, 2009, pp. 519-528.
21. D. Jiang, B. C. Ooi, L. Shi, and S. Wu, "The performance of mapreduce: an in-depth study," Proc. VLDB Endow., vol. 3, no. 1-2, pp. 472-483, Sep. 2010.
22. Y. Chen, S. Alspaugh, D. Borthakur, and R. Katz, "Energy efficiency for large-scale mapreduce workloads with significant interactive analysis," in Proceedings of the 7th ACM european conference on Computer Systems, ser. EuroSys '12, 2012, pp. 43-56.
23. I. n. Goiri et al., "Greenhadoop: leveraging green energy in dataprocessing frameworks," in Proceedings of the 7th ACM european conference on Computer Systems, ser. EuroSys '12, 2012, pp. 57-70.