Dynamic energy efficient data placement and cluster reconfiguration algorithm for MapReduce framework

Future Generation Computer Systems

Volumn 28, Issue 1, 2012, Pages 119-127

  Maheshwari, Nitesh ^a   Nanduri, Radheshyam ^a   Varma, Vasudeva ^a  

^a INTERNATIONAL INSTITUTE OF INFORMATION TECHNOLOGY   (India)

Author keywords

Cloud computing; Energy efficiency; MapReduce

Indexed keywords

CLUSTER NODES; CLUSTER UTILIZATION; CLUSTER-BASED; DATA PLACEMENT; DISTRIBUTED FILE SYSTEMS; DYNAMIC ENERGY; ENERGY REDUCTION; GRIDSIM TOOLKITS; LARGE SCALE DATA; MAPREDUCE; OPERATIONAL COSTS; RECONFIGURATION ALGORITHM;

CLOUD COMPUTING; CLUSTER COMPUTING; ENERGY EFFICIENCY; INTERCONNECTION NETWORKS;

CLUSTERING ALGORITHMS;

EID: 81155161014     PISSN: 0167739X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.future.2011.07.001     Document Type: Conference Paper

References (38)

1. EPA Report to Congress on Server and Data Center Energy Efficiency, Tech. Rep., US Environmental Protection Agency, 2007.
2. J. Dean, and S. Ghemawat Mapreduce: simplified data processing on large clusters Commun. ACM 51 1 2008 107 113 10.1145/1327452.1327492
3. Server energy and efficiency report, Tech. Rep., 1E, 2009.
4. Apache Hadoop. http://hadoop.apache.org.
5. Hadoop distributed file system. http://hadoop.apache.org/hdfs/.
6. R. Buyya, and M. Murshed Gridsim: a toolkit for the modeling and simulation of distributed resource management and scheduling for grid computing Concurr. Comput. Pract. Exp. 14 2002 1175 1220 10.1002/cpe.710
7. M. Weiser, B. Welch, A. Demers, S. Shenker, Scheduling for reduced CPU energy, in: OSDI'94: Proceedings of the 1st USENIX Conference on Operating Systems Design and Implementation, USENIX Association, Berkeley, CA, USA, 1994, p. 2.
8. A. Rangasamy, R. Nagpal, and Y. Srikant Compiler-directed frequency and voltage scaling for a multiple clock domain microarchitecture CF'08: Proceedings of the 5th Conference on Computing Frontiers 2008 ACM New York, NY, USA 209 218 10.1145/1366230.1366267
9. A.R. Lebeck, X. Fan, H. Zeng, and C. Ellis Power aware page allocation ASPLOS-IX: Proceedings of the Ninth International Conference on Architectural Support for Programming Languages and Operating Systems 2000 ACM New York, NY, USA 105 116 10.1145/378993.379007 (Pubitemid 32470890)
10. D.P. Helmbold, D.D. Long, T.L. Sconyers, and B. Sherrod Adaptive disk spin-down for mobile computers Mobile Netw. Appl. 5 2000 285 297 (Pubitemid 32160131)
11. M. Elnozahy, M. Kistler, R. Rajamony, Energy conservation policies for web servers, in: USITS'03: Proceedings of the 4th Conference on USENIX Symposium on Internet Technologies and Systems, USENIX Association, Berkeley, CA, USA, 2003.
12. E.V. Carrera, E. Pinheiro, and R. Bianchini Conserving disk energy in network servers ICS'03: Proceedings of the 17th Annual International Conference on Supercomputing 2003 ACM New York, NY, USA 86 97 10.1145/782814.782829
13. S. Gurumurthi, A. Sivasubramaniam, M. Kandemir, H. Franke, DRPM: dynamic speed control for power management in server class disks, in: Computer Architecture, International Symposium on, vol. 0, 2003, p. 169. doi:10.1109/ISCA.2003.1206998.
14. J. Dhok, N. Maheshwari, and V. Varma Learning based opportunistic admission control algorithm for MapReduce as a service ISEC'10: Proceedings of the 3rd India Software Engineering Conference 2010 ACM 153 160 10.1145/1730874.1730903
15. Amazon elastic MapReduce. http://aws.amazon.com/elasticmapreduce/.
16. R. Buyya, C.S. Yeo, S. Venugopal, J. Broberg, and I. Brandic Cloud computing and emerging it platforms: vision, hype, and reality for delivering computing as the 5th utility Future Gener. Comput. Syst. 25 6 2009 599 616 10.1016/j.future.2008.12.001
17. L. Barroso, and U. Holzle The case for energy-proportional computing Computer 40 12 2007 33 37
18. S. Ghemawat, H. Gobioff, and S.-T. Leung The google file system SIGOPS Oper. Syst. Rev. 37 5 2003 29 43 10.1145/1165389.945450 (Pubitemid 40929686)
19. N. Vasi, M. Barisits, V. Salzgeber, and D. Kostic Making cluster applications energy-aware ACDC'09: Proceedings of the 1st Workshop on Automated Control for Datacenters and Clouds 2009 ACM New York, NY, USA 37 42 10.1145/1555271.1555281
20. [HDFS-385] Design a pluggable interface to place replicas of blocks in HDFS. https://issues.apache.org/jira/browse/HDFS-385.
21. R. Buyya, A. Beloglazov, J.H. Abawajy, Energy-efficient management of data center resources for cloud computing: a vision, architectural elements, and open challenges. CoRR abs/1006.0308.
22. X. Fan, W.-D. Weber, and L.A. Barroso Power provisioning for a warehouse-sized computer ISCA'07: Proceedings of the 34th Annual International Symposium on Computer Architecture 2007 ACM New York, NY, USA 13 23 10.1145/1250662.1250665 (Pubitemid 47582087)
23. J. Leverich, and C. Kozyrakis On the energy (in)efficiency of hadoop clusters SIGOPS Oper. Syst. Rev. 44 1 2010 61 65 10.1145/1740390.1740405
24. Y. Chen, T.X. Wang, Energy efficiency of MapReduce, 2008.
25. E. Pinheiro, R. Bianchini, E.V. Carrera, T. Heath, Load balancing and unbalancing for power and performance in cluster-based systems, 2001.
26. E. Pinheiro, R. Bianchini, E.V. Carrera, and T. Heath Dynamic cluster reconfiguration for power and performance Power and Energy Management for Server Systems 2004 IEEE Computer Society
27. M.S. Pérez, A. Sánchez, J.M. Pena, and V. Robles A new formalism for dynamic reconfiguration of data servers in a cluster J. Parallel Distrib. Comput. 65 10 2005 1134 1145 10.1016/j.jpdc.2005.04.018 Design and Performance of Networks for Super-, Cluster-, and Grid-Computing Part I (Pubitemid 41240619)
28. T.V.T. Duy, Y. Sato, Y. Inoguchi, Performance evaluation of a green scheduling algorithm for energy savings in cloud computing, in: Parallel Distributed Processing, Workshops and Ph.D. Forum, IPDPSW, 2010 IEEE International Symposium on, 2010, pp. 18. doi:10.1109/IPDPSW.2010.5470908.
29. K.H. Kim, R. Buyya, J. Kim, Power aware scheduling of bag-of-tasks applications with deadline constraints on DVS-enabled clusters, in: Cluster Computing and the Grid, 2007. CCGRID 2007. Seventh IEEE International Symposium on, 2007, pp. 541548. doi:10.1109/CCGRID.2007.85. (Pubitemid 47334594)
30. Y.C. Lee, and A.Y. Zomaya Minimizing energy consumption for precedence-constrained applications using dynamic voltage scaling CCGRID'09: Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid 2009 IEEE Computer Society Washington, DC, USA 92 99 10.1109/CCGRID.2009.16
31. C. Hsu, and W. Feng A power-aware run-time system for high-performance computing SC'05: Proceedings of the 2005 ACM/IEEE Conference on Supercomputing 2005 IEEE Computer Society Washington, DC, USA 1 10.1109/SC.2005.3 (Pubitemid 44902344)
32. A. Berl, E. Gelenbe, M. DiGirolamo, G. Giuliani, H. DeMeer, M.Q. Dang, and K. Pentikousis Energy-efficient cloud computing Comput. J. 53 7 2010 1045 1051 10.1093/comjnl/bxp080
33. L. Liu, H. Wang, X. Liu, X. Jin, W.B. He, Q.B. Wang, and Y. Chen Greencloud: a new architecture for green data center ICAC-INDST'09: Proceedings of the 6th International Conference Industry Session on Autonomic Computing and Communications Industry Session 2009 ACM New York, NY, USA 29 38 10.1145/1555312.1555319
34. L. Hu, H. Jin, X. Liao, X. Xiong, H. Liu, Magnet: a novel scheduling policy for power reduction in cluster with virtual machines, in: Cluster Computing, 2008 IEEE International Conference on, 2008, pp. 1322. doi:10.1109/CLUSTR.2008.4663751.
35. M. Milenkovic, E. Castro-Leon, and J. Blakley Power-aware management in cloud data centers M. Jaatun, G. Zhao, C. Rong, Cloud Computing Lecture Notes in Computer Science vol. 5931 2009 Springer Berlin, Heidelberg 668 673
36. K.H. Kim, A. Beloglazov, and R. Buyya Power-aware provisioning of cloud resources for real-time services MGC'09: Proceedings of the 7th International Workshop on Middleware for Grids, Clouds and e-Science 2009 ACM New York, NY, USA 1 6 10.1145/1657120.1657121
37. G. von Laszewski, L. Wang, A. Younge, X. He, Power-aware scheduling of virtual machines in DVFS-enabled clusters, in: Cluster Computing and Workshops, 2009. CLUSTER '09. IEEE International Conference on, 2009, pp. 110. doi:10.1109/CLUSTR.2009.5289182.
38. Advanced Configuration and Power Interface (ACPI). http://www.acpi.info/.