GreenHDFS: Towards an energy-conserving, storage-efficient, hybrid hadoop compute cluster

HotPower 2010 - 2010 Workshop on Power Aware Computing and Systems

Volumn , Issue , 2010, Pages

  Kaushik, Rini T ^a   Bhandarkar, Milind ^b  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b YAHOO INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSIFICATION (OF INFORMATION); DATA HANDLING; DIGITAL STORAGE; ENERGY UTILIZATION; FILE ORGANIZATION; GREEN COMPUTING; POWER MANAGEMENT;

ACCESS PATTERNS; ANALYTICAL COSTS; DATA CLASSIFICATION; DATA INTENSIVE; ENERGY-CONSERVING; HADOOP DISTRIBUTED FILE SYSTEM (HDFS); TOTAL COST OF OWNERSHIP; TRACE DRIVEN SIMULATION;

ENERGY CONSERVATION;

EID: 85092758863     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (20)

1. http://yahoo.github.com/oozie/.
2. H. Amur, J. Cipar, V. Gupta, G. R. Ganger, M. A. Kozuch, and K. Schwan. Robust and flexible power-proportional storage. In SoCC, 2010.
3. H. Amur and K. Schwan. Achieving power-efficiency in clusters without distributed file system complexity. 2010.
4. E. Baldeschwieler. http://developer.yahoo.com/events/hadoopsummit2010/agenda.html.
5. L. A. Barroso and U. Hölzle. The case for energy-proportional computing. Computer, 40(12), 2007.
6. C. Bash and G. Forman. Cool job allocation: measuring the power savings of placing jobs at cooling-efficient locations in the data center. In ATC'07.
7. C. Belady. In the data center, power and cooling costs more than the it equipment it supports. Electronics Cooling, February, 2010.
8. G. Chen, W. He, J. Liu, S. Nath, L. Rigas, L. Xiao, and F. Zhao. Energy-aware server provisioning and load dispatching for connection-intensive internet services. In NSDI, 2008.
9. Y. Chen, A. Ganapathi, A. Fox, R. H. Katz, and D. A. Patterson. Statistical workloads for energy efficient mapreduce. Technical report, UC, Berkeley, 2010.
10. S. Ghemawat, H. Gobioff, and S.-T. Leung. The google file system. SIGOPS Oper. Syst. Rev., 37(5):29-43, 2003.
11. U. Hoelzle and L. Barroso. The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines. Morgan and Claypool Publishers, May 29, 2009.
12. R. T. Kaushik, L. Cherkasova, R. Campbell, and K. Nahrstedt. Lightning: Self-adaptive, energy-conserving, multi-zoned, commodity green cloud storage system. HPDC, 2010.
13. S. Konstantin, H. Kuang, S. Radia, and R. Chansler. The hadoop distributed file system. MSST, 2010.
14. K. Le, R. Bianchini, M. Martonosi, and T. Nguyen. Cost- and energy-aware load distribution across data centers. In HotPower, 2009.
15. J. Leverich and C. Kozyrakis. On the energy (in)efficiency of hadoop clusters. SIGOPS Operating Systems Review, 2010.
16. J. Moore, J. Chase, P. Ranganathan, and R. Sharma. Making scheduling”cool”: temperature-aware workload placement in data centers. In ATC'05.
17. D. Narayanan, A. Donnelly, and A. Rowstron. Write off-loading: Practical power management for enterprise storage. Trans. Storage, 4(3):1-23, 2008.
18. C. Patel, E. Bash, R. Sharma, and M. Beitelmal. Smart cooling of data centers. In IPACK, 2003.
19. R. K. Sharma, C. E. Bash, C. D. Patel, R. J. Friedrich, and J. S. Chase. Balance of power: Dynamic thermal management for internet data centers. IEEE Internet Computing, 9:42-49, 2005.
20. T. White. Hadoop: The Definitive Guide. O'Reilly Media, May, 2009.