Environment-conscious scheduling of HPC applications on distributed Cloud-oriented data centers

Journal of Parallel and Distributed Computing

Volumn 71, Issue 6, 2011, Pages 732-749

  Garg, Saurabh Kumar ^a   Yeo, Chee Shin ^b   Anandasivam, Arun ^c   Buyya, Rajkumar ^a  

^a UNIVERSITY OF MELBOURNE   (Australia)

^b INSTITUTE OF HIGH PERFORMANCE COMPUTING   (Singapore)

^c KARLSRUHE INSTITUTE OF TECHNOLOGY   (Germany)

Author keywords

Cloud computing; Dynamic Voltage Scaling (DVS); Energy efficient scheduling; Green IT; High Performance Computing (HPC)

Indexed keywords

ACCESS APPLICATIONS; CARBON EMISSIONS; CLOUD PROVIDERS; COMPUTING INFRASTRUCTURES; CPU POWER; CRITICAL ISSUES; DATA CENTERS; DYNAMIC VOLTAGE SCALING (DVS); EFFICIENCY FACTOR; ENERGY CONSUMPTION; ENERGY COST; ENERGY EFFICIENT; ENERGY SAVING; ENERGY-EFFICIENT SCHEDULING; GREEN IT; GROWING DEMAND; HIGH ENERGY CONSUMPTION; HIGH PERFORMANCE COMPUTING (HPC); HIGH-END COMPUTING; HIGH-ENERGY COSTS; HIGH-PERFORMANCE COMPUTING; IT APPLICATIONS; MANAGEMENT SYSTEMS; MULTIPLE DATA; OPTIMAL SCHEDULING; PROFIT MARGIN; SCHEDULING POLICIES;

ARCHITECTURAL DESIGN; COMPUTER SOFTWARE SELECTION AND EVALUATION; COMPUTER SYSTEMS; ENERGY EFFICIENCY; ENERGY UTILIZATION; HIGH ENERGY PHYSICS; INFORMATION MANAGEMENT; INFORMATION TECHNOLOGY; PROFITABILITY; SCHEDULING; VOLTAGE STABILIZING CIRCUITS;

CLOUD COMPUTING;

EID: 79955616336     PISSN: 07437315     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jpdc.2010.04.004     Document Type: Article

References (60)

1. Amazon, Amazon Elastic Compute Cloud (EC2), Aug. 2009. http://www.amazon.com/ec2/.
2. Alpiron, Alpiron Suite, 2009. http://www.alpiron.com.
3. C. Belady In the data center, power and cooling costs more than the it equipment it supports Electronics Cooling 13 1 2007 24
4. F. Berman, H. Casanova, A. Chien, K. Cooper, H. Dail, A. Dasgupta, W. Deng, J. Dongarra, L. Johnsson, K. Kennedy, C. Koelbel, B. Liu, X. Liu, A. Mandal, G. Marin, M. Mazina, J. Mellor-Crummey, C. Mendes, A. Olugbile, J.M. Patel, D. Reed, Z. Shi, O. Sievert, H. Xia, and A. YarKhan New grid scheduling and rescheduling methods in the grads project International Journal of Parallel Programming 33 2 2005 209 229 (Pubitemid 41202442)
5. R. Bianchini, and R. Rajamony Power and energy management for server systems Computer 37 11 2004 68 74
6. D. Bradley, R. Harper, and S. Hunter Workload-based power management for parallel computer systems IBM Journal of Research and Development 47 5 2003 703 718
7. T. Braun, H. Siegel, N. Beck, L. Boloni, M. Maheswaran, A. Reuther, J. Robertson, M. Theys, B. Yao, and D. Hensgen A comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems Journal of Parallel and Distributed Computing 61 6 2001 810 837 (Pubitemid 33658407)
8. T. Burd, R. Brodersen, Energy efficient CMOS microprocessor design, in: Proceedings of the 28th Hawaii International Conference on System Sciences, HICSS'95, vol. 1060, 1995.
9. J. Burge, P. Ranganathan, J.L. Wiener, Cost-aware scheduling for heterogeneous enterprise machines (CASH'EM), Technical Report HPL-2007-63, HP Labs, Palo Alto, Apr. 2007.
10. R. Buyya, C.S. Yeo, S. Venugopal, J. Broberg, and I. Brandic Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility Future Generation Computer Systems 25 6 2009 599 616
11. J.S. Chase, D.C. Anderson, P.N. Thakar, A.M. Vahdat, and R.P. Doyle Managing energy and server resources in hosting centers SIGOPS Operating Systems Review 35 5 2001 103 116 (Pubitemid 33612608)
12. Y. Chen, A. Das, W. Qin, A. Sivasubramaniam, Q. Wang, and N. Gautam Managing server energy and operational costs in hosting centers ACM SIGMETRICS Performance Evaluation Review 33 1 2005 303 314 (Pubitemid 43275430)
13. M. Chin, Desktop cpu power survey, Silentpcreview. com, 2006.
14. US Department of Energy, Voluntary reporting of greenhouse gases: Appendix F. Electricity emission factors, 2007. http://www.eia.doe.gov/oiaf/ 1605/pdf/Appendix20Fr071023.pdf.
15. K. Corrigan, A. Shah, C. Patel, Estimating environmental costs, in: Proceedings of the Ist USENIX Workshop on Sustainable Information Technology, San Jose, CA, USA, 2009.
16. US Department of Energy, US Energy Information Administration (EIA) report, 2007. http://www.eia.doe.gov/cneaf/electricity/epm/table56a.html.
17. A. Elyada, R. Ginosar, and U. Weiser Low-complexity policies for energy-performance tradeoff in chip-multi-processors IEEE Transactions on Very Large Scale Integration (VLSI) Systems 16 9 2008 1243 1248
18. United States Environmental Protection Agency, Letter to enterprise server manufacturer or other interested stakeholder, Dec. 2006. http://www.energystar.gov/ia/products/downloads/ServerAnnouncement.pdf.
19. United States Environmental Protection Agency, Report to congress on server and data center energy efficiency, Public Law 109-431, Aug. 2007. http://www.energystar.gov/ia/partners/proddevelopment/downloads/ EPADatacenterReportCongressFinal1.pdf.
20. M. Etinski, J. Corbalan, J. Labarta, M. Valero, A. Veidenbaum, Power-aware load balancing of large scale MPI applications, in: Proceedings of the 2009 IEEE International Symposium on Parallel & Distributed Processing, Rome, Italy, 2009.
21. 2 emissions from cars, Dec. 2007. http://www.eubusiness.com/Environ/co2-cars-eu-guide/.
22. X. Fan, W.-D. Weber, and L.A. Barroso Power provisioning for a warehouse-sized computer Proceedings of the 34th Annual International Symposium on Computer Architecture 2007 ACM New York, NY, USA 13 23 (Pubitemid 47582087)
23. D. Feitelson, Parallel workloads archive, Aug. 2009. http://www.cs.huji. ac.il/labs/parallel/workload.
24. D.G. Feitelson, L. Rudolph, U. Schwiegelshohn, K.C. Sevcik, P. Wong, Theory and practice in parallel job scheduling, in: Proceedings of the 1997 International Workshop on Job Scheduling Strategies for Parallel Processing, London, UK, 1997.
25. W. Feng, and K. Cameron The green500 list: encouraging sustainable supercomputing Computer 2007 50 55
26. X. Feng, R. Ge, K.W. Cameron, Power and energy profiling of scientific applications on distributed systems, in: Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium, Los Alamitos, CA, USA, 2005.
27. W. Feng, T. Scogland, The green500 list: year one, in: Proceedings of the 2009 IEEE International Symposium on Parallel & Distributed Processing, Rome, Italy, 2009.
28. V. Freeh, D. Lowenthal, F. Pan, N. Kappiah, R. Springer, B. Rountree, and M. Femal Analyzing the energy-time trade-off in high-performance computing applications IEEE Transactions on Parallel and Distributed Systems 18 6 2007 835
29. A. Gandhi, M. Harchol-Balter, R. Das, C. Lefurgy, Optimal power allocation in server farms, in: Proceedings of the 11th International Joint Conference on Measurement and Modeling of Computer Systems, Seattle, WA, USA, 2009.
30. 2 emissions, Apr. 2007. http://www.gartner.com/it/page.jsp?id= 503867.
31. M.R. Garey, and D.S. Johnson Computers and Intractability: A Guide to the Theory of NP-Completeness 1979 W. H. Freeman San Francisco, USA
32. S. Greenberg, E. Mills, B. Tschudi, P. Rumsey, B. Myatt, Best practices for data centers: results from benchmarking 22 data centers, in: Proceedings of the 2006 ACEEE Summer Study on Energy Efficiency in Buildings, Pacific Grove, USA, 2006.
33. T. Hardware, Cpu performance charts, Toms Hardware, 2006.
34. C. Hsu, U. Kremer, The design, implementation, and evaluation of a compiler algorithm for CPU energy reduction, in: Proceedings of the ACM SIGPLAN 2003 Conference on Programming Language Design and Implementation, Sweden, 2003.
35. O. Ibarra, and C. Kim Heuristic Algorithms for Scheduling Independent Tasks on Nonidentical Processors Journal of the ACM 24 2 1977 280 289
36. D. Irwin, L. Grit, J. Chase, Balancing risk and reward in a market-based task service, in: Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing, Honolulu, USA, 2004.
37. S.-H. Jang, V.E. Taylor, X. Wu, M. Prajugo, E. Deelman, G. Mehta, K. Vahi, Performance prediction-based versus load-based site selection: quantifying the difference, in: M. J. Oudshoorn, S. Rajasekaran (Eds.), ISCA PDCS, ISCA, 2005, pp. 148153.
38. K. Kim, R. Buyya, J. Kim, Power aware scheduling of bag-of-tasks applications with deadline constraints on dvs-enabled clusters, in: Proceedings of the Seventh IEEE International Symposium on Cluster Computing and the Grid, Rio de Janeiro, Brazil, 2007.
39. B. Lawson, E. Smirni, Power-aware resource allocation in high-end systems via online simulation, in: Proceedings of the 19th Annual International Conference on Supercomputing, Cambridge, USA, 2005.
40. J. Markoff, S. Hansell, Hiding in Plain Sight, google seeks more power, 2006. http://www.nytimes.com/2006/06/14/technology/14/search.html.
41. S. Martello, and P. Toth An algorithm for the generalized assignment problem Operational Research 81 1981 589 603
42. J. Moore, J. Chase, P. Ranganathan, R. Sharma, Making scheduling "cool": temperature-aware workload placement in data centers, in: Proceedings of the 2005 Annual Conference on USENIX Annual Technical Conference, Anaheim, CA, 2005.
43. A.W. Mu'alem, and D.G. Feitelson Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling IEEE Transactions on Parallel and Distributed Systems 12 6 2001 529 543 (Pubitemid 32611060)
44. G.R. Nudd, D.J. Kerbyson, E. Papaefstathiou, S.C. Perry, J.S. Harper, and D.V. Wilcox Pacea toolset for the performance prediction of parallel and distributed systems International Journal of High Performance Computing Application 14 3 2000 228 251 (Pubitemid 32025043)
45. A. Orgerie, L. Lefvre, J. Gelas, Save watts in your grid: green strategies for energy-aware framework in large scale distributed systems, in: Proceedings of the 2008 14th IEEE International Conference on Parallel and Distributed Systems, Melbourne, Australia, 2008.
46. C. Patel, R. Sharma, C. Bash, M. Beitelmal, Energy flow in the information technology stack: coefficient of performance of the ensemble and its impact on the total cost of ownership, HP Labs External Technical Report, HPL-2006-55.
47. C. Patel, R. Sharma, C. Bash, S. Graupner, Energy aware grid: global workload placement based on energy efficiency, Technical Report HPL-2002-329, HP Labs, Palo Alto, Nov. 2002.
48. P. Pillai, K. Shin, Real-time dynamic voltage scaling for low-power embedded operating systems, in: Proceedings of the 18th ACM Symposium on Operating Systems Principles, Banff, Canada, 2001.
49. R. Porter, Mechanism design for online real-time scheduling, in: Proceeding of the 5th ACM Conference on Electronic Commerce, New York, USA, 2004.
50. S. Rivoire, M.A. Shah, P. Ranganathan, and C. Kozyrakis Joulesort: a balanced energy-efficiency benchmark SIGMOD'07: Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data 2007 ACM New York, NY, USA 365 376 (Pubitemid 47630821)
51. C. Rusu, A. Ferreira, C. Scordino, A. Watson, Energy-efficient real-time heterogeneous server clusters, in: Proceedings of the 12th IEEE Real-Time and Embedded Technology and Applications Symposium, Stockholm, Sweden, 2006.
52. V. Salapura, et al. Power and performance optimization at the system level, in: Proceedings of the 2nd Conference on Computing Frontiers, Ischia, Italy, 2005.
53. H.A. Sanjay, and S. Vadhiyar Performance modeling of parallel applications for grid scheduling Journal of Parallel Distributed Computing 68 8 2008 1135 1145
54. G. Singh, C. Kesselman, E. Deelman, A provisioning model and its comparison with best-effort for performance-cost optimization in grids, in: Proceedings of the 16th International Symposium on High Performance Distributed Computing, California, USA, 2007.
55. W. Smith, I. Foster, and V. Taylor Predicting application run times using historical information D.G. Feitelson, L. Rudolph, Job Scheduling Strategies for Parallel Processing Lect. Notes Comput. Sci. vol. 1459 1998 Springer Verlag 122 142 (Pubitemid 128119034)
56. Q. Tang, S.K.S. Gupta, D. Stanzione, and P. Cayton Thermal-aware task scheduling to minimize energy usage of blade server based datacenters Proceedings of the 2nd IEEE International Symposium on Dependable, Autonomic and Secure Computing, DASC 2006 2006 IEEE Computer Society Los Alamitos, CA, USA
57. G. Tesauro, et al. Managing power consumption and performance of computing systems using reinforcement learning, in: Proceedings of the 21st Annual Conference on Neural Information Processing Systems, Vancouver, Canada, 2007.
58. TOP500 Supercomputers, Supercomputer's Application Area Share, 2009. http://www.top500.org/stats/list/33/apparea.
59. G. Verdun, D. Azevedo, H. Barrass, S. Berard, M. Bramfitt, T. Cader, T. Darby, C. Long, N. Gruendler, B. Macarthur, et al. The green grid metrics: data center infrastructure efficiency (DCIE) detailed analysis, the green grid.
60. L. Wang, Y. Lu, Efficient power management of heterogeneous soft real-time clusters, in: Proceedings of the 2008 Real-Time Systems Symposium, Barcelona, Spain, 2008.