Technological drivers in data centers and telecom systems: Multiscale thermal, electrical, and energy management

Applied Energy

Volumn 107, Issue , 2013, Pages 66-80

  Garimella, Suresh V ^a   Persoons, Tim ^b   Weibel, Justin ^a   Yeh, Lian Tuu ^c  

^a PURDUE UNIVERSITY   (United States)

^b TRINITY COLLEGE DUBLIN   (Ireland)

^c HUAWEI TECHNOLOGIES   (United States)

Author keywords

3D chip packaging; Alternative air movers; Cloud computing; Electronics cooling; Power utilization effectiveness; Total cost of ownership

Indexed keywords

CLOUD COMPUTING; ELECTRIC POWER UTILIZATION; ELECTRONIC COOLING; ENERGY MANAGEMENT; TELECOMMUNICATION SYSTEMS; TEMPERATURE CONTROL;

AIR MOVERS; CALIFORNIA; CHIP PACKAGING; COOLING TECHNIQUE; DATA CENTERS; ELECTRONIC SYSTEMS; ELECTRONICS COOLING; ENERGY USAGE; MANAGEMENT CHALLENGES; MULTISCALES; SANTA CLARA; TECHNOLOGICAL CHALLENGES; TELECOM SYSTEMS; TOTAL COST OF OWNERSHIP;

INFORMATION TECHNOLOGY;

COOLING; LITERATURE REVIEW; OWNERSHIP; SPATIOTEMPORAL ANALYSIS; TELECOMMUNICATION; THREE-DIMENSIONAL FLOW;

CALIFORNIA; SANTA CLARA COUNTY; UNITED STATES;

EID: 84874709125     PISSN: 03062619     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.apenergy.2013.02.047     Document Type: Review

References (98)

1. Koomey J.G. Worldwide electricity used in data centers. Environ Res Lett 2008, 3(3):034008.
2. Koomey JG. Growth in data center electricity use 2005 to 2010. Analytics Press; August 1, 2011.
3. International Energy Agency. Key world energy statistics; 2010. [retrieved 08.10.12]. http://www.iea.org.
4. International Energy Agency. Energy balances of OECD countries; 2010. [retrieved 08.10.12]. http://www.iea.org.
5. International Energy Agency. Energy balances of non-OECD countries, 2010. [retrieved 08.10.12]. http://www.iea.org.
6. Turner V, Bigliani R, Ingle C. White paper: reducing greenhouse gases through intense use of information and communication technology: Part 1. International data corporation, Framingham, MA; November 2009.
7. Policy statement on green ICT. Australian Computer Society; August 2007.
8. Brill KG. White Paper: Data center energy efficiency and productivity. The Uptime Institute, Inc., Santa Fe, NM; February 2007.
9. Gartner, Inc. Press release, Stamford, CT; September 29, 2010.
10. Horvath A., Masanet E. An analysis of measures to reduce the life-cycle energy consumption and greenhouse gas emissions of California's personal computers 2007, University of California Energy Institute, Berkeley, CA, January.
11. Belady C, Rawson A, Pflueger J, Cadir T. The Green Grid data center power efficiency metrics: PUE and DCiE. The Green Grid: White Paper #6; 2008.
12. Energy Star for data centers. U.S. Environmental Protection Agency; February 2010.
13. ESIF high performance computing. U.S. Department of Energy. National Renewable Energy Laboratory, Golden, CO.; October, 2011.
14. Shah AJ, Bash CE, Kumari N, Cader T, Breen TJ, Walsh E, Punch J. On the need for energy efficiency metrics that span integrated IT-facility infrastructures. In: Proc ASME 2011 pacific rim technical conference and exhibition on packaging and integration of electronic and photonic systems (InterPACK); 2011, Paper IPACK2011-52140.
15. Hoover C, Kumari N, Felix CJ, Bash CE. A data center environmental advisory system. In: Proc ASME 2011 pacific rim technical conference and exhibition on packaging and integration of electronic and photonic systems (InterPACK); 2011 [Paper IPACK2011-52128].
16. Sharma RK, Bash CE, Patel CD. Dimensionless parameters for evaluation of thermal design and performance of large-scale data centers. In: Proc ASME/AIAA joint thermophysics and heat transfer conf; 2002 [Paper AIAA-2002-3091].
17. VanGilder J.W., Shrivastava S.K. Real-time prediction of rack-cooling performance. ASHRAE Trans 2006, 112(2):151-162.
18. Bash C, Forman G. Data center workload placement for energy efficiency. In: Proc ASME international electronic packaging technical conference and exhibition (InterPACK); 2007. p. 733-41.
19. Kumari N, Shah A, Bash C, Chen Y, Liu Z, Wang Z, Cader T, Slaby M, Cepulis D, Felix C, Aviles A, Figueroa M. Optimizing data center energy efficiency via ambient-aware IT workload scheduling. In: Proc intersoc conf thermal and thermomechanical phenomena in electronics systems (ITHERM); 2012 [Paper 2867].
20. Chen Y, Gmach D, Hyser C, Wang Z, Bash C, Hoover C, Singhal S. Integrated management of application performance, power and cooling in data centers. In: Proc 12th IEEE/IFIP network operations and management symposium (NOMS); April 2010.
21. IBM system X M4 servers deliver compelling performance and energy efficiency. IBM White Paper; March 2012.
22. Huber H. The petascale power challenge. 3rd European Workshop on HPC centre infrastructures, Garching, Germany; September 2011.
23. TOP500; June 2012. http://www.top500.org.
24. Whitepaper: 2011. Thermal guidelines for data processing environments - expanded data center classes and usage guidance. ASHRAE Technical Committee 9.9; 2011.
25. IBM Comments on Energy Star server specifications: Revised definitions for computer servers. IBM; May 12, 2008. [retrieved 19.11.12]. http://www.energystar.gov/ia/partners/prod_development/new_specs/downloads/IBM_Comments_Revised_Definitions.pdf.
26. 2 compression heat pump: techno-economic optimization of a high-efficiency Smart Grid option for buildings. Energy Build 2012, 50:128-138.
27. District Heating and Cooling - 2009 Statistics. Euroheat and power. [retrieved 19.11.12]. http://www.euroheat.org/Admin/Public/DWSDownload.aspx?File=%2fFiles%2fFiler%2fdocuments%2fDistrict+Heating%2fDHC_2009_Statistics_Table.pdf.
28. Eurostat. Electricity half-yearly prices. European commission; March 27, 2009. [retrieved 01.10.12]. http://epp.eurostat.ec.europa.eu/portal/page/portal/energy/data/database.
29. Tiwari MK, Zimmermann S, Sharma CS, Alfieri F, Renfer A, Brunschwiler T, Meijer I, Michel B, Poulikakos D. Waste heat recovery in supercomputers and 3D integrated liquid cooled electronics. In: Proc 13th IEEE intersoc conf thermal and thermomechanical phenomena in electronic systems (ITHERM); 2012. p. 545-51.
30. Kennedy J. Intel futurist Brian David Johnson on creating the tech of tomorrow. Siliconrepublic.com; July 26, 2012. [retrieved 27.07.12]. http://www.siliconrepublic.com/digital-21/item/28510-intel-futurist-brian-david/.
31. Mobile phone access reaches three quarters of planet's population (press release). The World Bank, Washington, DC; July 17, 2012. [retrieved 27.07.12]. http://www.worldbank.org/en/news/2012/07/17/mobile-phone-access-reaches-three-quarters-planets-population.
32. Bakken R. Energy-efficient management of a world class global network. Samsung Semiconductor CIO Forum, Munich, Germany; April 13, 2011.
33. Inside one of the world's largest data centers. CNET; November 2, 2009. [retrieved 07.07.12]. http://news.cnet.com/8301-13860_3-10371840-56.html.
34. Inside Microsoft's new Quincy cloud farm. Data Center Knowledge, January 4, 2011. [retrieved 07.07.12]. http://www.datacenterknowledge.com/inside-microsofts-new-quincy-cloud-farm/.
35. Garimella S.V., Yeh L.-T., Persoons T. Thermal management challenges in telecommunication systems and data centers. IEEE Trans Compon Packag Manuf Technol 2012, 2(8):1307-1316.
36. Michel B. Roadmap towards ultimately efficient datacenters. IBM; May 4, 2012. [retrieved 08.10.12]. http://www-950.ibm.com/events/wwe/grp/grp011.nsf/vLookupPDFs/Roadmap_Towards_Ultimately_Efficient_Datacenters_20120430_V1/$file/Roadmap_Towards_Ultimately_Efficient_Datacenters_20120430_V1.pdf.
37. Ruch P., Brunschwiler T., Escher W., Paredes S., Michel B. Towards five-dimensional scaling: how density improves efficiency in future computers. IBM J Res, Develop 2011, 55(5). p. 15:1-15:13.
38. IBM, Press Release, Armonk, NY; June 9, 2008. [retrieved 19.11.12]. http://www-03.ibm.com/press/us/en/pressrelease/24405.wss.
39. Merkle RC. Energy limits to the computational power of the human brain. Foresight Update 6, Foresight Institute; August 1, 1989.
40. Unlocking energy efficiency in the U.S. economy. McKinsey & Co.' July 2009.
41. Weik MH. A survey of domestic electronic digital computing systems. US Department of Commerce, Ballistic Research Laboratories Report No. 971; December 1955, p. 41. [retrieved 26.07.12]. http://ed-thelen.org/comp-hist/BRL.html.
42. Schmidt RR. Liquid cooling is back. Electronics Cooling; August, 2005.
43. Sri-Jayantha S.M., McVicker G., Bernstein K., Knickerbocker J.U. Thermomechanical modeling of 3D electronic packages. IBM J Res Develop 2008, 52(6):623-634.
44. Alfieri F, Tiwari MK, Zinovik I, Poulikakos D, Brunschwiler T, Michel B. 3D integrated water cooling of a composite multilayer stack of chips. In: Proc 14th int heat transfer conf (IHTC14); 2010 [Paper IHTC14-22812].
45. Cray-3 supercomputer systems. Cray Computer Corporation, Colorado Springs, CO.; August 1993. [retrieved 27.07.12]. http://archive.computerhistory.org/resources/access/text/2010/02/102685990-05-01-acc.pdf.
46. ICECool to crack thermal management barrier, enable breakthrough electronics (press release). DARPA, Arlington, VA; June 7, 2012. [retrieved 27.07.12]. http://www.darpa.mil/NewsEvents/Releases/2012/06/07.aspx.
47. Kogge P, Bergman K, Borkar S, Campbell D, Carlson W, Dally W, et al. ExaScale computing study: technology challenges in achieving exascale systems. DARPA, Report; September 28, 2008.
48. Alfieri F., Tiwari M.K., Zinovik I., Brunschwiler T., Michel B., Poulikakos D. On the significance of developing boundary layers in integrated water cooled 3D chip stacks. Int J Heat Mass Transfer 2012, 55(19-20):5222-5232.
49. Brunschwiler T, Paredes S, Drechsler U, Michel B, Wunderle B, Reichl H. Extended tensor description to design non-uniform heat-removal in interlayer cooled chip stacks. In: Proc 13th IEEE intersoc conf thermal and thermomechanical phenomena in electronic systems (ITHERM); 2012. p. 574-87.
50. Colgan EV, Bezama RJ, Gaynes M, Marston KC. A practical implementation of silicon microchannel coolers. Electron Cool; November 2007.
51. Wei J. Hybrid cooling technology for large-scale computing systems: from back to the future. In: Proc ASME 2011 pacific rim technical conference and exhibition on packaging and integration of electronic and photonic systems (InterPACK2011); 2011. p. 107-11.
52. Ellsworth MJ, Campbell LA, Simons RE, Iyengar MK, Schmidt RR, Chu RC. The evolution of water cooling for IBM large servers: back to the future. In: Proc 11th intersoc conf on thermal and thermomechanical phenomena in electronic systems (ITHERM); 2008. p. 266-74.
53. Made in IBM labs: IBM hot water-cooled supercomputer goes live at ETH Zurich (press release). IBM, Zurich, Switzerland; July 2, 2010. [retrieved 27.07.12]. http://www-03.ibm.com/press/us/en/pressrelease/32049.wss.
54. First commercial IBM hot-water cooled supercomputer to consume 40% less energy (press release). IBM, Munich, Germany; June 18, 2012. [retrieved 27.07.12]. http://www-03.ibm.com/press/us/en/pressrelease/38065.wss.
55. Kumari N., Bahadur V., Hodes M., Salamon T., Kolodner P., Lyons A., et al. Analysis of evaporating mist flow for enhanced convective heat transfer. Int J Heat Mass Transfer 2010, 53:3346-3356.
56. ISO 3744: 2010. Acoustics: Determination of sound power levels and sound energy levels of noise sources using sound pressure - engineering methods for an essentially free field over a reflecting plane. International Organization for Standardization; September 2010.
57. ISO 3741: 2010. Acoustics: determination of sound power levels and sound energy levels of noise sources using sound pressure - precision methods for reverberation test rooms. International Organization for Standardization; September 2010.
58. ISO 11201: 2010. Acoustics: noise emitted by machinery and equipment - determination of emission sound pressure levels at a work station and at other specified positions in an essentially free field over a reflecting plane with negligible environmental corrections. International Organization for Standardization; May 2010.
59. ISO 7779: 2010. Acoustics: measurement of airborne noise emitted by information technology and telecommunications equipment. International Organization for Standardization; July 2010.
60. Standard ECMA-74. Measurement of airborne noise emitted by information technology and telecommunications equipment, 11th ed. European Computer Manufacturers Association (ECMA), Geneva, Switzerland; December 2010.
61. ISO 10302: 1996. Acoustics: method for the measurement of airborne noise emitted by small air-moving devices. International organization for standardization; January 2011.
62. ISO 9296: 1988. Acoustics: declared noise emission values of computer and business equipment. International organization for standardization; February 1998.
63. Statskontoret Technical Standard 26:6. Acoustical noise emission of information technology equipment. Statskontoret, Swedish Agency for Public Management; July 2004.
64. ETS 300 753. Equipment engineering (EE): acoustic noise emitted by telecommunications equipment. European Telecommunications Standards Institute; October 1997.
65. OHSA Regulation 1910.95. Occupational noise exposure. Occupational safety and health administration; December 2008.
66. Council Directive 89/391/EEC on the introduction of measures to encourage improvements in the safety and health of workers at work. European Agency for Safety and Health at Work; June 1989.
67. Lyon R.H., Bergles A.E. Noise and cooling in electronics packages. IEEE Trans Compon Packag Technol 2006, 29(3):535-542.
68. Wakil J, Questad D, Gaynes M, Hamann H, Weger A, Wang M, et al. Thermal development, modeling and characterization of the cell processor module. In: Proc 10th intersoc conf thermal and thermomechanical phenomena in electronics systems (ITHERM); 2006. p. 289-96.
69. Kaplan J.M., Forrest W., Kindler N. Revolutionizing data center energy efficiency 2008, McKinsey & Co., July.
70. Hughes J. Google's new Finland data center is cooled by Baltic Sea's water (press release). Digitaltrends.com, May 24, 2011. [retrieved 27.07.12]. http://www.digitaltrends.com/computing/googles-new-finland-data-center-is-cooled-by-baltic-seas-water/.
71. Fehrenbacher K. The alternative option for energy storage makers: off-grid. Gigaom.com, June 21, 2011. [retrieved 30.07.12]. http://gigaom.com/cleantech/the-alternative-option-for-energy-storage-makers-off-grid/.
72. Wang Y., Lee H.C., Li K.M., Gu Z., Chen J. Experimental and numerical study of flow over a cavity for reduction of buffeting noise. Acta Acust united Ac 2012, 98(4):600-610.
73. Sony. BCU-100 Computing Unit with Cell/B.E. and RSX. White Paper. Sony Corporation, 2008. [retrieved 30.07.12]. http://pro.sony.com/bbsccms/ext/ZEGO/files/BCU-100_Whitepaper.pdf.
74. Krishnan S., Hernon D., Hodes M., Mullins J., Lyons A.M. Design of complex structured monolithic heat sinks for enhanced air cooling. IEEE Trans Compon Packag Manuf Technol 2012, 2(2):266-277.
75. Acikalin T., Garimella S.V. Analysis and prediction of the thermal performance of piezoelectrically actuated fans. Heat Transf Eng 2009, 30(6):487-498.
76. Kimber M.L., Suzuki K., Kitsunai N., Seki K., Garimella S.V. Pressure and flow rate performance of piezoelectric fans. IEEE Trans Compon Packaging Technol 2009, 32(4):766-775.
77. Kimber M.L., Garimella S.V. Cooling performance of arrays of vibrating cantilevers. J Heat Transfer-Trans ASME 2009, 131(11):111401.
78. Sauciuc I, Gupta A. Electronics cooling advancements, challenges and innovation needs: System/chip level thermal management. In: 2nd Workshop on thermal management in telecommunication systems and data centers in Santa Clara, CA; April 25-26, 2012.
79. Persoons T. General reduced-order model to design and operate synthetic jet actuators. AIAA J 2012, 50(4):916-927.
80. Persoons T., McGuinn A., Murray D.B. A general correlation for the stagnation point Nusselt number of an axisymmetric impinging synthetic jet. Int J Heat Mass Transfer 2011, 54(17-18):3900-3908.
81. Murata. Features and technical specifications of microblower. Murata Manufacturing Company, Ltd.; 2012. [retrieved 30.07.12]. http://www.murata.com/products/micromechatronics/feature/microblower/index.html.
82. Mahalingam R., Heffington S., Jones L., Williams R. Synthetic jets for forced air cooling of electronics. Electron Cool 2007, 2-5.
83. Persoons T, O'Donovan TS, Murray DB. Heat transfer in adjacent interacting impinging synthetic jets. In: Proc ASME summer heat transfer conference (HT2009), vol. 1; 2009. p. 955-62.
84. Go D.B., Garimella S.V., Fisher T.S., Mongia R.K. Ionic winds for locally enhanced cooling. J Appl Phys 2007, 102(5):053302.
85. Go D.B., Maturana R.A., Fisher T.S., Garimella S.V. Enhancement of external forced convection by ionic wind. Int J Heat Mass Transfer 2008, 51(25-26):6047-6053.
86. Schlitz DJ, Singhal V. An electro-aerodynamic solid state fan and cooling system. In: Proc 24th IEEE semiconductor thermal measurement and management symposium (SEMI-THERM); 2008. p. 46-9.
87. Harirchian T., Garimella S.V. A comprehensive flow regime map for microchannel flow boiling with quantitative transition criteria. Int J Heat Mass Transfer 2010, 53(13-14):2694-2702.
88. Garimella S.V., Rice R.A. Confined and submerged liquid jet impingement heat transfer. J Heat Transfer-Trans ASME 1995, 117(4):871-877.
89. ASHRAE. 2011 Thermal guidelines for liquid cooled data processing environments. White Paper, ASHRAE Technical Committee 9.9, American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc.; 2011.
90. Sahu V, Yazawa K, Ziabari A, Shakouri A, Fedorov AG, Joshi Y. Energy efficient liquid-thermoelectric hybrid cooling for hot-spot removal. In: Proc 28th IEEE semiconductor thermal measurement and management symposium (SEMI-THERM); 2012. p. 130-134.
91. Shakouri A. Nanoscale thermal transport and microrefrigerators on a chip. Proc IEEE 2006, 94(8):1613-1638.
92. Bahadur V., Garimella S.V. Electrowetting-based control of static droplet states on rough surfaces. Langmuir 2007, 23(9):4918-4924.
93. Kumari N., Bahadur V., Garimella S.V. Electrical actuation of dielectric droplets. J Micromech Microeng 2008, 18(8):085018.
94. Annapragada S.R., Dash S., Garimella S.V., Murthy J.Y. Dynamics of droplet motion under electrowetting actuation. Langmuir 2011, 27(13):8198-8204.
95. Boreyko J.B., Zhao Y., Chen C.-H. Planar jumping-drop thermal diodes. Appl Phys Lett 2011, 99(23):234105.
96. Altman DH, Weibel JA, Garimella SV, Fisher TS, Nadler JH, North N. Thermal ground plane vapor chamber heat spreaders for high power packaging density electronic systems. Electron Cool; March 2012. p. 20-7.
97. Weibel J.A., Garimella S.V., North M.T. Characterization of evaporation and boiling from sintered powder wicks fed by capillary action. Int J Heat Mass Transfer 2010, 53(19-20):4204-4215.
98. Weibel J.A., Garimella S.V., Murthy J.Y., Altman D.H. Design of integrated nanostructured wicks for high-performance vapor chambers. IEEE Trans Compon Packag Manuf Technol 2011, 1(6):859-867.