A Survey Of Techniques for Architecting DRAM Caches

IEEE Transactions on Parallel and Distributed Systems

Volumn 27, Issue 6, 2016, Pages 1852-1863

  Mittal, Sparsh ^a   Vetter, Jeffrey S ^a,b  

^a OAK RIDGE NATIONAL LABORATORY   (United States)

^b GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

3D; architectural techniques; bandwidth wall; classification; die stacking; extreme scale system; last level cache; Review; stacked DRAM

Indexed keywords

CLASSIFICATION (OF INFORMATION); DYNAMIC RANDOM ACCESS STORAGE; REVIEWS; SURVEYS;

ARCHITECTURAL TECHNIQUES; CACHE CAPACITY; COMPUTING SYSTEM; DIE STACKING; EXTREME SCALE; GAINING INSIGHTS; LAST-LEVEL CACHES; MAJOR OVERHAULS;

CACHE MEMORY;

EID: 84969930755     PISSN: 10459219     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPDS.2015.2461155     Document Type: Review

References (67)

1. S. Rusu, H. Muljono, D. Ayers, S. Tam, W. Chen, A. Martin, S. Li, S. Vora, R. Varada, and E. Wang, "Ivytown: A 22nm 15-core enterprise Xeon® processor family," in Proc. IEEE Int. Solid-State Circuits Conf., 2014, pp. 102-103.
2. D. Milojevic, S. Idgunji, D. Jevdjic, E. Ozer, P. Lotfi-Kamran, A. Panteli, A. Prodromou, C. Nicopoulos, D. Hardy, B. Falsari et al., "Thermal characterization of cloud workloads on a powerefficient Server-on-chip," in Proc. Int. Conf. Comput. Des., 2012, pp. 175-182.
3. M. R. Meswani, S. Blagodurov, D. Roberts, J. Slice, M. Ignatowski, and G. Loh, "Heterogeneous memory Architectures: A HW/SW approach for mixing die-stacked and off-package memories," in Proc. IEEE 21st Int. Symp. High Perform. Comput. Archit., 2015, pp. 126-136.
4. B. M. Rogers, A. Krishna, G. B. Bell, K. Vu, X. Jiang, and Y. Solihin, "Scaling the bandwidth wall: Challenges in and avenues for CMP scaling," in Proc. Int. Symp. Comput. Archit., 2009, pp. 371-382.
5. N. Hardavellas, M. Ferdman, A. Ailamaki, and B. Falsafi, "Power scaling: The ultimate obstacle to 1K-core chips," Northwestern Univ., Evanston, IL, USA, Tech. Rep. NWU-EECS-10-05, 2010.
6. B. Black, M. Annavaram, N. Brekelbaum, J. De Vale, L. Jiang, G. H. Loh, D. McCauley, P. Morrow, D. W. Nelson, D. Pantuso et al., "Die stacking (3D) microarchitecture," in Proc. Int. Symp. Microarchit., 2006, pp. 469-479.
7. T. C. Xu, P. Liljeberg, and H. Tenhunen, "Exploring DRAM last level cache for 3D network-on-chip architecture," Adv. Materials Res., vol. 403, pp. 4009-4018, 2012.
8. M. K. Qureshi and G. H. Loh, "Fundamental latency Trade-off in architecting DRAM caches: Outperforming impractical SRAMtags with a simple and practical design," in Proc. Int. Symp. Microarchit., 2012, pp. 235-246.
9. (2014). Intel Xeon Phi knights landing processors to feature onboard stacked DRAM supercharged hybrid memory cube (HMC) upto 16 GB. [Online]. Available: http://wccftech.com/intelxeon-phi-knights-landing-processors-stacked-dram-hmc-16gb/
10. C. C. Chou, A. Jaleel, and M. K. Qureshi, "CAMEO: A two-level memory organization with capacity of main memory and flexibility of hardware-managed cache," in Proc. Int. Symp. Microarchit., 2014, pp. 1-12.
11. S. Yin, J. Li, L. Liu, S. Wei, and Y. Guo, "Cooperatively managing dynamic writeback and insertion policies in a last-level DRAM cache," in Proc. Des., Autom. Test Eur., 2015, pp. 187-192.
12. X. Jiang, N. Madan, L. Zhao, M. Upton, R. Iyer, S. Makineni, D. Newell, D. Solihin, and R. Balasubramonian, "CHOP: Adaptive filter-based DRAM caching for CMP server platforms," in Proc. Int. Symp. High Perform. Comput. Archit., 2010, pp. 1-12.
13. J.-S. Kim, C. S. Oh, H. Lee, D. Lee, H.-R. Hwang, S. Hwang, B. Na, J. Moon, J.-G. Kim, H. Park, J.-W. Ryu, K. Park, S.-K. Kang, S.-Y. Kim, H. Kim, J.-M. Bang, H. Cho, M. Jang, C. Han, J.-B. Lee, K. Kyung, J.-S. Choi, and Y.-H. Jun, "A 1.2V 12.8GB/s 2Gb mobile Wide-I/O DRAM with 4X128 I/Os using TSV-based stacking," in Proc. IEEE Int. Solid-State Circuits Conf. Digest Tech. Papers, 2011, pp. 496-498.
14. U. Kang, H.-J. Chung, S. Heo, S.-H. Ahn, H. Lee, S.-H. Cha, J. Ahn, D. Kwon, J. H. Kim, J.-W. Lee et al., "8Gb 3D DDR3 DRAM using through-silicon-via technology," in Proc. IEEE Int. Solid-State Circuits Conf.-Digest Tech. Papers, 2009, pp. 130-131.
15. Tezzaron semiconductor. (2010). Octopus 8-Port DRAM for diestack applications. [Online]. Available: www.tachyonsemi.com/memory/datasheets/TSC10080x-0-1.pdf
16. J. Jeddeloh and B. Keeth, "Hybrid memory cube new DRAM architecture increases density and performance," in Proc. Symp. VLSI Technol., 2012, pp. 87-88.
17. D. U. Lee, K. W. Kim, K. W. Kim, K. S. Lee, S. J. Byeon, J. H. Kim, J. H. Cho, J. Lee, and J. H. Chun, "A 1.2 V 8 Gb 8-Channel 128 GB/s high-bandwidth memory (HBM) stacked DRAM with effective I/O test circuits," IEEE J. Solid-State Circuits, vol. 50, no. 1, pp. 191-203, 2015.
18. (2015). [Online]. Available: http://www.pcper.com/reviews/General-Tech/High-Bandwidth-Memory-HBM-Architecture-AMD-Plans-Future-GPUs
19. NVIDIA. (2014). [Online]. Available: http://devblogs.nvidia.com/parallelforall/nvlink-pascal-stacked-memory-feeding-appetite-big-data/
20. S. Mittal, J. S. Vetter, and D. Li, "A survey of architectural approaches for managing embedded dram and non-volatile onchip caches," IEEE Trans. Parallel Distrib. Syst., vol. 26, no. 6, pp. 1524-1537, Jun. 2015.
21. N. Gulur, M. Mehendale, R. Manikantan, and R. Govindarajan, "Bi-Modal DRAM cache: Improving hit rate, hit latency and bandwidth," in Proc. Int. Symp. Microarchit., 2014, pp. 38-50.
22. N. Gulur, M. Mehendale, and R. Govindarajan, "A comprehensive analytical performance model of DRAM caches," in Proc. Int. Conf. Perform. Eng., 2015, pp. 157-168.
23. K. Chen, S. Li, N. Muralimanohar, J. H. Ahn, J. B. Brockman, and N. P. Jouppi, "CACTI-3DD: Architecture-level modeling for 3D die-stacked DRAM main memory," in Proc. Conf. Des., Autom. Test Eur., 2012, pp. 33-38.
24. S. Mittal, "A survey of architectural techniques for DRAM power management," Int. J. High Perform. Syst. Archit., vol. 4, no. 2, pp. 110-119, 2012.
25. G. H. Loh, "3D-stacked memory architectures for multi-core processors," in Proc. Int. Symp. Comput. Archit., 2008, pp. 453-464.
26. S. Mittal, "A survey of architectural techniques for improving cache power efficiency," Elsevier Sustainable Comput.: Inform. Syst., vol. 4, no. 1, pp. 33-43, Mar. 2014.
27. F. Hameed, L. Bauer, and J. Henkel, "Reducing latency in an SRAM/DRAM cache hierarchy via a novel tag-cache architecture," in Proc. 51st Annu. Des. Autom. Conf., 2014, pp. 1-6.
28. X. Dong, Y. Xie, N. Muralimanohar, and N. P. Jouppi, "Simple but effective heterogeneous main memory with on-chip memory controller support," in Proc. Int. Conf. High Perform. Comput., Netw., Storage Anal., 2010, pp. 1-11.
29. G. H. Loh and M. D. Hill, "Efficiently enabling conventional block sizes for very large die-stacked DRAM caches," in Proc. Int. Symp. Microarchit., 2011, pp. 454-464.
30. G. H. Loh, N. Jayasena, K. McGrath, M. O'Connor, S. Reinhardt, and J. Chung, "Challenges in heterogeneous die-stacked and offchip memory systems," in Proc. 3rd Workshop SoCs, Heterogeneity, Workloads, 2012.
31. M. El-Nacouzi, I. Atta, M. Papadopoulou, J. Zebchuk, N. E. Jerger, and A. Moshovos, "A dual grain hit-miss detector for large diestacked DRAM caches," in Proc. Des., Autom. Test Eur., 2013, pp. 89-92.
32. J. Meza, J. Chang, H. Yoon, O. Mutlu, and P. Ranganathan, "Enabling efficient and scalable hybrid memories using Finegranularity DRAM cache management," Comput. Archit. Lett., vol. 11, no. 2, pp. 61-64, 2012.
33. M. Poremba, S. Mittal, D. Li, J. S. Vetter, and Y. Xie, "DESTINY: A tool for modeling emerging 3D NVM and eDRAM caches," in Proc. Des. Autom. Test Eur., 2015, pp. 1543-1546.
34. C. Chou, A. Jaleel, and M. K. Qureshi, "BATMAN: Maximizing bandwidth utilization of hybrid memory systems," School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, Georgia, USA 30332-0250 Georgia Tech, Tech. Rep. TR-CARET-2015-01, 2015.
35. J. Sim, G. H. Loh, H. Kim, M. O'Connor, and M. Thottethodi, "A mostly-clean DRAM cache for effective hit speculation and selfbalancing dispatch," in Proc. Int. Symp. Microarchit., 2012, pp. 247-257.
36. J. Sim, G. H. Loh, V. Sridharan, and M. O'Connor, "Resilient diestacked DRAM caches," in Proc. Int. Symp. Comput. Archit., 2013, pp. 416-427.
37. S. Mittal and Z. Zhang, "EnCache: A dynamic profiling based reconfiguration technique for improving cache energy efficiency," J. Circuits, Syst. Comput., vol. 23, no. 10, p. 1450147, 2014.
38. S. Mittal and J. S. Vetter, "AYUSH: A technique for extending lifetime of SRAM-NVM hybrid caches," IEEE Comput. Archit. Lett., 2015, DOI: 10.1109/LCA.2014.2355193.
39. L. Zhao, R. Iyer, R. Illikkal, and D. Newell, "Exploring DRAM cache architectures for CMP server platforms," in Proc. Int. Conf. Comput. Des., 2007, pp. 55-62.
40. Y. Chen, E. Li, C. Kim, and Z. Tang, "Efficient shared cache management through sharing-aware replacement and streamingaware insertion policy," in Proc. IEEE Int. Symp. Parallel Distrib. Process., 2009, pp. 1-11.
41. H. Sun, J. Liu, R. Anigundi, N. Zheng, J. Lu, R. Ken, and T. Zhang, "Design of 3D DRAM and its application in 3D integrated multicore computing systems," IEEE Des. Test Comput., 2013, DOI: 10.1109/MDT.2009.93.
42. G. H. Loh, "Extending the effectiveness of 3D-stacked DRAM caches with an adaptive multi-queue policy," in Proc. Int. Symp. Microarchit., 2009, pp. 201-212.
43. K. Inoue, S. Hashiguchi, S. Ueno, N. Fukumoto, and K. Murakami, "3D implemented SRAM/DRAM hybrid cache architecture for high-performance and low power consumption," in Proc. Int. Midwest Symp. Circuits Syst., 2011, pp. 1-4.
44. F. Hameed, L. Bauer, and J. Henkel, "Adaptive cache management for a combined SRAM and DRAM cache hierarchy for multi-cores," in Proc. Des., Autom. Test Eur. Conf. Exhib., 2013, pp. 77-82.
45. F. Hameed, L. Bauer, and J. Henkel, "Reducing inter-core cache contention with an adaptive bank mapping policy in DRAM cache," in Proc. Int. Conf. Hardware/Softw. Codes. Syst. Synthesis, 2013, pp. 1-8.
46. F. Hameed, L. Bauer, and J. Henkel, "Simultaneously optimizing dram cache hit latency and miss rate via novel set mapping policies," in Proc. Int. Conf. Compilers, Archit. Synthesis Embedded Syst., 2013, pp. 11:1-11:10.
47. N. M. Tshibangu, P. D. Franzon, E. Rotenberg, and W. R. Davis, "Design of controller for L2 cache mapped in Tezzaron stacked DRAM," in Proc. IEEE Int. 3D Syst. Integration Conf., 2013, pp. 1-4.
48. M. R. Meswani, G. H. Loh, S. Blagodurov, D. Roberts, J. Slice, and M. Ignatowski, "Toward efficient programmer-managed twolevel memory hierarchies in exascale computers," in Proc. Hardware-Softw. Co-Des. High Perform. Comput., 2014, pp. 9-16.
49. N. Madan, L. Zhao, N. Muralimanohar, A. Udipi, R. Balasubramonian, R. Iyer, S. Makineni, and D. Newell, "Optimizing communication and capacity in a 3D stacked reconfigurable cache hierarchy," in Proc. Int. Symp. High Perform. Comput. Archit., 2009, pp. 262-274.
50. B. Zhao, Y. Du, Y. Zhang, and J. Yang, "Variation-tolerant Nonuniform 3D cache management in die stacked multicore processor," in Proc. IEEE/ACM Int. Symp. Microarchit., 2009, pp. 222-231.
51. D. Jevdjic, G. H. Loh, C. Kaynak, and B. Falsafi, "Unison cache: A scalable and effective die-stacked DRAM cache," in Proc. Int. Symp. Microarchit., 2014, pp. 25-37.
52. C.-C. Huang and V. Nagarajan, "ATCache: Reducing DRAM cache latency via a small SRAM tag cache," in Proc. Int. Conf. Parallel Archit. Compilation, 2014, pp. 51-60.
53. D. Jevdjic, S. Volos, and B. Falsafi, "Die-stacked DRAM caches for servers: Hit ratio, latency, or bandwidth? Have it all with footprint cache," in Proc. Int. Symp. Comput. Archit., 2013, pp. 404-415.
54. K. Chang, G. H. Loh, M. Thottethodi, Y. Eckert, M. O'Connor, L. Subramanian, and O. Mutlu, "Enabling efficient dynamic resizing of large DRAM caches via a hardware consistent hashing mechanism," Carnegie Mellon Univ., Pittsburgh, PA, USA, Tech. Rep. 2013-001, 2013.
55. K. H. Park, S. K. Park, H. Seok, W. Hwang, D.-J. Shin, J. H. Choi, and K.-W. Park, "Efficient memory management of a hierarchical and a hybrid main memory for MN-MATE platform," in Proc. Int. Workshop Programm. Models Appl. Multicores Manycores, 2012, pp. 83-92.
56. X. Dong, X. Wu, G. Sun, Y. Xie, H. Li, and Y. Chen, "Circuit and microarchitecture evaluation of 3D stacking magnetic RAM (MRAM) as a universal memory replacement," in Proc. Des. Autom. Conf., 2008, pp. 554-559.
57. S. Lee, J. Jung, and C.-M. Kyung, "Hybrid cache architecture replacing SRAM cache with future memory technology," in Proc. IEEE Int. Symp. Circuits Syst., 2012, pp. 2481-2484.
58. C. Chou, A. Jaleel, and M. K. Qureshi, "BEAR: Techniques for mitigating bandwidth bloat in gigascale DRAM caches," in Proc. 42nd Annu. Int. Symp. Comput. Archit., 2015, pp. 198-210.
59. Z. Jaksic and R. Canal, "DRAM-based coherent caches and how to take advantage of the coherence protocol to reduce the refresh energy," in Proc. Des., Autom. Test Eur. Conf. Exhib., 2014, pp. 1-4.
60. W. Yun, J. Jung, K. Kang, and C.-M. Kyung, "Temperature-aware energy minimization of 3D-stacked L2 DRAM cache through DVFS," in Proc. Int. SoC Des. Conf., 2012, pp. 475-478.
61. M. Ghosh and H.-H. S. Lee, "Smart refresh: An enhanced memory controller design for reducing energy in conventional and 3D die-Stacked DRAMs," in Proc. Int. Symp. Microarchit., 2007, pp. 134-145.
62. Y. Pan and T. Zhang, "Improving VLIW processor performance using three-dimensional (3d) DRAM stacking," in Proc. IEEE Int. Conf. Appl.-Specific Syst., Archit. Process., 2009, pp. 38-45.
63. C. Anderson and J.-L. Baer, "Design and evaluation of a subblock cache coherence protocol for bus-based multiprocessors," Univ. of Washington, Seattle, WA, USA, Tech. Rep. 94-05-02, 1994.
64. S. Mittal and J. Vetter, "A survey of techniques for modeling and improving reliability of computing systems," IEEE Trans. Parallel Distrib. Syst., 2015, DOI: 10.1109/TPDS.2015.2426179.
65. N. H. Seong, S. Yeo, and H.-H. S. Lee, "Tri-level-cell phase change memory: Toward an efficient and reliable memory system," in Proc. Int. Symp. Comput. Archit., 2013, pp. 440-451.
66. S. Mittal, "A survey of power management techniques for phase change memory," Int. J. Comput. Aided Eng. Technol., 2014.
67. B. Giridhar, M. Cieslak, D. Duggal, R. Dreslinski, H. M. Chen, R. Patti, B. Hold, C. Chakrabarti, T. Mudge, and D. Blaauw, "Exploring DRAM organizations for energy-efficient and resilient exascale memories," in Proc. Int. Conf. High Perform. Comput., Netw., Storage Anal., 2013, p. 23.