Self-optimizing memory controllers: A reinforcement learning approach

Proceedings - International Symposium on Computer Architecture

Volumn , Issue , 2008, Pages 39-50

  Ipek, Engin ^a,b   Mutlu, Onur ^b   Martínez, José F ^a   Caruana, Rich ^a  

^a Department of Obstetrics Gynecology   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER SCIENCE; COMPUTER SYSTEMS; COMPUTERS; COST EFFECTIVENESS; DYNAMIC RANDOM ACCESS STORAGE; NANOTECHNOLOGY; OPTIMIZATION; REINFORCEMENT; REINFORCEMENT LEARNING; SCHEDULING; TELECOMMUNICATION SYSTEMS;

AVERAGE-CASE; BANDWIDTH UTILIZATION; CHIP MULTI PROCESSORS; INTERNATIONAL SYMPOSIUM; LONG-TERM PERFORMANCE; MEMORY CONTROLLERS; OFF-CHIP; ON THE FLY; PARALLEL APPLICATIONS; REINFORCEMENT LEARNING APPROACH; SCHEDULING DECISIONS; SCHEDULING POLICIES; SELF-OPTIMIZING; SYSTEM STATES;

COMPUTER ARCHITECTURE;

EID: 52649148744     PISSN: 10636897     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ISCA.2008.21     Document Type: Conference Paper

References (49)

1. Advanced Micro Devices, Inc. AMD Athlon(™) XP Processor Model 10 Data Sheet, Feb. 2003.
2. J. H. Ahn, M. Erez, and W. J. Dally. The design space of data-parallel memory systems. In SC, 2006.
3. Anandtech. Intel Developer Forum 2007. http://www.anandtech.com/ cpuchipsets/intel/showdoc.aspx?i=3102.
4. V. Aslot and R. Eigenmann. Quantitative performance analysis of the SPEC OMPM2001 benchmarks. Scientific Programming, 11(2):105-124, 2003.
5. D. H. Bailey et al. NAS parallel benchmarks. Technical report, NASA Ames Research Center, March 1994. Tech. Rep. RNR-94-007.
6. D. Bertsekas. Neuro Dynamic Programming. Athena Scientific, Belmont, MA, 1996.
7. B. Calder, D. Grunwald, M. Jones, D. Lindsay, J. Martin, M. Mozer, and B. Zorn. Evidence-based static branch prediction using machine learning. ACM TOPLAS, 19(1), 1997.
8. J. Carter et al. Impulse: Building a smarter memory controller. In HPCA-5, 1999.
9. R. Caruana and D. Freitag. Greedy attribute selection. In International Conference on Machine Learning, pages 28-36, New Brunswick, NJ, July 1994.
10. Y. Chang, T. Hoe, and L. Kaelbling. Mobilized ad-hoc networks: A reinforcement learning approach. In International Conference on Autonomic Computing, 2004.
11. V. Cuppu, B. Jacob, B. T. Davis, and T. Mudge. A performance comparison of contemporary DRAM architectures. In ISCA-26, 1999.
12. B. T. Davis. Modern DRAM Architectures. Ph.D. dissertation, Dept. of EECS, University of Michigan, Nov. 2000.
13. T. Dunigan, M. R. Fahey, J. White, and P. Worley. Early evaluation of the Cray X1. In SC, 2003.
14. J. Emer and N. Gloy. A language for describing predictors and its application to automatic synthesis. In ISCA-24, 1997.
15. M. Ghosh and H.-H. S. Lee. Smart refresh: An enhanced memory controller design for reducing energy in conventional and 3D Die-Stacked DRAMs. In MICRO-40, 2007.
16. S. I. Hong, S. A. McKee, M. H. Salinas, R. H.Klenke, J. H. Aylor, and W. A. Wulf. Access order and effective bandwidth for streams on a direct Rambus memory. In HPCA-5, 1999.
17. I. Hur and C. Lin. Adaptive history-based memory schedulers. In MICRO-37, 2004.
18. ITRS. International Technology Roadmap for Semiconductors: 2005 Edition, Assembly and packaging. http://www.itrs.net/Links/2005ITRS/AP2005. pdf.
19. D. A. Jimenez and C. Lin. Dynamic branch prediction with perceptrons. In HPCA-7, 2001.
20. M. Joshi, G. Karypis, and V. Kumar. ScalParC: A new scalable and efficient parallel classification algorithm for mining large datasets. In IPPS, 1998.
21. R. E. Kessler. The Alpha 21264 microprocessor. IEEE Micro, 9(2):24-36, Mar. 1999.
22. P. Kongetira, K. Aingaran, and K. Olukotun. Niagara: A 32-way multithreaded sparc processor. IEEE Micro, 25(2):21-29, 2005.
23. J. D. McCalpin. Sustainable Memory Bandwidth in Current High Performance Computers. http://home.austin.rr.com/mccalpin/papers/bandwidth/.
24. A. McGovern and E. Moss. Scheduling straight line code using reinforcement learning and rollouts. In Advances in Neural Information Processing Systems, 1999.
25. S. A. McKee et al. Dynamic access ordering for streamed computations. IEEE Transactions on Computers, 49(11):1255-1271, Nov. 2000.
26. Micron. 512Mb DDR2 SDRAM Component Data Sheet: MT47H128M4B6-25, March 2006. http://download.micron.com/pdf/datasheets/dram/ddr2/512MbDDR2.pdf.
27. Micron. Technical Note TN-47-04: Calculating Memory System Power for DDR2, June 2006. http://download.micron.com/pdf/technotes/ddr2/TN4704.pdf.
28. T. Mitchell. Machine Learning. McGraw-Hill, Boston, MA, 1997.
29. O. Mutlu and T. Moscibroda. Stall-time fair memory access scheduling for chip multiprocessors. In MICRO-40, 2007.
30. C. Natarajan, B. Christenson, and F. Briggs. A study of performance impact of memory controller features in multi-processor server environment. In WMPI, 2004.
31. K. J. Nesbit, N. Aggarwal, J. Laudon, and J. E. Smith. Fair queuing memory systems. In MICRO-39, 2006.
32. J. Nie and S. Haykin. A Q-learning-based dynamic channel assignment technique for mobile communication systems. In IEE Transactions on Vehicular Technology, Sept. 1999.
33. J. Pisharath, Y. Liu, W. Liao, A. Choudhary, G. Memik, and J. Parhi. NU-MineBench 2.0. Technical report, Northwestern University, August 2005. Tech. Rep. CUCIS-2005-08-01.
34. N. Rafique, W.-T. Lim, and M. Thottethodi. Effective management of DRAM bandwidth in multicore processors. In PACT, 2007.
35. J. Renau, B. Fraguela, J. Tuck, W. Liu, M. Prvulovic, L. Ceze, S. Sarangi, P. Sack, K. Strauss, and P. Montesinos. SESC simulator, January 2005. http://sesc.sourceforge.net.
36. S. Rixner. Memory controller optimizations for web servers. In MICRO-37, 2004.
37. S. Rixner, W. J. Dally, U. J. Kapasi, P. Mattson, and J. D. Owens. Memory access scheduling. In ISCA-27, 2000.
38. J. Shao and B. T. Davis. A burst scheduling access reordering mechanism. In HPCA-13, 2007.
39. B. Sinharoy, R. N. Kalla, J. M. Tendler, R. J. Eickemeyer, and J. B. Joyner. POWER5 system microarchitecture. IBM Journal of Research and Development, 49(4/5):505-521, 2005.
40. L. Spracklen and S. G. Abraham. Chip multithreading: Opportunities and challenges. In HPCA-11, 2005.
41. R. Sutton. Generalization in reinforcement learning: Successful examples using sparse coarse coding. In Neural Information Processing Systems Conference, Denver, CO, June 1996.
42. R. Sutton and A. Barto. Reinforcement Learning. MIT Press, Cambridge, MA, 1998.
43. G. Tesauro et al. Online resource allocation using decompositional reinforcement learning. In Conference for the American Association for Artificial Intelligence, July 2005.
44. M. Valero et al. Increasing the number of strides for conflict-free vector access. In ISCA-19, 1992.
45. D. Vengerov and N. Iakovlev. A reinforcement learning framework for dynamic resource allocation: First results. In ICAC, 2005.
46. S. C. Woo, M. Ohara, E. Torrie, J. P. Singh, and A. Gupta. The SPLASH-2 programs: Characterization and methodological considerations. In ISCA-22, 1995.
47. Z. Zhang et al. A permutation-based page interleaving scheme to reduce row-buffer conflicts and exploit data locality. In MICRO-33, 2000.
48. Z. Zhu and Z. Zhang. A performance comparison of DRAM memory system optimizations for SMT processors. In HPCA-11, 2005.
49. W. K. Zuravleff and T. Robinson. Controller for a synchronous DRAM that maximizes throughput by allowing memory requests and commands to be issued out of order. United States Patent #5,630,096, May 1995.