Reinforcement learning in autonomic computing: A manifesto and case studies

IEEE Internet Computing

Volumn 11, Issue 1, 2007, Pages 22-30

  Tesauro, Gerald ^a  

^a IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

Autonomic computing; Reinforcement learning; Systems management; Training

Indexed keywords

AUTONOMIC COMPUTING; SYSTEMS MANAGEMENT;

ARTIFICIAL INTELLIGENCE; MANAGEMENT; PERSONNEL TRAINING;

REINFORCEMENT LEARNING;

EID: 33847379922     PISSN: 10897801     EISSN: None     Source Type: Journal    
DOI: 10.1109/MIC.2007.21     Document Type: Article

References (16)

1. D.A, Menascé, V.A.F. Almedia, and L.W. Dowdy, Performance by Design: Computer Capacity Planning by Example, Prentice Hall, 2004.
2. J.L. Hellerstein et al., Feedback Control of Computing Systems, Wiley & Sons, 2004.
3. J.O. Kephart and D.M. Chess, "The Vision of Autonomic Computing," Computer, vol. 36, no. 1, 2003, pp. 41-52.
4. B. Urgaonkar et al., "An Analytical Model for Multi-Tier Internet Services and Its Applications," Proc. Sigmetrics, ACM Press, 2005, pp. 291-302.
5. K.J. Astrom and B. Wittenmark, Adaptive Control, 2nd ed., Prentice Hall, 1994.
6. R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction, MIT Press, 1998.
7. G. Tesauro, "Temporal Difference Learning and TD-Gammon," Comm. ACM, vol. 38, no. 3, 1995, pp. 58-68.
8. J. Moody and M. Saffell, "Learning to Trade via Direct Reinforcement," IEEE Trans. Neural Networks, vol. 12, no. 4, 2001, pp. 875-889.
9. A.Y. Ng et al., "Inverted Autonomous Helicopter Flight via Reinforcement Learning," Proc. Int'l Symp. Experimental Robotics, 2004; http://ai.stanford.edu/~ang/papers/iser04-invertedflight.pdf.
10. L. Baird, "Residual Algorithms: Reinforcement Learning with Function Approximation," Proc. Int'l Conf. Machine Learning (ICML), ACM Press, 1995, pp. 30-37.
11. M.S. Squillante, D.D. Yao, and L. Zhang, "Internet Traffic: Periodicity, Tail Behavior, and Performance Implications," System Performance Evaluation: Methodologies and Applications, E. Gelenbe, ed., CRC Press, 1999, pp. 23-37.
12. G. Tesauro et al., "A Hybrid Reinforcement Learning Approach to Autonomic Resource Allocation," Proc. Int'l Conf. Autonomic Computing (ICAC), IEEE CS Press, 2006, pp. 65-73.
13. G. Tesauro, "Online Resource Allocation Using Decompositional Reinforcement Learning," Proc. AAAI, 2005, pp. 886-891.
14. D. Vengerov and N. Iakovlev, "A Reinforcement Learning Framework for Dynamic Resource Allocation: First Results," Proc. Int'l Conf. Autonomic Computing (ICAC), IEEE CS Press, 2005, pp. 339-340.
15. S. Whiteson and P. Stone, "Adaptive Job Routing and Scheduling," Eng. Applications of Artificial Intelligence, vol. 17, no. 7, 2004, pp. 855-869.
16. D. Vengerov, A Reinforcement Learning Framework for Utility-based Scheduling in Resource-Constrained Systems, tech. report TR-2005-141, Sun Microsystems, 2005.