Model-based average reward reinforcement learning

Artificial Intelligence

Volumn 100, Issue 1-2, 1998, Pages 177-224

  Tadepalli, Prasad ^a   Ok, DoKyeong ^b  

^a Oregon State University   (United States)

^b NonSanSi DuMaMyeon BuNamRi   (South Korea)

Author keywords

AGV scheduling; Average reward; Bayesian networks; Exploration; Linear regression; Machine learning; Model based; Reinforcement learning

Indexed keywords

COMPUTER SIMULATION; FUNCTIONS; MATHEMATICAL MODELS; OPTIMIZATION; PERFORMANCE; REGRESSION ANALYSIS; STATE SPACE METHODS;

BAYESIAN NETWORK; LINEAR REGRESSION; REINFORCEMENT LEARNING;

LEARNING SYSTEMS;

EID: 0032050241     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/s0004-3702(98)00002-2     Document Type: Article

References (47)

1. D.W. Aha, D. Kibler, M.K. Albert, Instance-based learning algorithms, Machine Learning 6 (1991) 37-66.
2. C.G. Atkeson, A.W. Moore, S. Schaal, Locally weighted learning, Artificial Intelligence Review 11 (1997) 11-73.
3. A.G. Barto, S.J. Bradtke, S.P. Singh, Learning to act using real-time dynamic programming. Artificial Intelligence 73 (1995) 81-138.
4. D. Bertsekas, A new value-iteration method for the average cost dynamic programming problem, Technical Report LIDS-P-2307, MIT, Boston, MA, 1995.
5. D.P. Bertsekas, Distributed dynamic programming, IEEE Trans. Automatic Control 27 (3) (1982).
6. D. P. Bertsekas, Dynamic Programming and Optimal Control, Athena Scientific, Belmont, MA, 1995.
7. C. Boutilier, R. Dearden, M. Goldszmidt, Exploiting structure in policy construction, in: Proceedings 14th International Joint Conference on Artificial Intelligence (IJCAI-95), Montreal, Que., 1995.
8. J. Boyan, A.W. Moore, Generalizing reinforcement learning: safely approximating the value function, in: Proceedings Neural Information Processing Systems, 1994.
9. L. Brieman, J.H. Friedman, R.A. Olshen, C.J. Stone, Classification and Regression Trees, Wadsworth International Group, Belmont, MA, 1984.
10. G.C. Canavos, Applied Probability and Statistical Methods, Little, Brown and Company, Boston, MA, 1984.
11. R.H. Crites, A.G. Barto, Improving elevator performance using reinforcement learning, in: Advances in Neural Information Processing Systems, Vol. 8, MIT Press, Cambridge, MA, 1996.
12. T. Dean, K. Kanazawa, A model for reasoning about persistence and causation, Computational Intelligence 5 (3) (1989) 142-150.
13. G.W. Gates, The reduced nearest neighbor rule, IEEE Trans. Inform. Theory (1972) 431-433.
14. P.E. Hart, The condensed nearest neighbor rule, IEEE Trans. Inform. Theory 14 (1968) 515-516.
15. R.A. Howard, Dynamic Programming and Markov Processes, MIT Press and Wiley, Cambridge, MA, 1960.
16. A. Jalali, M. Ferguson, Computationally efficient adaptive control algorithms for markov chains, in: IEEE Proceedings 28th Conference on Decision and Control, Tampa, FL, 1989.
17. L.P. Kaelbling, Learning in Embedded Systems, MIT Press, Cambridge, MA, 1990.
18. L.P. Kaelbling, M.L. Littman, A.W. Moore, Reinforcement learning: a survey, J. Artificial Intelligence Research 4 (1996) 237-285.
19. U. Kjaerulff, A computational scheme for reasoning in dynamic probabilistic networks, in: Proceedings 8th Conference on Uncertainty in Artificial Intelligence (1992) 121-129.
20. S. Koenig, R.G Simmons, The effect of representation and knowledge on goal-directed exploration with reinforcement-learning algorithms, Machine Learning 22 (1996) 227-250.
21. L-J. Lin, Self-improving reactive agents based on reinforcement learning, planning, and teaching, Machine Learning 8 (1992) 293-321.
22. M.L. Littman, A. Cassandra, L.P. Kaelbling, Learning policies for partially observable environments: scaling up, in: Proceedings of International Machine Learning Conference, San Fransisco, CA, 1995, pp. 362-370.
23. S. Mahadevan, An average reward reinforcement learning algorithm for computing bias-optimal policies, in: Proceedings National Conference on Artificial Intelligence (AAAI-96), Portland, OR, 1996.
24. S. Mahadevan, Average reward reinforcement learning: foundations, algorithms, and empirical results, Machine Learning 22 (1996) 159-195.
25. S. Mahadevan, Sensitive discount optimality: Unifying discounted and average reward reinforcement learning, in: Proceedings International Machine Learning Conference, Bari, Italy, 1996.
26. S. Mahadevan, J. Connell, Automatic programming of behavior-based robots using reinforcement learning, Artificial Intelligence 55 (1992) 311-365.
27. W.L. Maxwell, J.A. Muckstadt, Design of automatic guided vehicle systems, Institute of Industrial Engineers Trans. 14 (2) (1982) 114-124.
28. A.W. Moore, A.G. Atkeson, Prioritized sweeping: Reinforcement learning with less data and less time, Machine Learning J. 13 (1993) 103-130.
29. A.W. Moore, C.G. Atkeson, S. Schaal, Locally weighted learning for control, Artificial Intelligence Review 11 (1997) 75-113.
30. A.E. Nicholson, J.M. Brady, The data association problem when monitoring robot vehicles using dynamic belief networks, in: ECAI 92: 10th European Conference on Artificial Intelligence Proceedings, Vienna, Austria, Wiley, New York, 1992, pp. 689-693.
31. D. Ok, A study of model-based average reward reinforcement learning, Ph.D. Thesis, Technical Report, 96-30-2, Department of Computer Science, Oregon State University, Corvallis, OR, 1996.
32. D. Ok, P. Tadepalli, Auto-exploratory average reward reinforcement learning, in: Proceedings National Conference on Artificial Intelligence (AAAI-96), Portland, OR, 1996.
33. R. Parr, S. Russell, Approximating optimal policies for partially observable stochastic domains, in: Proceedings National Conference on Artificial Intelligence (AAAI-94), Seattle, WA, 1994, pp. 1088-1093.
34. M.L. Puterman, Markov Decision Processes: Discrete Dynamic Stochastic Programming, John Wiley, New York, 1994.
35. S. Russell, P. Norvig, Artificial Intelligence: A Modern Approach, Prentice-Hall, Englewood Cliffs, NJ, 1995.
36. S. Schaal, C. Atkeson, Robot juggling: an implementation of memory-based learning, in: IEEE Control Systems, Vol. 14, 1994, pp. 57-71.
37. A. Schwartz, A reinforcement learning method for maximizing undiscounted rewards, in: Proceedings 10th International Conference on Machine Learning, Morgan Kaufmann, San Mateo, CA, 1993.
38. S.P. Singh, Reinforcement learning algorithms for average-payoff markovian decision processes, in: Proceedings National Conference on Artificial Intelligence (AAAI-94), Seattle, WA, MIT Press, Cambridge, MA, 1994.
39. R.S. Sutton, Learning to predict by the methods of temporal differences, Machine Learning 3 (1988) 9-44.
40. R.S. Sutton, Integrating architectures for learning, planning and reacting based on approximating dynamic programming, in: Proceedings Seventh International Conference on Machine Learning, Austin, TX, 1990.
41. P. Tadepalli, D. Ok, H-learning: A reinforcement learning method for optimizing undiscounted average reward, Technical Report 94-30-1, Department of Computer Science, Oregon State University, 1994.
42. P. Tadepalli, D. Ok, Scaling up average reward reinforcement learning by approximating the domain models and the value function, in: Proceedings 13th International Conference on Machine Learning, 1996.
43. M. Tan, Multi-agent reinforcement learning: independent vs. cooperative agents, in: Proceedings 10th International Conference on Machine Learning, Morgan Kaufmann, San Mateo, CA, 1993.
44. G. Tesauro, Practical issues in temporal difference learning, Machine Learning 8 (3-4) (1992) 257-277.
45. S. Thrun, The role of exploration in learning control, in: Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, Van Nostrand Reinhold, New York, 1994.
46. C.J.C.H. Watkins, P. Dayan, Q-learning, Machine Learning 8 (1992) 279-292.
47. W. Zhang, T. Dietterich, A reinforcement learning approach to job-shop scheduling, in: Proceedings 14th International Joint Conference on Artificial Intelligence (IJCAI-95), Montreal, Que., 1995.