Tracking value function dynamics to improve reinforcement learning with piecewise linear function approximation

ACM International Conference Proceeding Series

Volumn 227, Issue , 2007, Pages 751-758

  Phua, Chee Wee ^a   Fitch, Robert ^a  

^a UNIVERSITY OF NEW SOUTH WALES   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; ERROR CORRECTION; FUNCTION EVALUATION; KALMAN FILTERS; LINEAR SYSTEMS;

MAXQ ALGORITHMS; MEAN-SQUARE BELLMAN ERROR; Q-LEARNING; RANDOM-WALK MODEL;

REINFORCEMENT LEARNING;

EID: 34547974097     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1273496.1273591     Document Type: Conference Paper

References (18)

1. Baird, L. (1995). Residual algorithms: Reinforcement learning with function approximation. Twelfth International Conference on Machine Learning (pp. 30-37). San Francisco: Morgan Kaufman Publishers.
2. Boyan, J. (2002). Technical update: Least-squares temporal difference learning. Machine Learning, 49, 233-246.
3. Bradtke, S. J., amp; Barto, A. G. (1996). Linear least-squares algorithms for temporal difference learning. Machine Learning, 22, 33-57.
4. Choi, D., amp; Roy, B. V. (2006). A generalized kalman filter for fixed point approximation and efficient temporal difference learning. Discrete Event Dynamic Systems, 16, 207-239.
5. Dietterieh, T. G. (2000). Hierarchical reinforcement learning with the MAXQ value function decomposition. Journal of Artificial Intelligence Research, 13, 227-303.
6. Geramifard, A., Bowling, M., amp; Sutton, R. S. (2006). Incremental least-squares temporal difference learning. Twenty-First National Conference on Artificial Intelligence (AAAI-06) (pp. 356-361).
7. Haykin, S. (2001). Adaptive filter theory (4th edition). Prentice Hall.
8. Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Transactions of the ASME - Journal of Basic Engineering, 82, 35-45.
9. Lagoudakis, M. G., amp; Parr, R. (2003). Least-squares policy iteration. The Journal of Machine Learning Research, 4, 1107-1149.
10. Munos, R., amp; Moore, A. (2002). Variable resolution discretization in optimal control. Machine Learning, 49, 291-323.
11. Perkins, T. J., amp; Precup, D. (2002). A convergent form of approximate policy iteration. Neural Information Processing Systems (pp. 1595-1602). Vancouver, British Columbia, Canada: MIT Press.
12. Potts, D., amp; Sammut, C. (2005). Incremental learning of linear model trees. Machine Learning, 61, 5-48.
13. Simon, D. (2006). Optimal state estimation: Kalman, H-infinity, and nonlinear approaches. Wiley-Interscience.
14. Soderstrom, T., amp; Stoica, P. (2002). Instrumental variable methods for system identification. Circuits Systems Signal Processing, 21, 1-9.
15. Sutton, R. S., amp; Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.
16. Tsitsiklis, J. N., amp; Roy, B. V. (1997). An analysis of temporal-difference learning with function approximation. IEEE Transactions on Automatic Control, 42, 674-690.
17. Widrow, B., amp; Hoff, M. E. (1960). Adaptive switching circuits. IRE WESCON Convention Record (pp. 96-104). New York.
18. Xu, X., gen He, H., & Hu, D. (2002). Efficient reinforcement learning using recursive least squares methods. Journal of Artificial Intelligence Research, 16, 259-292.