Two-time-scale online actor-critic paradigm driven by POMDP

2010 International Conference on Networking, Sensing and Control, ICNSC 2010

Volumn , Issue , 2010, Pages 243-248

  Liu, Bo ^a   He, Haibo ^a,b   Repperger, Daniel W ^c  

^a Stevens Institute of Technology   (United States)

^b University of Rhode Island   (United States)

^c AIR FORCE RESEARCH LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACTOR CRITIC; ACTOR-CRITIC ALGORITHM; ADAPTIVE DYNAMIC PROGRAMMING; ANALYSIS AND SIMULATION; GREEDY ALGORITHMS; HIDDEN STATE; NONLINEAR FUNCTIONS; PARTIALLY OBSERVABLE MARKOV DECISION PROCESS; STATE ESTIMATORS; STOCHASTIC GRADIENT; TEMPORAL DIFFERENCES; TWO TIME SCALE;

MARKOV PROCESSES; STATE ESTIMATION;

NEURAL NETWORKS;

EID: 77953105620     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICNSC.2010.546149     Document Type: Conference Paper

References (16)

1. L. P. Kaelbling, M. L. Littman, and A. W. Moore, "Reinforcement learning: A survey," Journal of Artificial Intelligence Research, vol. 4, pp. 237-285, 1996.
2. E. Alpaydin, Introduction to Machine Learning. MIT Press, October 2004.
3. T. Jaakkola, S. P. Singh, and M. I. Jordan, "Reinforcement learning algorithm for partially observable markov decision problems," in Advances in Neural Information Processing Systems 7. MIT Press, 1995, pp. 345-352.
4. S. P. Singh, T. Jaakkola, and M. I. Jordan, "Learning without stateestimation in partially observable markovian decision processes," in In Proceedings of the Eleventh International Conference on Machine Learning. Morgan Kaufmann, 1994, pp. 284-292.
5. S. D. Whitehead and L. J. Lin, "Reinforcement learning in non-markov environments," Artificial Intelligence, vol. 8, pp. 3-4, 1992.
6. L. ji Lin and T. M. Mitchell, "Memory approaches to reinforcement learning in non-markovian domains," Tech. Rep., 1992.
7. M. Hauskrecht, "Value-function approximations for partially observable markov decision processes," Journal of Artificial Intelligence Research, vol. 13, pp. 33-94, 2000.
8. K. P. Murphy, "A Survey of POMDP Solution Techniques," Tech. Rep., 2000.
9. J. Si and Y.-T. Wang, "Online learning control by association and reinforcement," IEEE Transactions on Neural Networks, vol. 12, no. 2, pp. 264-276, Mar 2001.
10. T. Jaakkola, M. I. Jordan, and S. P. Singh, "Convergence of stochastic iterative dynamic programming algorithms," Neural Computation, vol. 6, pp. 1185-1201, 1994.
11. J. Tsitsiklis, D. Bertsekas, and M. Athans, "Distributed asynchronous deterministic and stochastic gradient optimization algorithms," IEEE Transactions on Automatic Control, vol. 31, no. 9, pp. 803-812, Sep 1986.
12. K. Miroslav Krstic, Petar V. Kokotovic, Nonlinear and Adaptive Control Design. Wiley-Interscience, 1995.
13. V. Konda and J. Tsitsiklis, "Actor-critic algorithms," in SIAM Journal on Control and Optimization. MIT Press, 2000, pp. 1008-1014.
14. E. Eweda and O. Macchi, "Convergence of an adaptive linear estimation algorithm," IEEE Transactions on Automatic Control, vol. 29, no. 2, pp. 119-127, Feb 1984.
15. V. R. Konda and J. N. Tsitsiklis, "Linear stochastic approximation driven by slowly varying markov chains," Systems and Control Letters, vol. 50, pp. 95-102, 2003.
16. L. Aguilar, I. Boiko, L. Fridman, and R. Iriarte, "Generating self-excited oscillations via two-relay controller," IEEE Transactions on Automatic Control, vol. 54, no. 2, pp. 416-420, Feb. 2009.