Policy improvement by a model-free Dyna architecture

IEEE Transactions on Neural Networks and Learning Systems

Volumn 24, Issue 5, 2013, Pages 776-788

  Hwang, Kao Shing ^a   Lo, Chia Yue ^b  

^a NATIONAL SUN YAT SEN UNIVERSITY   (Taiwan)

^b NATIONAL CHUNG CHENG UNIVERSITY   (Taiwan)

Author keywords

Critic actor structure; Dyna style reinforcement learning; POMDP; Temporal difference

Indexed keywords

ACTOR-CRITIC ARCHITECTURES; ADAPTIVE HEURISTIC CRITICS; DESIRED TRAJECTORIES; INDIRECT LEARNING; POMDP; REINFORCEMENT SIGNAL; TEMPORAL DIFFERENCE METHODS; TEMPORAL DIFFERENCES;

HEURISTIC METHODS; PENDULUMS; REINFORCEMENT LEARNING;

COMPUTER SIMULATION;

EID: 84884963190     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2013.2244100     Document Type: Article

References (18)

1. R. S. Sutton, D. McAllester, S. Singh, and Y. Mansour, "Policy-gradient methods for reinforcement learning with function approximation," Adv. Neural Inf. Process. Syst., vol. 12, no. 22, pp. 1057-1063, 2000.
2. K. S. Hwang and C. S. Lin, "Smoothing trajectory tracking of three-link robot: A self-organizing CMAC approach," IEEE Trans. Syst., Man, Cybern.-Part B, Cybern., vol. 28, no. 5, pp. 680-692, Oct. 1998.
3. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, Cambridge, MA, USA: MIT Press, 1998.
4. M. A. Wiering and H. V. Hasselt, "Ensemble algorithms in reinforcement learning," IEEE Trans. Syst., Man, Cybern.-Part B, Cybern., vol. 38, no. 4, pp. 930-936, Aug. 2008.
5. G. G. Lendaris, "Higher level application of adp: A next phase for the control field?" IEEE Trans. Syst., Man, Cybern.-Part B, Cybern., vol. 38, no. 4, pp. 901-912, Aug. 2008.
6. D. Dong, C. Chen, H. Li, and T. Tarn, "Quantum reinforcement learning," IEEE Trans. Syst., Man, Cybern.-Part B, Cybern., vol. 38, no. 5, pp. 1207-1220, Oct. 2008.
7. B. Baddeley, "Reinforcement learning in continuous time and space: Interference and not ill conditioning is the main problem when using distributed function approximators," IEEE Trans. Syst., Man, Cybern.-Part B, Cybern., vol. 38, no. 4, pp. 950-956, Aug. 2008.
8. S. Dries and M. A. Wiering, "Neural-Fitted TD-Leaf Learning for Playing Othello with Structured Neural Networks," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 11, pp. 1701-1713, Nov. 2012.
9. H.-N. Wu and B. Luo, "Neural network based Online simultaneous policy update algorithm for solving the HJI equation in nonlinear H∞ Control," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 12, pp. 1884-1895, Dec. 2012.
10. S. P. Singh, "Reinforcement learning algorithms for average-payoff markovian decision processes," in Proc. 12th Amer. Assoc. Artif. Intell., 1994, pp. 700-705.
11. J. J. Govindhasamy, S. F. McLoone, and G. W. Irwin, "Second-order training of adaptive critics for Online process control," IEEE Trans. Syst., Man, Cybern.-Part B, Cybern., vol. 35, no. 2, pp. 381-385, Apr. 2005.
12. R. Zajdel, "Epoch-incremental queue-dyna algorithm," in Proc. Lect. Notes Artif. Intell., pp. 1160-1170, 2008.
13. R. Sutton, "Dyna, an integrated architecture for learning, planning, and reacting," Special Interest Group Artif. intell. Bulletin, vol. 2, no. 4, pp. 160-163, Aug. 1991.
14. J. N. Tsitsiklis and B. Van Roy, "On average versus discounted reward temporal-difference learning," Mach. Learn., vol. 49, no. 2, pp. 179-191, 2002. (Pubitemid 34325685)
15. N. D. Daw and D. S. Touretzky, "Behavioral considerations suggest an average reward TD model of the dopamine system," Neurocomputing, pp. 679-684, 2000.
16. V. Gullapalli, "A stochastic reinforcement learning algorithm for learning real-valued functions," Neural Netw., vol. 3, no. 6, pp. 671-692, 1990.
17. H. D. Patino and D. Liu, "Neural Network-Based Model Reference Adaptive Control System," IEEE Trans. Syst., Man, Cybern.-Part B, Cybern., vol. 30, no. 1, pp. 198-204, Feb. 2000. (Pubitemid 30588328)
18. J. G. Ziegler and N. B. Nichols, "Optimum settings for automatic controllers," Trans. of the ASME, vol. 64, no. 11, pp. 759-768, 1942.