A survey of approximate dynamic programming

2009 International Conference on Intelligent Human-Machine Systems and Cybernetics, IHMSC 2009

Volumn 2, Issue , 2009, Pages 396-399

  Wang, Lin ^a   Peng, Hui ^a   Zhu, Hua Yong ^a   Shen, Lin Cheng ^a  

^a NATIONAL UNIVERSITY OF DEFENSE TECHNOLOGY   (China)

Author keywords

Approximate dynamic programming; Dynamic programming; Markov decision processes; Reinforcement learning

Indexed keywords

APPROXIMATE DYNAMIC PROGRAMMING; COMPUTATIONAL REQUIREMENTS; DECISION PROBLEMS; FUNCTION APPROXIMATION; IN-PROCESS; MARKOV DECISION PROCESSES; MATHEMATICAL FORMULATION; MULTI-STAGE; REAL APPLICATIONS; RESEARCH DIRECTIONS; STANDARD METHOD; STATE SPACE;

CYBERNETICS; LEARNING ALGORITHMS; MARKOV PROCESSES; REINFORCEMENT; REINFORCEMENT LEARNING; SURVEYS;

DYNAMIC PROGRAMMING;

EID: 73649096185     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IHMSC.2009.222     Document Type: Conference Paper

References (37)

1. P. Werbos, "ADP: Goals, Opportunities and Principles," HANDBOOK of LEARNING and APPROXIMATE DYNAMIC PROGRAMMING, IEEE Press John Wiley & sons, Inc. 2004, pp.1-42.
2. W. B. Powell and B. Van Roy, "Approximate Dynamic Programming for High-Dimensional Resource Allocation Problems," HANDBOOK of LEARNING and APPROXIMATE DYNAMIC PROGRAMMING, IEEE Press John Wiley & sons, Inc. 2004, pp.261-284
3. P. Werbos, "Stable Adaptive Control Using New Critic Designs," 1998 , (ArXiv.org: adaporg/9810001).
4. P. Werbos, "Advanced forecasting for global crisis warning and models of intelligence," General Systems Yearbook, 1977.
5. P. Werbos, "Neural networks for control and system identification," IEEE Proceeding of Conference on Decision and Control, 1989.
6. B. Widrow, N, Gupta and S. Maitra, "Punish/reward: learning with a Critic in adaptive threshold systems," IEEE Trans. SMC, vol. 5, 1973, pp.455-465.
7. A. Barto, R. Sutton and C. Anderson, "Neuronlike adaptive elements that can solve difficult learning control problems," IEEE Trans. SMC, vol. 13, 1983, pp.834-846.
8. D. Dimitri, P. Bertsekas, "Dynamic Programming and Suboptimal Control: A Survey from ADP to MPC," 2005, Report LIDS 2632.
9. Xin Xu, Han-gen He and Dewen Hu, "Efficient Reinforcement Learning Using Recursive Least-Squares Methods," Journal of Artificial Intelligence Research , Vol.16 , 2002, pp.259-292.
10. P. D. Dayan, "The Convergence of TD(λ) for General λ," Machine Learning, vol. 8, 1992, pp.341-362.
11. John N. Tsitsiklis and Benjamin Van Roy, "An Analysis of Temporal-Difference Learning with Function Approximation" Van Roy's homepages , 1997.
12. T. T. Shannon, R. A. Santiago and G. Lendaris, "Accelerating Critic Learning in Approximate Dynamic Programming Via Value Templates and Perceptual Learning," IEEEE 0-7803-7898-9/03, 2003, pp.2922-2927
13. S. Mohagheghi and Ganesh K. Venayagamoorthy, "Adaptive Critic Design Based Neuro-Fuzzy Controller for a Static Compensator in a Multimachine Power System," IEEE Transactions on Power Syatems, vol. 21, NO. 4, 2006 pp.1744-1755.
14. X.-R. Cao, "Learning and optimization - from a system theoretic perspective," Hanudbook of Learning and Approximnate Dynamic Programmning: Scaling up to the Real World, Eds. Wiley-IEEE Press, 2004.
15. J. L. Williams, J. W. Fisher and A. S. Willsky, "An Approximate Dynamic Programming Approach to a Communication Constrained Sensor Management Problem," 2005 7th International Conference on Information Fusion, 2005, pp.582-560.
16. H. Javaherian, D. Liu, and Olesia Kovalenko, "Automotive Engine Torque and Air-Fuel Ratio Control Using Dual Heuristic Dynamic Programming," 2006 International Joint Conference on Neural Networks, 2006, pp.518-526.
17. H. Javaherian, D. Liu, Y. Zhang and O. Kovalenko, "Adaptive critic learning techniques for automotive engine control," Proceedings of the American Control Conference, ,2004, pp.4066-4071.
18. O. Kovalenko, D. Liu, and H. Javaherian, "Neural network modeling and adaptive critic control of automotive fuel-injection systems," Proceedings of the IEEE International Symposium on Intelligent Control, 2004, pp.386-373.
19. R. Ilin, R. Kozma and P. Werbos, "Cellular SRN Trained by Extended Kalman Filter Shows Promise for ADP," 2006 International Joint Conference on Neural Networks, 2006, pp.506-511.
20. H. Lakshmanan and D. P. Farias, "Decentralized Approximate Dynamic Programming for Dynamic Networks of Agents," Proceedings of the 2006 American Control Conference, 2006, pp.1648-1654.
21. A. Rantzer, "On Approximate Dynamic Programming in Switching Systems," 44th IEEE Conference on Decision and Control, and the European Control Conference, 2006, pp.1391-1397.
22. B. Lincoln and A. Rantzer, "Relaxing Dynamic Programming," IEEE Transactions on Automatic Control, vol.51, No. 8, 2006, pp.249-1261.
23. Ju Jiang and M. S. Kamel, "Aggregation of Reinforcement Learning Algorithms," 2006 International Joint Conference on Neural Networks, 2006.
24. Ju Jiang, M. Kamel and Lei Chen, "Reinforcement Learning and Aggregation," Proceedings of IEEE International Conference on Systems, Man, and Cybernetics 04, 2004, pp.1303-1308.
25. C. J, C. H. Watkins and P. Dayan, " Q-learning ", Machine Learning, vol. 8, no. 3, 1992, pp.279-292.
26. R. S. Sutton and A. G. Barto, "Reinforcement Learning, An Introduction." A Bradford Book, The MIT Press, Cambridge, Massachusetts, London, England, ISBN 0-262-19398-1, 1998.
27. L. P. Kaelbling, M. L. Littman and A. W. Moore, "Reinforcement Learning: A Survey ", Journal of Artificial Intelligence Research, no.4, 1996, pp.237-255.
28. H. Hatakeyama and S. Mabu, "An Extension of Genetic Network Programming with Reinforcement Learning Using Actor-Critic," 2006 IEEE Congress on Evolutionary Computation, 2006.
29. S. Mabu, K. Hirasawa and J. Hu, "Genetic Network Programming with Reinforcement Learning and its Performance Evaluation", 2004 Gnenetic and Evolutionary Computation Conference, part II, 2004, pp.710-711.
30. K. Hirasawa, M. Okubo, H. Katagiri, J. Hu, and J. Murata, "Comparison between genetic network programming (GNP) and genetic programming (GP)," Proc. of 2001 Congress on Evolutionary Computation, 2001, pp.1276-1282.
31. M. Shokri, H. R. Tizhoosh and M. Kamel, "Opposition-Based Q(λ) Algorithm," 2006 International Joint Conference on Neural Networks, 2006.
32. TANG Hao, ZHOU Lei and YUAN Ji-bin, "Unified NDP method based on TD(0) learning for both average and discounted Markov decision processes," Control Theory& Application, vo1.23, no.2, 2006, pp.292-297.
33. TANG Hao, YUAN Ji-Bin, LU Yang, and CHENG Wen-Juan, "Performance Potential-based Neuro-dynamic Programming for SMDPs," ACTA AUTOMATICA SINICA, vol. 31, no. 4, 2005, pp.642-646.
34. TANG Hao, XI Hong-Sheng and YIN Bo-Qun, "A Simulation Optimization Algorithm for CTMDPs Based on Randomized Stationary Policies," ACTA AUTOMATICA SINICA, vol. 30, No.2, 2004, pp.229-235.
35. J. H. Lee and J. M. Lee, "Approximate dynamic programming based approach to process control and scheduling," Computers and Chemical Engineering, no. 30, 2006, pp.1603-1618.
36. J. M. Lee and J. H. Lee, "Approximate dynamic programming based approaches for input-output data-driven control of nonlinear processes," Automatica, vol. 41, no. 7, 2005, pp.281-1288.
37. J. M. Lee, N. S. Kaisare and J. H. Lee, "Choice of approximator and design of penalty function for an approximate dynamic programming based control approach," Journal of Process Control, vol.16, no. 2, 2006, pp.135-156.