Approximate dynamic programming for optimal stationary control with control-dependent noise

IEEE Transactions on Neural Networks

Volumn 22, Issue 12 PART 2, 2011, Pages 2392-2398

  Jiang, Yu ^a   Jiang, Zhong Ping ^a,b  

^a POLYTECHNIC UNIVERSITY   (United States)

^b PEKING UNIVERSITY   (China)

Author keywords

Approximate dynamic programming; control dependent noise; optimal stationary control; stochastic systems

Indexed keywords

ADDITIVE AND MULTIPLICATIVE NOISE; ALGEBRAIC RICCATI EQUATIONS; APPROXIMATE DYNAMIC PROGRAMMING; CONTROL-DEPENDENT NOISE; COST MATRICES; NUMERICAL EXAMPLE; OPTIMAL COSTS; POLICY ITERATION ALGORITHMS; STOCHASTIC OPTIMAL CONTROL PROBLEM; TIME INTERVAL;

ALGORITHMS; COVARIANCE MATRIX; OPTIMAL CONTROL SYSTEMS; OPTIMIZATION; REINFORCEMENT LEARNING; RICCATI EQUATIONS; STOCHASTIC CONTROL SYSTEMS; STOCHASTIC SYSTEMS;

DYNAMIC PROGRAMMING;

ARTICLE; ARTIFICIAL INTELLIGENCE; DATA MINING; FACTUAL DATABASE; FEEDBACK SYSTEM; METHODOLOGY; SYSTEM ANALYSIS; THEORETICAL MODEL;

ARTIFICIAL INTELLIGENCE; DATA MINING; DATABASES, FACTUAL; FEEDBACK; MODELS, THEORETICAL; PROGRAMMING, LINEAR;

EID: 83655167263     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2011.2165729     Document Type: Article

References (18)

1. A. G. Barto and R. S. Sutton, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
2. P. J. Werbos, "Beyond regression: New tools for prediction and analysis in the behavioral sciences," Ph.D. dissertation, Committee Appl. Math., Harvard Univ., Cambridge, MA, 1974.
3. P. J. Werbos, "Neural networks for control and system identification," in Proc. IEEE Conf. Decis. Control, vol. 1. Tampa, FL, Dec. 1989, pp. 260-265.
4. P. J. Webros, A Menu of Designs for Reinforcement Learning Over Time, Neural Networks for Control. Cambridge, MA: MIT Press, 1990.
5. P. J. Werbos, "Approximate dynamic programming for real-time control and neural modeling," in Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, vol. 15. New York: Van Nostrand, 1992, pp. 493-525.
6. L. Busoniu, R. Babuska, B. De Schutter, and D. Ernst, Reinforcement Learning and Dynamic Programming Using Function Approximators. Boca Raton, FL: CRC Press, 2010.
7. X. Xu, D. Hu, and X. Lu, "Kernel-based least squares policy iteration for reinforcement learning," IEEE Trans. Neural Netw., vol. 18, no. 4, pp. 973-992, Jul. 2007. (Pubitemid 47098876)
8. W. B. Powell, Approximate Dynamic Programming: Solving the Curses of Dimensionality. New York: Wiley, Sep. 2007.
9. H. Yu and D. Bertsekas, "Basis function adaptation methods for cost approximation in MDP," in Proc. IEEE Symp. Adapt. Dyn. Program. Reinforce. Learn., Mar.-Apr. 2009, pp. 74-81.
10. F. L. Lewis and D. Vrabie, "Reinforcement learning and adaptive dynamic programming for feedback control," IEEE Circuits Syst. Mag., vol. 9, no. 3, pp. 32-50, Sep. 2009.
11. S. J. Bradtke, B. E. Ydstie, and A. G. Barto, "Adaptive linear quadratic control using policy iteration," in Proc. Amer. Control Conf., vol. 3. Jul. 1994, pp. 3475-3479.
12. D. Vrabie, O. Pastravanu, M. Abu-Khalaf, and F. L. Lewis, "Adaptive optimal control for continuous-time linear systems based on policy iteration," Automatica, vol. 45, no. 2, pp. 477-484, Feb. 2009.
13. Y. Jiang and Z. P. Jiang, "Approximate dynamic programming for output feedback control," in Proc. 29th Chin. Control Conf., Beijing, China, Jul. 2010, pp. 5815-5820.
14. F. L. Lewis and K. G. Vamvoudakis, "Reinforcement learning for partially observable dynamic processes: Adaptive dynamic programming using measured output data," IEEE Trans. Syst., Man, Cybern., Part B: Cybern., vol. 41, no. 1, pp. 14-25, Feb. 2011.
15. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Model-free Q-learning designs for linear discrete-time zero-sum games with application to Hinfinity control," Automatica, vol. 43, no. 3, pp. 473-481, Mar. 2007. (Pubitemid 46209050)
16. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof," IEEE Trans. Syst., Man, Cybern., Part B: Cybern., vol. 38, no. 4, pp. 943-949, Aug. 2008.
17. D. Kleinman, "Optimal stationary control of linear systems with controldependent noise," IEEE Trans. Autom. Control, vol. 14, no. 6, pp. 673-677, Dec. 1969.
18. F. Wang, H. Zhang, and D. Liu, "Adaptive dynamic programming: An introduction," IEEE Comp. Intell. Mag., vol. 4, no. 2, pp. 39-47, May 2009.