Adaptive optimal control of partially-unknown constrained-input systems using policy iteration with experience replay

AIAA Guidance, Navigation, and Control (GNC) Conference

Volumn , Issue , 2013, Pages

  Modares, Hamidreza ^a   Lewis, Frank L ^b   Naghibi Sistani, Mohammad Bagher ^a   Chowdhary, Girish ^c,e   Yucelen, Tansel ^d  

^a FERDOWSI UNIVERSITY OF MASHHAD   (Iran)

^b UNIVERSITY OF TEXAS AT ARLINGTON   (United States)

^c MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^d University of Missouri Rolla   (United States)

^e Oklahoma State University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE OPTIMAL CONTROL; FEEDBACK CONTROL LAW; NEAR-OPTIMAL CONTROL; ONLINE LEARNING ALGORITHMS; OPTIMAL CONTROL PROBLEM; OPTIMAL CONTROL SOLUTION; PERSISTENCE OF EXCITATION; POLICY ITERATION ALGORITHMS;

ALGORITHMS; CONTROL; CONTROL THEORY; ITERATIVE METHODS; NEURAL NETWORKS;

OPTIMAL CONTROL SYSTEMS;

EID: 84883680649     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (23)

1. nd ed., Wiley, 2012.
2. Sutton, R. S., and Barto, A. G., Reinforcement learning-an introduction, Cambridge, MA: MIT Press, 1998.
3. Powell, W. B., Approximate Dynamic Programming: solving the curses of dimensionality, Wiley-Interscience, 2007.
4. nd ed., MA: Athena Scientific, 2012.
5. Howard, R. A., Dynamic programming and markov processes. Cambridge, MA: MIT Press, 1960.
6. Beard, R. W., "Improving the Closed-loop Performance of Nonlinear Systems, " Ph. D. Dissertation, Electrical Engineering Dep., Rensselaer Polytech Ins., Troy, New York, 1995.
7. Abu-Khalaf, M., and Lewis, F. L., "Nearly Optimal Control Laws for Nonlinear Systems with Saturating Actuators Using a Neural Network HJB Approach, " Automatica, Vol. 41, 2005, pp. 779, 791.
8. Doya, K., "Reinforcement Learning in Continuous-time and Space, " Neural Computation, Vol. 12, No. 1, 2000, pp. 219, 245.
9. Vamvoudakis, K., and Lewis, F. L., "Online Actor-critic Algorithm to Solve the Continuous Infinite-time Horizon Optimal Control Problem, " Automatica, Vol. 46, 2010, pp. 878, 888.
10. Murray, J. J., Cox, C. J., Lendaris, G. G., and Saeks, R., "Adaptive Dynamic Programming, " IEEE Trans. Syst., Man, Cybern., Part C: Appl. Rev., Vol. 32, No. 2, 2002, pp. 140, 153.
11. Vrabie, D., and Lewis, F. L., "Neural Network Approach to Continuous-time Direct Adaptive Optimal Control for Partially Unknown Nonlinear Systems, " Neural Netw., Vol. 22, 2009, pp. 237, 246.
12. Bhasin, S., "Reinforcement Learning and Optimal Control Methods for Uncertain Nonlinear Systems, " Ph. D. Dissertation, Florida Univ, 2011.
13. Vrabie, D., Pastravanu, O., Abu-Khalaf, M., and Lewis, F. L., "Adaptive Optimal Control for Continuous-time Linear Systems Based on Policy Iteration, " Automatica, Vol. 45, No. 2, 2009, pp. 477, 484.
14. Tsitsiklis, J. N., and Van Roy, B., "An Analysis of Temporal-Difference Learning with Function Approximation, " IEEE Trans. Automatic Control, Vol. 42, 1997, pp. 674, 690.
15. Wawrzynski, P., "Real-time Reinforcement Learning by Sequential Actor-critics and Experience Replay. " Neural Netw., Vol. 22, 2009, pp. 1484, 1497.
16. Dung, L. T., Komeda, T., and Takagi, M., "Efficient Experience Reuse in Non-Markovian Environments. " Proceeding of the Internatinal Conference Instrum, Control Inf. Technol., Tokyo, Japan, 2008, pp. 3327-3332.
17. Kalyanakrishnan, S., and Stone, P., "Batch reinforcement learning in a complex domain. " Proceeding of the 6th Internation Conference on Autoomus Agents and Multi-Agent Systms, Honolulu, HI, pp. 650-657, 2007.
18. Lin, L. J., "Self-improving Reactive Agents Based on Reinforcement Learning, Planning and Teaching. " Machine Learning, Vol. 8, 1992, pp. 293, 321.
19. Adam, S., Busoniu, L., and Babuska, R., "Experience Replay for Real-Time Reinforcement Learning Control. " IEEE Trans. Syst. Man, Cybern., Part C: Appl. Rev., Vol. 42, 2012, pp. 201, 212.
20. Chowdhary, G. V., "Concurrent learning for convergence in adaptive control without persistency of excitation, " PhD. Dissertation, Georgia institute of technology, 2010.
21. Chowdhary, G. V., and Johnson, E., "Concurrent Learning for Convergence in Adaptive Control without, " IEEE CDC, Atlanta GA, 2010, pp. 3675-3679.
22. Lyshevski, S. E., "Constrained optimization and control of nonlinear systems: New results in optimal control, " Proceeding of the IEEE Conference Decision and Control, 1996, pp. 541-546.
23. Finlayson, B. A., The method of weighted residuals and variational principles. New York: Academic Press, 1990.