A model-free robust policy iteration algorithm for optimal control of nonlinear systems

Proceedings of the IEEE Conference on Decision and Control

Volumn , Issue , 2010, Pages 3060-3065

  Bhasin, S ^a   Johnson, M ^a   Dixon, W E ^a  

^a UNIVERSITY OF FLORIDA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONTINUOUS TIME SYSTEMS; GRADIENT METHODS; NEURAL NETWORKS; NONLINEAR SYSTEMS; ONLINE SYSTEMS; OPTIMAL CONTROL SYSTEMS; OPTIMIZATION;

CONTINUOUS TIME NONLINEAR SYSTEMS; DYNAMIC NEURAL NETWORKS; GRADIENT DESCENT METHOD; NEURAL NETWORK STRUCTURES; OPTIMAL CONTROL PROBLEM; POLICY ITERATION ALGORITHMS; TEMPORAL DIFFERENCE ERRORS; UNIFORMLY ULTIMATELY BOUNDED;

CLOSED LOOP SYSTEMS;

EID: 79953151751     PISSN: 07431546     EISSN: 25762370     Source Type: Conference Proceeding    
DOI: 10.1109/CDC.2010.5717295     Document Type: Conference Paper

References (34)

1. R. Sutton, "Learning to predict by the methods of temporal differences," Mach. Learn., vol. 3, no. 1, pp. 9-44, 1988.
2. R. Sutton and A. Barto, Introduction to reinforcement learning. MIT Press Cambridge, MA, USA, 1998.
3. B. Widrow, N. Gupta, and S. Maitra, "Punish/reward: Learning with a critic in adaptive threshold systems," IEEE Trans. Syst. Man Cybern., vol. 3, no. 5, pp. 455-465, 1973.
4. A. Barto, R. Sutton, and C. Anderson, "Neuronlike adaptive elements that can solve difficult learning control problems," IEEE Trans. Syst. Man Cybern., vol. 13, no. 5, pp. 834-846, 1983.
5. R. Sutton, A. Barto, and R. Williams, "Reinforcement learning is direct adaptive optimal control," IEEE Contr. Syst. Mag., vol. 12, no. 2, pp. 19-22, 1992.
6. D. Kirk, Optimal Control Theory: An Introduction. Dover Pubns, 2004.
7. P. Webros, "A menu of designs for reinforcement learning over time," in Neural networks for control, ser. MIT Press Series In Neural Network Modeling And Connectionism. Cambridge, MA, USA: MIT Press, 1990, pp. 67-95.
8. D. Bertsekas and J. Tsitsiklis, Neuro-Dynamic Programming. Athena Scientific, 1996.
9. D. V. Prokhorov and I. Wunsch, D. C., "Adaptive critic designs," IEEE Trans. Neural Networks, vol. 8, no. 5, pp. 997-1007, Sep. 1997.
10. J. Si, A. Barto, W. Powell, and D. Wunsch, Eds., Handbook of Learning and Approximate Dynamic Programming. Wiley-IEEE Press, 2004.
11. S. Balakrishnan, "Adaptive-critic-based neural networks for aircraft optimal control," J. Guid. Contr. Dynam., vol. 19, no. 4, pp. 893-898, 1996.
12. G. Lendaris, L. Schultz, and T. Shannon, "Adaptive critic design for intelligent steering and speed control of a 2-axle vehicle," in Int. Joint Conf. Neural Netw., 2000, pp. 73-78.
13. S. Ferrari and R. Stengel, "An adaptive critic global controller," in Proc. IEEE Am. Control Conf., vol. 4, 2002.
14. D. Han and S. Balakrishnan, "State-constrained agile missile control with adaptive-critic-based neural networks," IEEE Trans. Control Syst. Technol., vol. 10, no. 4, pp. 481-489, 2002.
15. P. He and S. Jagannathan, "Reinforcement learning neural-network-based controller for nonlinear discrete-time systems with input constraints," IEEE Trans. Syst. Man Cybern. Part B Cybern., vol. 37, no. 2, pp. 425-436, 2007.
16. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof," IEEE Trans. Syst. Man Cybern. Part B Cybern., vol. 38, no. 4, pp. 943-949, Aug. 2008.
17. L. Baird, "Advantage updating," Wright Lab, Wright-Patterson Air Force Base, OH, Tech. Rep., 1993.
18. K. Doya, "Reinforcement learning in continuous time and space," Neural Comput., vol. 12, no. 1, pp. 219-245, 2000.
19. J. Murray, C. Cox, G. Lendaris, and R. Saeks, "Adaptive dynamic programming," IEEE Trans. Syst. Man Cybern. Part C Appl. Rev., vol. 32, no. 2, pp. 140-153, 2002.
20. R. Beard, G. Saridis, and J. Wen, "Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation," Automatica, vol. 33, pp. 2159-2178, 1997.
21. M. Abu-Khalaf and F. Lewis, "Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach," Automatica, vol. 41, no. 5, pp. 779-791, 2005.
22. T. Hanselmann, L. Noakes, and A. Zaknich, "Continuous-time adaptive critics," IEEE Trans. Neural Networks, vol. 18, no. 3, pp. 631-647, 2007.
23. D. Vrabie and F. Lewis, "Neural network approach to continuoustime direct adaptive optimal control for partially unknown nonlinear systems," Neural Networks, vol. 22, no. 3, pp. 237-246, 2009.
24. K. Vamvoudakis and F. Lewis, "Online synchronous policy iteration method for optimal control," in Recent Advances in Intelligent Control Systems. Springer, 2009, pp. 357-374.
25. P. Werbos, "Approximate dynamic programming for real-time control and neural modeling," in Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, D. A. White and D. A. Sofge, Eds. New York: Van Nostrand Reinhold, 1992.
26. J. Hopfield, "Neurons with graded response have collective computational properties like those of two-state neurons," Proc. Nat. Acad. Sci. U.S.A., vol. 81, no. 10, p. 3088, 1984.
27. A. Poznyak, E. Sanchez, and W. Yu, Differential neural networks for robust nonlinear control: identification, state estimation and trajectory tracking. World Scientific Pub Co Inc, 2001.
28. M. Polycarpou and P. Ioannou, "Identification and control of nonlinear systems using neural network models: Design and stability analysis," Systems Report 91-09-01, University of Southern California, 1991.
29. G. Cybenko, "Approximation by superpositions of a sigmoidal function," Math. Control Signals Syst., vol. 2, pp. 303-314, 1989.
30. F. L. Lewis, "Nonlinear network structures for feedback control," Asian J. Control, vol. 1, no. 4, pp. 205-228, 1999.
31. W. E. Dixon, A. Behal, D. M. Dawson, and S. Nagarkatti, Nonlinear Control of Engineering Systems: A Lyapunov-Based Approach. Birkhuser Boston, 2003.
32. M. Krstic, P. V. Kokotovic, and I. Kanellakopoulos, Nonlinear and Adaptive Control Design, S. Haykin, Ed. John Wiley & Sons, 1995.
33. J. Slotine and W. Li, Applied Nonlinear Control. Prentice Hall, 1991.
34. H. K. Khalil, Nonlinear Systems, 3rd ed. Upper Saddle River, NJ: Prentice Hall, 2002.