Guidance in the use of adaptive critics for control

Handbook of Learning and Approximate Dynamic Programming

Volumn , Issue , 2004, Pages 97-124

  Lendaris, George G ^a   Neidhoefer, James C ^b  

^a PORTLAND STATE UNIVERSITY   (United States)

^b Accurate Automation Corporation   (United States)

Author keywords

Adaptation model; Dynamic programming; Equations; Function approximation; Learning; Training

Indexed keywords

PERSONNEL TRAINING;

ADAPTATION MODELS; EQUATIONS; FUNCTION APPROXIMATION; LEARNING; PRACTICAL ISSUES; PROBLEM FORMULATION; THEORETICAL FOUNDATIONS; UTILITY FUNCTIONS;

DYNAMIC PROGRAMMING;

EID: 80053055883     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1109/9780470544785.ch4     Document Type: Chapter

References (62)

1. C. Anderson, Approximating a policy can be easier than approximating a value function, Technical Report CS-00-101, Colorado State University, 2000.
2. C. Anderson, R. M. Kretchner, R M. Young, and D. C. Hittle, Robust reinforcement learning control with static and dynamic stability, International Journal of Robust and Nonlinear Control, vol. 11,2001.
3. N. Barabanov and D. Prokhorov, Stability analysis of discrete-time recurrent neural networks, IEEE Trans. Neural Networks, 2002.
4. N. Barabanov and D. Prokhorov, A new method for stability analyis of nonlinear discrete-time systems, IEEE Trans. Automatic Control, vol. 48, no. 12, 2003.
5. A. G. Barto, Handbook of Intelligent Control, chapter Reinforcement Learning and Adaptive Critic Methods, pp. 469-491, Van Nostrand-Reinhold, New York, 1992.
6. A. G. Barto and R. S. Sutton, Reinforcement Learning: An Introduction, MIT Press, Cambridge, MA, 1998.
7. R. E. Bellman, Dynamic Programming, Princeton University Press, Princeton, NJ, 1957.
8. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming, Athena Scientific, Belmont, MA, 1996.
9. P. Eaton, D. Prokhorov, and D. Wunsch, Neurocontroller for fuzzy ball-and-beam systems with nonlinear, nonuniform friction, IEEE Trans. Neural Networks, pp. 423-435,2000.
10. Z. Huang and S. N. Balakrishnan, Robust adaptive critic based neurocontrollers for missiles with model uncertainties, 2001 AAA Guidance, Navigation and Control Conference, Montreal, Canada, 2001.
11. Z. Huang and S.N. Balakrishnan, Robust adaptive critic based neurocontrollers for systems with input uncertainties, Proc. OfIJCNN’2000, pp. B-263, 2000.
12. L. P. Kaelbling, M. L. Littman, and A. W. Moore. Reinforcement learning: A survey, Journal of Artificial Intelligence Research, vol. 4, pp. 237-285,1996.
13. K. Krishna Kumar and J. Neidhoefer, Immunized adaptive critics, invited session on Adaptive Critics, ICNN’97, Houston, 1997. A version of this was presented at ANNIE’96, November, St. Louis, MO.
14. K. KrishnaKumar and J. Neidhoefer, Immunized adaptive critic for an autonomous aircraft control application, Artificial immune systems and their applications, Springer-Verlag, New York, 1998.
15. G. G. Lendaris, T. T. Shannon, and C. Paintz, More on training strategies for critic and action neural networks in dual heuristic programming method (invited paper), Proc. Of Systems, Man and Cybernetics Society International Conference’97, Orlando, FL, 1997.
16. G. G. Lendaris and C. Paintz, Training strategies for critic and action neural networks in dual heuristic programming method, Proc. Of International Conference on Neural Networks’97 (ICNN’97), Houston, 1997.
17. G. G. Lendaris, R. A. Santiago, and M. S. Carroll, Dual heuristic programming for fuzzy control, Proceeedings of IFSA /NAFIPS Conference, Vancouver, B.C., 2002.
18. G. G. Lendaris, R. A. Santiago, and M. S. Carroll, Proposed framework for applying adaptive critics in real-time realm, Proc. Of International Conference on Neural Networks’02 (IJCNN’ 2002), Hawaii, 2002.
19. G. G. Lendaris, R. A. Santiago, J. McCarthy, and M. S. Carroll, Controller design via adaptive critic and model reference methods, Proc. Of International Conference on Neural Networks’03 (IJCNN’ 2003), Portland, OR, 2003.
20. G. G. Lendaris and L. J. Schultz, Controller design (from scratch) using approximate dynamic programming, Proc. Of IEEE International Symposium on Intelligent Control’2000, (IEEE-ISIC’2000), Patras, Greece, 2000.
21. G. G. Lendaris, L. J. Schultz, and T. T. Shannon, Adaptive critic design for intelligent steering and speed control of a 2-axle vehicle, Proc. Of International Conference on Neural Networks’00 (IJCNN’2000), Italy, 2000.
22. G. G. Lendaris and T. T. Shannon, Application considerations for the dhp methodology, Proc. Of the International Joint Conference on Neural Networks’98 (IJCNN’98), Anchorage, 1998.
23. G. G. Lendaris, T. T. Shannon, and A. Rustan, A comparison of training algorithms for dhp adaptive critic neuro-control, Proc. Of International Conference on Neural Networks’99 (IJCNN’99), Washington, DC, 1999.
24. G. G. Lendaris, T. T. Shannon, L. J. Schultz, S. Hutsell, and A. Rogers, Dual heuristic programming for fuzzy control, Proceeedings of IFSA /NAFIPS Conference, Vancouver, B.C., 2001.
25. X. Liu and S. N. Balakrishnan, Convergence analysis of adaptive critic based neural networks, Proc. Of2000 American Control Conference, Chicago, 2000.
26. R. Luus, Iterative Dynamic Programmings, CRC Press, Boca Raton, FL, 2000.
27. A. McGovern and A. G. Barto, Automatic discovery of subgoals in reinforcement learning using diverse density, Proc. Of the 18th International Conference on Machine Learning, pp. 361-368, 2001.
28. A. Megretski and A. Rantzer, System analysis via integral quadratic constraints: Part II, Technical Report ISRN LUTFD2/TFRT-7559-SE, Lund Institute of Technology, 1997.
29. J. J. Murray, C. Cox, G.G. Lendaris, and R. Saeks, Adaptive dynamic programming, IEEE Trans, on Systems, Man, and Cybernetics, Part C: Applications and Reviews, vol. 32, no. 2, pp. 140-153, 2002.
30. K. S. Narendra and S. Mukhopadhyay, Adaptive control of nonlinear multivariable systems using neural networks, Neural Networks, vol. 7, no. 5, pp. 737-752, 1994.
31. J. C. Neidhoefer, Technical Report AAC-01-055, Accurate Automation Corp, 2001.
32. J. C. Neidhoefer, Technical Report AAC-02-016, Accurate Automation Corp, 2002.
33. J. C. Neidhoefer and K. Krishnakumar, Intelligent control for autonomous aircraft missions, IEEE Trans, on Systems, Man, and Cybernetics, Part A, 2001.
34. D. Nguyen and B. Widrow, The Truck Backer-Upper: An Example of Self Learning in Neural Networks, Neural Networks for Control, MIT Press, Cambridge, MA, 1957.
35. R. Padhi and S. N. Balakrishnan, Adaptive critic based optimal control for distributed parameter systems, Proc. International Conference on Information, Communication and Signal Proc., 1999.
36. R. Padhi and S. N. Balakrishnan, A systematic synthesis of optimal process control with neural networks, Proc. American Control Conference, Washington, DC, 2001.
37. R. Padhi, S. N. Balakrishnan, and T. Randolph, Adaptive critic based optimal neuro control synthesis for distributed parameter systems, Automatica, vol. 37, pp.1223-1234,2001.
38. J. C. Pearce, The Biology of Transcendence, Park Street Press, Rochester, VT, 2002.
39. T. J. Perkins and A. G. Barto, Lyapunov design for safe reinforcement learning, AAAI Spring Symposium on Safe Learning Agents.
40. D. Prokhorov, Adaptive Critic Designs and their Application, Ph.D. Thesis, Texas Tech University, 1997.
41. D.V. Prokhorov and L.A. Feldkamp, Application of SVM to Lyapunov Function Approximation, Proc. Of International Conference on Neural Networks’99 (IJCNN’1999), Washington, DC, 1999.
42. D. Prokhorov, R. Santiago, and D. Wunsch, Adaptive critic designs: A case study for neurocontrol, Neural Networks, vol. 8, pp. 1367-1372,1995.
43. D. Prokhorov and D. Wunsch, Adaptive critic designs, IEEE Trans. Neural Networks, vol. 8, no. 5, pp. 997-1007,1997.
44. A. Rogers, T. T. Shannon, and G. G. Lendaris, A comparison of dhp based antecedent parameter tuning strategies for fuzzy control, Proc. Of IFSA/NAFIPS Conference, Vancouver, B.C., 2001.
45. R. Saeks, C. Cox, J. Neidhoefer, and D. Escher, Adaptive critic control of the power train in a hybrid electric vehicle, Proc. SMCia Workshop, 1999.
46. R. Saeks, C. Cox, J. Neidhoefer, P. Mays, and J. Murray, Adaptive critic control of a hybrid electric vehicle, IEEE Trans, on Intelligent Transportation Systems, vol. 3, no. 4,2002.
47. R. Santiago and P. Werbos, New progress towards truly brain-like intelligent control, Proc. WCNN’94, pp. 12-133, Erlbaum, Hillsdale, NJ, 1994.
48. L. J. Schultz, T. T. Shannon, and G. G. Lendaris, Using dhp adaptive critic methods to tune a fuzzy automobile steering controller, Proc. Of IFSA/NAFIPS Conference, Vancouver, B.C., 2001.
49. T. T. Shannon, Partial, noisy and qualitative models for adaptive critic based neuro-control, Proc. Of International Conference on Neural Networks’99 (IJCNN’99), Washington, DC, 1999.
50. T. T. Shannon and G. G. Lendaris, Qualitative models for adaptive critic neurocontrol, Proc. Of IEEE SMC’99 Conference, Tokyo, 1999.
51. T. T. Shannon and G. G. Lendaris, Adaptive critic based approximate dynamic programming for tuning fuzzy controllers, Proc. Of IEEE-FUZZ 2000, 2000.
52. T. T. Shannon and G. G. Lendaris, A new hybrid critic-training method for approximate dynamic programming, Proc. Of International Societyfor the System Sciences, ISSS’2000, Toronto, 2000.
53. T. T. Shannon and G. G. Lendaris, Adaptive critic based design of a fuzzy motor speed controller, Proc. OfISIC2001, Mexico City, 2001.
54. T. T. Shannon, R. A. Santiago, and G. G. Lendaris, Accelerated critic learning in approximate dynamic programming via value templates and perceptual learning, Proa of IJCNN’03, Portland, OR, 2003.
55. S. Shervais and T. T. Shannon, Adaptive critic based adaptation of a fuzzy policy manager for a logistic system, Proa of IFSA /NAFIPS Conference, Vancouver, B.C., 2001.
56. R. S. Sutton, A. G. Barto, and R. J. Williams, Reinforcement learning is direct adaptive optimal control, Proc. Of the American Control Conference, Boston, pp. 2143-2146,1991.
57. N. Visnevski and D. Prokhorov, Control of a nonlinear multivariable system with adaptive critic designs, Proc. Of Artificial Neural Networks in Engineering (ANNIE), vol. 6, pp. 559-565,1996.
58. L. X. Wang, A Course in Fuzzy Systems and Control, Prentice-Hall, Englewood Cliffs, NJ, 1997.
59. P. J. Werbos, A Menu of Designs for Reinforcement Learning Over Time, Neural Networks for Control, pp. 67-95, MIT Press, Cambridge, MA, 1990.
60. P. J. Werbos, Approximate Dynamic Programming for Real-Time Control and Neural Modeling, Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, pp. 493-525, Van Nostrand Reinhold, New York, 1994.
61. B. Widrow, N. Gupta, and S. Maitra, Punish/reward: Learning with a critic in adaptive threshhold systems, IEEE Trans, on Systems, Man and Cybernetics, vol. 3, no. 5, pp. 455-465,1973.
62. J. Yen and R. Langari, Fuzzy Logic: Intelligence, Control and Information, Prentice-Hall, Englewood Cliffs, NJ, 1999.