A retrospective on Adaptive Dynamic Programming for control

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2009, Pages 1750-1757

  Lendaris, George G ^a  

^a PORTLAND STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE DYNAMIC PROGRAMMING; COMPUTATIONAL INTELLIGENCE METHODS; DYNAMIC PROGRAMMING METHODS; NONLINEAR PLANT; OPTIMAL CONTROL POLICY; PRESENT STATUS; RESEARCH AND DEVELOPMENT; SEQUENTIAL DECISION MAKING;

NEURAL NETWORKS; REINFORCEMENT; REINFORCEMENT LEARNING; SYSTEMS ENGINEERING;

DYNAMIC PROGRAMMING;

EID: 68249151304     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2009.5178716     Document Type: Conference Paper

References (77)

1. C. Anderson, R. M. Kretchner, P. M. Young, and D. C. Hittle, "Robust reinforcement learning control with static and dynamic stability, " Int. J. Robust Nonlinear Control, vol. 11, pp. 1469-1500, 2001.
2. C. W. Anderson, P. M. Young, M. R. Buehner, J. N. Knight, K. A. Bush, and D. C. Hittle, "Robust reinforcement learning control using integral quadratic constraints for recurrent neural networks, " IEEE Trans. Neural Networks, no. 4, pp. 993-1002, Jul. 2007.
3. K. J. Astrom and B. Wittenmark, Computer Controlled Systems, Theory and Design, Englewood Cliffs, NJ: Prentice-Hall, 1984.
4. M. Athans & P. L. Falb, 1966, Optimal Control Theory, McGraw Hill.
5. A. G. Barto, R. S. Sutton, & C. W. Anderson, 1983, "Neuronlike Elements That Can Solve Difficult Learning Control Problems, " IEEE Transactions on Systems Man & Cybernetics, vol. 13, pp 834-846.
6. R. E. Bellman. Dynamic Programming, Princeton Univ. Press, 1957.
7. D. P. Bertsekas, 1987, Dynamic Programming: Deterministic and Stochasic Models, Prentice-Hall.
8. D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control: The Discrete-Time Case, Belmont, MA: Athena Scientific, 1996.
9. Bryson, Jr. & Y. Ho, 1969, Applied Optimal Control, Blaisdell Pub.
10. T. Cheng and F. L. Lewis, "Neural network solution of finite-horizon H-infinity optimal controllers for nonlinear systems, " in Emerging Technologies, Robotics, and Control Systems, vol. 1, InternationalSAR, Palermo, Italy, pp. 59-65, ed. S. Pennacchio, 2007.
11. P. R. Cohen & E. A. Feigenbaum, 1982, The Handbook of Artificial Intelligence, vol 3, Kauffman.
12. G. Cybenko, "Approximation by superpositions of a sigmoidal function, " Mathematics of Control. Signals and Systems, vol. 2, no. 4, pp. 303-314, 1989.
13. R. C. Dorf and R. H. Bishop, Modern Control Systems, 9th ed. Englewood Cliffs, NJ: Prentice-Hall, 2001.
14. G. C. Goodwin, S. F. Graebe, and M. E. Salgado, Control System Design.
15. R. Howard, Dynamic Programming and Markov Processes, MIT Press, 1960.
16. Z. Huang and S. N. Balakrishnan, "Robust adaptive critic based neuro controllers for missiles with model uncertainties, " in Proc. AIAA Guid. Navigat., Control Conf., Montreal, QC, Canada, 2001.
17. Z. Huang and S. N. Balakrishnan. Robust adaptive critic based neurocontrollers for systems with input uncertainties. Proceedings of IJCNN' 2000, Como, Italy, pages B-263, July 2000.
18. K. KrishnaKumar and J. Neidhoefer. "Immunized adaptive critics". Adaptive Critics Session, ICNN '97, Houston, 1997.
19. K. KrishnaKumar and J. Neidhoefer, "Immunized Adaptive Critic for an Autonomous Aircraft Control Application, " Artificial immune systems and their applications. Springer-Verlag, Inc., 1998.
20. G. G. Lendaris, R. A. Santiago, and M. S. Carroll. "Dual heuristic programming for fuzzy control, " Proceeedings of IFSA /NAFIPS Conference, Vancouver, B. C, July 2002.
21. G. G. Lendaris, T. T. Shannon, & M. Zwick, "Prestructuring NNs via Extended Dependency Analysis with Application to Pattern Classification, " Proc. SPIE'99, Orlando, 1999.
22. G. G. Lendaris, R. A. Santiago, J. McCarthy, and M. S. Carroll, "Controller design via adaptive critic and model reference methods, " Proceedings of IJCNN' 2003, Portland, July 2003.
23. G. G. Lendaris and L. J. Schultz, "Controller design (from scratch) using approximate dynamic programming, " Proceedings of IEEE Intnl. Symp. on Intelligent Control '2000, Patras, Greece, July 2000.
24. G. G. Lendaris, L. J. Schultz, and T. T. Shannon, "Adaptive critic design for intelligent steering and speed control of a 2-axle vehicle, " Proceedings of IJCNN' 2000, Italy, Jan 2000.
25. G. G. Lendaris and T. T. Shannon, "Application considerations for the DHP methodology, " Proc of IJCNN'98, IEEE Press, May 1998.
26. G. G. Lendaris, T. T. Shannon, and A. Rustan, "A comparison of training algorithms for DHP adaptive critic neuro-control, " Proc. IJCNN' 99, Washington, DC, IEEE Press, 1999.
27. G. G. Lendaris, T. T. Shannon, L. J. Schultz, S. Hutsell, and A. Rogers, "Dual heuristic programming for fuzzy control, " Proc. of IFSA /NAFIPS Conference, Vancouver, B. C., July 2001.
28. G. G. Lendaris, R. A. Santiago, and M. S. Carroll, "Proposed frame- work for applying adaptive critics in real-time realm, " in Proc IJCNN, HI, May 2002.
29. G. G. Lendaris, R. A. Santiago, J. McCarthy, and M. S. Carroll, "Controller design via adaptive critic and model reference methods, " in Proc. IJCNN, pp. 3173-3178, Portland, OR, Jul. 2003.
30. G. G. Lendaris and J. C. Neidhoefer, "Guidance in the use of adaptive critics for control, " in Handbook of Learning and Approximate Dynamic Programming, Piscataway, NJ: ch. 4, pp. 97-124, IEEE Press, 2004.
31. G. G. Lendaris, "Higher Level Application of ADP: A Next Phase for the Control Field?" IEEE Transactions on Systems, Man, and Cybernetics - PartB: Cybernetics, vol. 38, no. 4, Aug. 2008.
32. F. L. Lewis and S. S. Ge, "Neural Networks in Feedback Control Systems, " in Mechanical Engineer's Handbook, Instrumentation, Systems, Controls, and MEMS Book 2, ed. Meyer Kutz, Chapter 19, John Wiley, N. Y. 2006.
33. F. L. Lewis, G. Lendaris, and Derong Liu, "Special Issue on Approximate Dynamic Programming and Reinforcement Learning for Feedback Control, " IEEE Trans. Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 38, no. 4, Aug. 2008.
34. F. L. Lewis, Applied Optimal Control and Estimation, Englewood Cliffs, NJ: Prentice-Hall, 1992.
35. F. L. Lewis and V. L. Syrmos, Optimal Control, 2nd ed. NJ:Wiley, 1995.
36. X. Liu and S. N. Balakrishnan, "Convergence analysis of adaptive critic based neural networks, " Proceedings of 2000 American Control Conference, Chicago, IL, June 2000.
37. R. Luus, Iterative Dynamic Programming, CRC Press, 2000.
38. W. T. Miller, R. S. Sutton, P. J. Werbos, ed., Neural Networks for Control, Cambridge: MIT Press, 1991.
39. E. Mosca, Optimal, Predictive, and Adaptive Control. Englewood Cliffs, NJ: Prentice-Hall, 1995.
40. J. J. Murray, C. Cox, G. G. Lendaris, and R. Saeks, "Adaptive dynamic programming, " IEEE Trans. Syst., Man, Cybernetics, Part C: Applications & Reviews, vol. 32, no. 2, pp. 140-153, May 2002.
41. K. S. Narendra and K. Parthasarathy, "Identification and control of dynamical systems using neural networks, " IEEE TNN, vol. 1, pp. 4-27, Mar. 1990.
42. K. S. Narendra and F. L. Lewis, "Special Issue on Neural Network feedback Control, " Automatica, vol. 37, no. 8, Aug. 2001.
43. K. S. Narendra and S. Mukhopadhyay, "Adaptive control of nonlinear multivariable systems using NNs, " Neural Netw., vol. 7, no. 5, pp. 737-752, 1994.
44. K. Ogata, Modern Control Engineering, 3rd ed. Englewood Cliffs, NJ: Prentice- Hall, 1997.
45. R. Padhi and S. N. Balakrishnan, "Adaptive critic based optimal control for distributed parameter systems, " Proc. Intnl. Conference on Information, Communication and Signal Processing, Dec. 1999.
46. R. Padhi and S. N. Balakrishnan, "A systematic synthesis of optimal process control with neural networks, " Proceedings American Control Conference, Washington, D. C., June 2001.
47. R. Padhi, S. N. Balakrishnan, and T. Randolph, "Adaptive critic based optimal neuro control synthesis for distributed parameter systems, " Automatica, 37:1223-1234, 2001.
48. J. C. Pearce, The Biology of Transcendence, Park Street Press, 2002.
49. C. L. Phillips and R. D. Harbor, Feedback Control Systems, 4th ed. Englewood Cliffs, NJ: Prentice-Hall, 2000.
50. D. Prokhorov, Adaptive Critic Designs and their Application. PhD thesis, EE Dept, Texas Tech University, 1997.
51. D. Prokhorov and D. Wunsch, "Adaptive critic designs, " IEEE Transactions on Neural Networks, 8 (5) :997-1007, 1997.
52. A. Rogers, T. T. Shannon, and G. G. Lendaris, "A comparison of DHP based antecedent parameter tuning strategies for fuzzy control, " Proceedings of IFSA/NAFIPS Conf, Vancouver B. C., July, 2001.
53. R. Saeks, C. Cox, J. Neidhoefer, and D. Escher, "Adaptive critic control of the power train in a hybrid electric vehicle, " Proceedings SMCia Workshop, 1999.
54. R. Saeks, C. Cox, J. Neidhoefer, P. Mays, and J. Murray, "Adaptive critic control of a hybrid electric vehicle" IEEE Transactions on Intelligent Transportation Systems, 3, No. 4, December, 2002.
55. A. L. Samuel, "Some Studies in Machine Learning Using the Game of Checkers, " IBM Jour of Res & Dev, pp 210-229, vol 3, 1959
56. S. Sastry and M. Bodson, Adaptive Control: Stability, Convergence, and Robustness, Englewood Cliffs, NJ: Prentice-Hall, 1989.
57. L. J. Schultz, T. T. Shannon, and G. G. Lendaris, "Using DHP adaptive critic methods to tune a fuzzy automobile steering controller, " Proc of IFSA/NAFIPS Conference, Vancouver, B. C., July, 2001.
58. T. T. Shannon and G. G. Lendaris, "Qualitative models for adaptive critic neurocontrol", Proc. IEEE SMC '99 Conf., Tokyo, June, 1999.
59. T. T. Shannon and G. G. Lendaris. "Adaptive critic based approximate dynamic programming for tuning fuzzy controllers, " Proc. of IEEE-FUZZ 2000, IEEE Press, 2000.
60. T. T. Shannon and G. G. Lendaris, "A new hybrid critic-training method for approximate dynamic programming, " Proc. of Intnl Society for the System Sciences, ISSS' 2000, Toronto, August, 2000.
61. T. T. Shannon and G. G. Lendaris, "Adaptive critic based design of a fuzzy motor speed controller, " Proc ISIC2001, Mexico, Sept., 2001.
62. T. T. Shannon, R. A. Santiago, and G. G. Lendaris, "Accelerated critic learning in approximate dynamic programming via value templates and perceptual learning, " Pro. IJCNN' 03, Portland, July, 2003.
63. S. Shervais and T. T. Shannon, "Adaptive critic based adaptation of a fuzzy policy manager for a logistic system, " Proc. of IFSA /NAFIPS Conference, Vancouver, B. C., July, 2001.
64. L. X. Wang, A Course in Fuzzy Systems and Control, Prentice, 1997.
65. J. Watt, Microsoft Encarta Online Encyclopedia 2005. 1997-2005 Microsoft Corporation. [Online]. Available: http://encarta.msn. com.
66. P. J. Werbos, "Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences, " Ph. D. Dissertation, Harvard University, 1974.
67. P J. Werbos, "Neural networks for control and system identification, " Proc. IEEE Conf. Decision and Control, Fla., 1989.
68. P. J. Werbos, "A Menu of Designs for Reinforcement Learning Over Time, " Neural Networks for Control, pp. 67-95. MIT Press, 1990.
69. P. J. Werbos, "Approximate Dynamic Programming for Real-Time Control and Neural Modeling, " Handbook of Intelligent Control, pp. 493-525. Van Nostrand Reinhold, NY, 1994.
70. B. Widrow, N. Gupta & S. Maitra, "Punish/Reward: Learning With a Critic in Adaptive Threshold Systems, " IEEE Transactions on Systems Man & Cybernetics, vol. 3 (5), pp 455-465, 1973.
71. D. A. White and D. A. Sofge, ed., Handbook of Intelligent Control, NY: Van Nostrand Reinhold, 1992.
72. J. Yen and R. Langari, Fuzzy Logic: Intelligence, Control and Information, Prentice Hall, 1999.
73. K. Zhou and J. C. Doyle, Essentials of Robust Control, Englewood Cliffs, NJ: Prentice-Hall, 1998.
74. G. G. Lendaris & K. Mathia, "Using A Priori Knowledge to Prestructure ANNs, " Australian Journal of Intelligent Information Systems, vol. 1, no. 1, pp. 25-33, March, 1994.
75. J. Si, A. G. Barto, W. B. Powell, and D. Wunsch, II, Handbook of Learning and Approximate Dynamic Programming. Piscataway, NJ: IEEE Press, 2004.
76. Proceedings of IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, IEEE Press, 2007 and 2008.
77. D. E. Rumelhart, G. E. Hinton, R. J. Williams, "Learning Internal Representations by Error Propagation" in Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol. 1: Foundations (eds Rumelhart, D. E. & McClelland, J. L.) 318-362 (MIT, Cambridge, 1986).