Model-based adaptive critic designs

Handbook of Learning and Approximate Dynamic Programming

Volumn , Issue , 2004, Pages 65-95

  Ferrari, Silvia ^a   Stengel, Robert F ^b  

^a Duke University   (United States)

^b Princeton University   (United States)

Author keywords

Adaptation model; Dynamic programming; Equations; Heuristic algorithms; Mathematical model; Optimal control; Trajectory

Indexed keywords

BACKPROPAGATION; HEURISTIC ALGORITHMS; HEURISTIC PROGRAMMING; MATHEMATICAL MODELS; TRAJECTORIES;

ADAPTATION MODELS; ADAPTIVE CRITIC DESIGNS; BACK-PROPAGATION THROUGH TIME; EQUATIONS; HEURISTIC DYNAMIC PROGRAMMING; MODEL REFERENCE ADAPTIVE; MODEL-BASED OPC; OPTIMAL CONTROLS;

DYNAMIC PROGRAMMING;

EID: 85032189594     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1109/9780470544785.ch3     Document Type: Chapter

References (41)

1. R. F. Stengel, Optimal Control and Estimation, Dover Publications, New York, 1994.
2. Y. Bar-Shalom and E. Tse, Dual effect, certainty equivalence, and separation in stochastic control, IEEE Trans. Automatic Control, vol. 19, no. 5, pp. 494-500, 1974.
3. R. Bellman, Dynamic Programming, Princeton University Press, Princeton, NJ 1957.
4. R. Bellman and R. Kalaba, Dynamic Programming and Modern Control Theory, Academic Press, New York, 1965.
5. R. Howard, Dynamic Programming and Markov Processes, MIT Press, Cambridge, MA, 1960.
6. A. A. Feldbaum, Optimal Control Systems, Academic Press, New York, 1965.
7. E. Tse, Y. Bar-Shalom, and L. Meier, III, Wide-sense adaptive dual control for nonlinear stochastic systems, IEEE Trans. Automatic Control, vol. 18, no. 2, pp. 98-108, 1973.
8. Y. Bar-Shalom, Stochastic dynamic programming: caution and probing, IEEE Trans. Automatic Control, vol. 26, no. 5, pp. 1184-1195,1981.
9. D. Q. Mayne and H. Michalska, Receding horizon control of non-linear systems, IEEE Trans. Automatic Control, vol. 35, no. 5, pp. 814-824, 1990.
10. G. C. Goodwin and K. S. Sin, Adaptive Filtering Prediction and Control, Prentice-Hall, Englewood Cliffs, NJ, 1984.
11. S. Ferrari and R. F. Stengel, Classical/neural synthesis of nonlinear control systems, Journal of Guidance, Control and Dynamics, vol. 25, no. 3, pp. 442-448, 2002.
12. S. Ferrari and R. F. Stengel, Algebraic training of a neural network, Proc. 2001 American Control Conference, Arlington, VA, pp. 1605-1610, 2001.
13. D. E. Kirk, Optimal Control Theory: An Introduction, Prentice-Hall, Englewood Cliffs, NJ, 1970.
14. R. E. Bellman and S. E. Dreyfus, Applied Dynamic Programming, Princeton University Press, Princeton, NJ, 1962.
15. R. Bellman, Methods of Nonlinear Analysis: Volume II, Academic Press, New York, 1973.
16. P. J. Werbos, Building and understanding adaptive systems: a statisti-cal/numerical approach for factory automation and brain research, IEEE Trans. Syst., Man, Cybern., vol. 17, no. 1, pp. 7-20, 1987.
17. D. P. Bertsekas, Distributed dynamic programming, IEEE Trans. Automatic Control, vol. 27, pp. 610-616,1982.
18. S. Ferrari and R. F. Stengel, An adaptive critic global controller, Proc. American Control Conference, Anchorage, AK, 2002.
19. P. J. Werbos, Neurocontrol and Supervised Learning: An Overview and Evaluation, Handbook of Intelligent Control, D. A. White and D. A. Sofge (eds.), pp. 65-86, Van Nostrand Reinhold, New York, 1992.
20. V. Prokhorov and D. C. Wunsch, II, Adaptive critic designs, IEEE Trans. Neural Networks, vol. 8, no. 5, pp. 997-1007, 1997.
21. P. J. Werbos, A Menu of Designs for Reinforcement Learning Over Time, Neural Networks for Control, W. T. Miller, R. S. Sutton, and P. J. Werbos (eds.), pp. 6796, MIT Press, Cambridge, MA, 1990.
22. P. J. Werbos, Advanced Forecasting Methods for Global Crisis Warning and Models of Intelligence, General Systems Yearbook, 1997.
23. G. G. Lendaris and T. Shannon, Application considerations for the DHP methodology, Proc. International Joint Conference on Neural Networks, Anchorage, AK, 1998.
24. G. K. Venayagamoorthy, R. G. Harley, and D. C. Wunsch, Comparison of heuristic dynamic programming and dual heuristic programming adaptive critics for neurocontrol of a turbogenerator, IEEE Trans. Neural Networks, vol. 13, no. 3, pp. 764-773, 2002.
25. A. Barto, R. Sutton, and C. Anderson, Neuronlike elements that can solve difficult learning control problems, IEEE Trans. Systems, Man, and Cybernetics, vol. 3, no. 5, pp. 834-846,1983.
26. P. J. Werbos, Applications of advances in nonlinear sensitivity analysis, System Modeling and Optimization: Proc. Of the I Oth IFIP Conference, R. F. Drenick and F. Kozin (eds.), Springer-Verlag, New York, 1982.
27. C. Watkins, Learning from Delayed Rewards, Ph.D. Thesis, Cambridge University, Cambridge, England, 1989.
28. nd Ed., Wiley, New York, 1977.
29. R. E. Bellman and S. E. Dreyfus, Functional Approximation and Dynamic Programming, Math. Tables and Other Aids Comp., Athena Scientific, Belmont, MA, 1995.
30. G. J. Gordon, Stable Function Approximation in Dynamic Programming, Technical Report CMU-CS-95-103, Carnegie Mellon University, Pittsburgh, 1995.
31. D. A. White and D. A. Sofge (eds.), Handbook of Intelligent Control, Van Nostrand Reinhold, New York, 1992.
32. rd Ed., Harcourt, Brace, Janovich, San Diego, 1988.
33. The MathWorks, Inc., Getting Started with MATLAB, http://www.mathworks.com, Version 5, September 1998.
34. J. N. Tsitsiklis and B. Van Roy, An analysis of temp oral-difference learning with function approximation, IEEE Trans. Automatic Control, vol. 42, no. 5, pp. 674-690,1997.
35. P. J. Werbos, Backpropagation through time: what it does and how to do it, Proc. Of the IEEE, vol. 78, no. 10, pp. 1550-1560, 1990.
36. G. G. Lendaris, L. Schultz, and T. T. Shannon, Adaptive critic design for intelligent steering and speed control of a 2-axle vehicle, Proc. International Joint Conference on Neural Networks, Como, Italy, 2000.
37. D. Han and S. N. Balakrishnan, Adaptive critic based neural networks for control-constrained agile missile control, Proc. American Control Conference, San Diego, pp. 2600-2604,1999.
38. G. Saini and S. N. Balakrishnan, Adaptive critic based neurocontroller for autolanding of aircraft, Proc. American Control Conference, Albuquerque, NM, pp. 1081-1085,1997.
39. S. Ferrari, Algebraic and adaptive learning in neural control systems, Ph.D. Thesis, Princeton University, Princeton, NJ, 2002.
40. S. N. Balakrishnan and V. Biega, Adaptive-critic-based neural networks for aircraft optimal control, Journal of Guidance, Control, and Dynamics, vol. 19, no. 4, pp. 893-898,1996.
41. K. KrishnaKumar and J. Neidhoefer, Immunized adaptive critics for level-2 intelligent control, Proc. IEEE Int. Conf. Systems, Man and Cybernetics, vol. 1, pp. 856-861,1997.