ADP: Goals, opportunities and principles

Handbook of Learning and Approximate Dynamic Programming

Volumn , Issue , 2004, Pages 3-44

  Werbos, Paul ^a  

^a NATIONAL SCIENCE FOUNDATION   (United States)

Author keywords

Books; Communities; Conferences; Dynamic programming; Humans; Optimization; Proposals

Indexed keywords

ECOSYSTEMS; OPTIMIZATION;

BOOKS; CONFERENCES; HUMANS; PROPOSALS;

DYNAMIC PROGRAMMING;

EID: 84893393162     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1109/9780470544785.ch1     Document Type: Chapter

References (60)

1. P. Werbos, What do neural nets and quantum theory tell us about mind and reality, in K. Yasue, M. Jibu and T Della Senta (eds.), No Matter, Never Mind: Proc. Of Toward a Science of Consciousness: Fundamental Approaches (Tokyo’99), John Benjamins Pub Co, 2002.
2. P. Werbos, “Optimization: A Foundation for understanding consciousness,” in D. Levine and W. Elsberry (eds.), Optimality in Biological and Artificial Networks, Erlbaum, 1997.
3. W. T. Miller, R. Sutton and P. Werbos (eds.), Neural Networks for Control, MIT Press, Cambridge, MA, 1990, now in paper.
4. D. White and D. Sofge (eds.), Handbook of Intelligent Control, Van Nostrand, New York, 1992.
5. P. Werbos, Neurocontrollers, in J. Webster (eds.), Encyclopedia of Electrical and Electronics Engineering, Wiley, New York, 1999.
6. M. Hoffert et al, Advanced Technology Paths to Global Climate Stability: Energy For a Greenhouse Planet, Science, 2002.
7. P. Werbos, Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences, Ph.D. Thesis, Committee on Applied Mathematics, Harvard U., 1974. Reprinted in its entirety in P. Werbos, The Roots of Backpropagation: From Ordered Derivatives to Neural Networks and Political Forecasting, Wiley, New York, 1994.
8. P. Werbos, Backpropagation through time: what it does and how to do it, Proc. IEEE, vol. 78, no. 10, 1990. Updated version reprinted as chapter 8 of [6].
9. J. A. Suykens, B. DeMoor and J. Vandewalle, Nlq theory: a neural control framework with global asymptotic stability criteria, Neural Networks, vol. 10, no. 4, pp. 615-637,1997.
10. P. Werbos, Stable Adaptive Control Using New Critic Designs. ArXiv.org: adap-org/9810001 (1998)
11. D. Jacobson and D. Mayne, Differential Dynamic Programming, American Elsevier, 1970.
12. P. He and J. Sarangapani, Neuro Emission Controller for Minimizing Cyclic Dispersion in Spark Ignition Engines, TNN submitted, 2003. Condensed versions are in press in IJCNN 2003 Proc. (IEEE) and CCA 2003 Proc. (IEEE).
13. F. Lewis, J. Campos and R. Selmic, Neuro-Fuzzy Control of Industrial Systems with Actuator Nonlinearities, SIAM, Philadelphia, 2002.
14. E. A. Feigenbaum and J. Feldman, Computers and Thought, McGraw-Hill, 1963.
15. J. Von Neumann and O. Morgenstem, The Theory of Games and Economic Behavior, Princeton University Press, Princeton, NJ, 1953.
16. H. Raiffa, Decision Analysis, Addison-Wesley, Reading, MA 1968.
17. P. Werbos, Rational approaches to identifying policy objectives, Energy: The International Journal, vol. 15, no. 3/4, pp. 171-185, 1990.
18. D. F. Walls and G. F. Milbum, Quantum Optics, Springer, New York, 1994.
19. D. B. Fogel, Blondie24: Playing at the Edge of AI, Morgan-Kauffman, San Francisco, 2001.
20. T. Landelius, Reinforcement Learning and Distributed Local Model Synthesis, Ph.D. thesis and Report No. 469, Department of Electrical Engineering, Linköping U., 58183, Linköping, Sweden.
21. R. F. Stengel, Optimal Control and Estimation, Dover edition, 1994.
22. K. Narendra and A. Annaswamy, Stable Adaptive Systems, Prentice-Hall, Englewood Cliffs, NJ, 1989; Hemisphere, Washington, DC, 1982.
23. R. Howard, Dynamic Programming and Markhov Processes, MIT Press, Cambridge, MA 1960.
24. C. J. C. H. Watkins, Learning From Delayed Rewards, Ph.D. thesis, University of Cambridge, England, 1989.
25. Watkins W. and Dayan D., Technical note: Q-leaming, Machine Learning, vol. 8, no. 3/4, pp. 279-292, 1992.
26. P. Werbos, Neural networks for control and system identification, IEEE Proc. CDC89, IEEE, 1989.
27. D. P. Bertsekas and J. N. Tsisiklis, Neuro-Dynamic Programming, Athena Scientific, Belmont, MA, 1996.
28. A. Barto, R. Sutton and C. Anderson, Neuronlike adaptive elements that can solve difficult learning control problems, IEEE Trans. SMC, vol. 13, no. 5, pp. 834-846,1983.
29. P. Werbos, The elements of intelligence, Cybernetica (Namur), no. 3,1968.
30. A. Biyson and Y. C. Ho, Applied Optimal Control, Ginn, 1969.
31. J. S. Baras and N. S. Patel, Information state for robust control of set-valued discrete time systems, Proc. 34th Conf. Decision and Control (CDC), IEEE, pp. 2302,1995.
32. S. Mukhopadhyay and B. Jain, Multi-agent Markhov decision processes with limited agent communication, Proc. Of the Inti Joint Conf on Control Applications and Inti Symposium on Intelligent Control (IEEE CCA/ISIC01), IEEE: 2001.
33. T. Kohonen, Self-Organizing Maps, New York: Spinger, 1997, Second Edition. Also see H. Ritter, T. Martinetz, and K. Schulten, Neural Computation and Self-Organizing Maps, Addison-Wesley, Reading, MA, 1992.
34. J. Albus, Outline of Intelligence, IEEE Trans. Systems, Man and Cybernetics, vol. 21, no. 2,1991.
35. P. Werbos, Changes in global policy analysis procedures suggested by new methods of optimization, Policy Analysis and Information Systems, vol. 3, no. 1, 1979.
36. P. Werbos, Backpropagation: General Principles and Issues for Biology, in D. Fogel and C. Robinson (eds.), Computational Intelligence: The Experts Speak, IEEE, 2003.
37. A. R. Barron, Universal approximation bounds for superpositions of a sigmoidal function, IEEE Trans. Info. Theory, vol. 39, no. 3, pp. 930-945,1993.
38. P. Werbos, Elastic fuzzy logic: a better fit to neurocontrol and true intelligence, J. Intelligent & Fuzzy Systems, vol. 1, no. 4,1993.
39. X. Z. Pang and P. Werbos, Neural network design for J function approximation in dynamic programming, Math. Modelling and Scientific Computing, vol. 5, no. 2/3,1996 (physically 1998). Available also asadap-org/9806001 atarXiv.org.
40. P. Werbos and X. Z. Pang, “Generalized maze navigation: SRN critics solve what feedforward or Hebbian nets cannot,” Proc. Conf. Systems, Man and Cybernetics (SMC), Beijing, IEEE, 1996.
41. T. Yang and L. O. Chua, Implementing Back-Propagation-Through-Time Learning Algorithm Using Cellular Neural Networks, Int 1 J. Bifurcation and Chaos, vol. 9, no. 9, pp. 1041-1074,1999.
42. P. Werbos posted at http://www.iamcm.org, and [47].
43. P. Werbos, Consistency of HDP applied to a simple reinforcement learning problem, Neural Networks, 1990.
44. P. Werbos, Advanced forecasting for global crisis warning and models of intelligence, General Systems Yearbook, 1977.
45. B. Widrow, N. Gupta and S. Maitra, Punish/reward: learning with a Critic in adaptive threshold systems, IEEE Trans. SMC, vol. 5, pp. 455-465,1973.
46. P. Werbos, Building and understanding adaptive systems: A statistical/numerical approach to factory automation and brain research, IEEE Trans. SMC, 1987.
47. J. Tsitsiklis and B. Van Roy, An analysis of temporal-difference learning with function approximation, IEEE Trans. Auto. Control, vol. 42, no. 5, 1997.
48. T. B. Trafalis and S. Kasap, Artificial neural networks in optimization and applications, Handbook of Applied Optimization, in P. M. Pardalos and M. G. C. Re-sende (eds.), Cambridge University Press, 2000.
49. K. Narendra and S. Mukhopadhyay, Intelligent control using neural networks, in M. Gupta andN. Sinha (eds.), Intelligent Control Systems, IEEE Press, 1996.
50. P. Werbos, A Brain-Like Design To Learn Optimal Decision Strategies in Complex Environments, in M. Kamy, K. Warwick and V. Kurkova (eds.), Dealing with Complexity: A Neural Networks Approach, Springer, London, 1998.
51. S. Amari and N. Kasabov, Brain-Like Computing and Intelligent Information Systems, Springer, 1998.
52. F. Yuan, L. Feldkamp, G. Puskorius and L. Davis, A simple solution to the bioreactor benchmark problem by application of Q-leaming, Proc. World Congress on Neural Networks, Erlbaum, New York, 1995.
53. T. Shibata, Hierarchical Intelligent Control of Robotic Motion, Master’s Thesis, chapter 5, Dept, of Electronic Mechanical Engineering, Nagoya University, Japan, 1992.
54. P. Werbos, Applications of advances in nonlinear sensitivity analysis, in R. Drenick and F. Kozin (eds.), System Modeling and Optimization: Proc. IFIP Conf. (1981), Springer 1982; reprinted as chapter 7 in [6].
55. D. O. Hebb, The Organization of Behavior, Wiley, New York, 1949.
56. J. C. Houk, J. L. Davis and D. G. Beiser (eds.), Models of Information Processing in the Basal Ganglia, MIT Press, Cambridge, MA, 1995.
57. K. H. Pribram, (ed.), Brain and Values, Erlbaum: Hillsdale, NJ, 1998. (See also earlier books edited by Pribram in the same series from Erlbaum.)
58. P. Werbos, Multiple Models for Approximate Dynamic Programming and True Intelligent Control: Why and How, in K. Narendra (ed.), Proc. 10th Yale Conf. on Learning and Adaptive Systems, New Haven: K. Narendra, EE Dept., Yale University, 1998.
59. R. Sutton, TD Models: Modeling the World at a Mixture of Time Scales, CMP-SCI Technical Report, pp. 95-114, University of Massachusetts at Amherst, December 1995, later published in Proa 12th Int. Conf. Macjine Learning, pp. 531-539, Morgan Kauhnann, 1995.
60. C. H. Anderson, B. Olshausen and D. Van Essen, Routing networks in visual cortex, in M. Arbib (ed.), The Handbook of Brain Theory and Neural Networks, First Edition, pp. 823-826, MIT Press, Cambridge, MA, 1995.