Hamilton-Jacobi-Bellman equations and approximate dynamic programming on time scales

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 38, Issue 4, 2008, Pages 918-923

  Seiffertt, John ^a   Sanyal, Suman ^b   Wunsch, Donald C ^a  

^a Architectural and Environmental Engineering   (United States)

^b MISSOURI UNIVERSITY OF SCIENCE AND TECHNOLOGY   (United States)

Author keywords

Approximate dynamic programming (ADP); Dynamic equations; Hamilton Jacobi Bellman (HJB) equation; Reinforcement learning; Time scales

Indexed keywords

CALCULATIONS; DIFFERENTIATION (CALCULUS); DYNAMIC PROGRAMMING; MATHEMATICAL PROGRAMMING; MATHEMATICAL TECHNIQUES; REINFORCEMENT; REINFORCEMENT LEARNING; SYSTEMS ENGINEERING; TREES (MATHEMATICS);

APPLICATIONS.; APPROXIMATE DYNAMIC PROGRAMMING; APPROXIMATE DYNAMIC PROGRAMMING (ADP); BACKWARD INDUCTION; DYNAMIC EQUATIONS; FUNDAMENTAL PROBLEMS; HAMILTON-JACOBI-BELLMAN (HJB) EQUATION; HAMILTON-JACOBI-BELLMAN EQUATIONS; ON TIME; STOCHASTIC CONTROL; TIME SCALES; TIME SCALING;

STOCHASTIC PROGRAMMING;

ARTICLE; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; FEEDBACK SYSTEM; NONLINEAR SYSTEM; SYSTEM ANALYSIS; SYSTEMS THEORY; THEORETICAL MODEL;

ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; FEEDBACK; MODELS, THEORETICAL; NONLINEAR DYNAMICS; PROGRAMMING, LINEAR; SYSTEMS THEORY;

EID: 49049117538     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2008.923532     Document Type: Article

References (47)

1. M. A. Abramson and C. Audet, "Convergence of mesh adaptive direct search to second-order stationary points," SIAM J. Optim., Vol. 17, no. 2, pp. 606-619, Aug. 2006
2. C. D. Ahlbrandt, M. Bonnet, and J. Ridenhour, "Hamiltonian system on time scales," J. Math. Anal. Appl., Vol. 250, no. 2, pp. 561-578, Oct. 2000.
3. C. D. Ahlbrandt, and C. Morian, "Partial differential equations on time scales," J. Comput. Appl. Math., Vol. 141, no. 1, pp. 35-55, Apr. 2002.
4. F. M. Atici, D. C. Biles, and A. Lebedinsky, "An application of time scales to economics," Math. Comput. Model., Vol. 43, no. 7/8, pp. 718-726, Apr. 2006.
5. M. Bardi and I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations, set. Systems & Control: Foundations & Applications. Boston, MA: Birkhäuser, 1997. With Appendices by Maurizio Falcone and Pierpaolo Soravia.
6. T. Başar and and G. J. Olsder, Dynamic Noncooperative Game Theory set. Classics in Applied Mathematics, Vol. 23. Philadelphia, PA: SIAM, 1999. reprint of the second (1995) edition.
7. R. Bellman, Dynamic Programming. Princeton, NJ: Princeton Univ. Press, 1957.
8. R. E. Bellman and S. E. Dreyfus, Programarea Dinamicã Aplicatã ser. Translated from the English. Bucharest, Romania: Editura Tehnicã, 1967.
9. D. P. Bertsekas, Dynamic Programming and Optimal Control, 2nd ed, Vol. I. Belmont, MA: Anthena Scientific, 2001.
10. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming, ser. Optimization and Neural Computation Series, 3. Belmont MA: Athena Scientific, 1996.
11. M. Bohner, "Calculus of variations on time scales," Dyn. Syst. Appl., vol. 13, no. 3/4, pp. 339-349, 2004.
12. M. Bohner, M. Fan, and J. Zhang, "Periodicity of scalar dynamic equations and applications to population models," J. Math. Anal. Appl., vol. 330, no. 1, pp. 1-9, Jun. 2007.
13. M. Bohner and T. Hudson, "Euler-type boundary value problems in quantum calculus," Int. J. Appl. Math. Stat., vol. 9, no. J07, pp. 19-23, Jun. 2007.
14. M. Bohner and G. S. Guseinov, "Double integral calculus of variations on time scales," Comput. Math. Appl., vol. 54, no. 1, pp. 45-57, Jul. 2007.
15. M. Bohner and G. S. Guseinov, "Multiple integration on time scales," Dyn. Syst. Appl., vol. 14, no. 3/4, pp. 579-606, 2005.
16. M. Bohner and G. S. Guseinov, "Partial differentiation on time scales," Dyn. Syst. Appl., vol. 13, no. 3/4, pp. 351-379, 2004.
17. M. Bohner and A. Peterson, Eds., Advances in Dynamic Equations on Time Scales, Boston MA: Birkhäuser 2003.
18. M. Bohner and A. Peterson, Dynamic Equations on Time Scales: An Introduction With Applications. Boston, MA: Birkhäuser, 2001.
19. J. J. DaCunha, "Instability results for slowly time varying linear dynamic systems on time scales," J. Math. Anal. Appl., vol. 328, no. 2, pp. 1278-1289, Apr. 2007.
20. J. J. DaCunha, "Stability for time varying linear dynamic systems on time scales," J. Comput. Appl. Math., vol. 176, no. 2, pp. 381-410, Apr. 2005.
21. J. J. DaCunha, J.M. Davis, and P. K. Singh, "Existence results for singular three point boundary value problems on time scales," J. Math. Anal. Appl., vol. 295, no. 2, pp. 378-391, Jul. 2004.
22. P. W. Eloe, S. Hilger, and Q. Sheng, "A qualitative analysis on nonconstant graininess of the adaptive grids via time scales," Rocky Mt. J. Math., vol. 36, no. 1, pp. 115-133, 2006.
23. R. Ferreira and D. Torres, "Remarks on the calculus of variations on time scales," Int. J. Econ. Ecol. Stat., vol. 9, no. F07, pp. 65-73, Sep. 2007.
24. W. H. Fleming and R. W. Rishel, "Deterministic and stochastic optimal control," in Applications of Mathematics, vol. 1. Berlin, Germany: Springer-Verlag, 1975.
25. I. M. Gelfand and S. V. Fomin, Calculus of Variations, ser. Revised English Edition Translated, R. A. Silverman, Ed. Englewood Cliffs, NJ: Prentice-Hall, 1963.
26. I. Gravagne, R. J. Marks, II, J. Davis, and J. DaCunha, "Application of time scales to real world time communications networks," in Proc. Amer. Math. Soc. West. Sect. Meeting, 2004.
27. I. Gravagne, J. Davis, R. J.Marks, II, "How deterministic must a real-time controller be?" in Proc. IEEE/RSJ Int. Conf., Edmonton, AB, Canada, Aug. 2005, pp. 3856-3861.
28. I. Gravagne, J. Davis, J. DaCunha, and R. J. Marks, II, "Bandwidth reduction for controller area networks using adaptive sampling," in Proc. Int. Conf. Robot. Autom., New Orleans, LA, Apr. 2004, pp. 2-6.
29. G. S. Guseinov, "Integration on time scales," J. Math. Anal. Appl. vol. 285, no. 1, pp. 107-127, Sep. 2003.
30. G. S. Guseinov and E. Özyý Imaz, "Tangent lines of generalized regular curves parametrized by time scales," Turkish J. Math., vol. 25, no. 4, pp. 553-562, 2001.
31. S. Hilger, "Analysis on measure chains - A unified approach to continuous and discrete calculus," Results Math., vol. 18, no. 1/2, pp. 18-56, 1990.
32. 1 solutions with variable endpoints," J. Math. Anal. Appl., vol. 289, no. 1, pp. 143-166, Jan. 2004.
33. D. G. Hull, Optimal Control Theory for Applications, ser. Mechanical Engineering Series. New York: Springer-Verlag, 2003.
34. B. Jackson, "Partial dynamic equations on time scales," J. Comput. Appl. Math., vol. 186, no. 2, pp. 391-415, Feb. 2006.
35. V. Kac and P. Cheung, Quantum Calculus, ser. Universitext. New York: Springer-Verlag, 2002.
36. F. L. Lewis, Optimal Control, ser. A Wiley-Interscience Publication. New York: Wiley, 1986.
37. R. J. Marks, II, I. A. Gravagne, J. M. Davis, and J. J. DaCunha, "Nonregressivity in switched linear circuits and mechanical systems," Math. Comput. Model., vol. 43, no. 11/12, pp. 1383-1392, Jun. 2006.
38. K. R. Messer, "Riccati techniques on a time scale," Panam. Math. J. vol. 13, no. 2, pp. 1-17, 2003.
39. C. Pötzsche, "Chain rule and invariance principle on measure chains," J. Comput. Appl. Math., vol. 141, no. 1/2, pp. 249-254, Apr. 2002.
40. W. B. Powell, Approximate Dynamic Programming: Solving the Curses of Dimensionality, ser. Wiley Series in Probability and Statistics. Hoboken, NJ: Wiley, 2007.
41. M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, ser. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. New York: Wiley, 1994.
42. J. Rogers and Q. Sheng, "Notes on the diamond-α dynamic derivative on time scales," J. Math. Anal. Appl., vol. 326, no. 1, pp. 228-241, 2007.
43. H. Rund, The Hamilton-Jacobi Theory in the Calculus of Variations: Its Role in Mathematics and Physics. New York: Van Nostrand, 1966.
44. S. Sanyal, "Stochastic dynamic equations," Ph.D. dissertation, Missouri Univ. Sci. Technol., Rolla, MO, 2008.
45. Q. Sheng, M. Fadag, J. Henderson, and J. M. Davis, "An exploration of combined dynamic derivatives on time scales and their applications," Nonlinear Anal.: Real World Appl., vol. 7, no. 3, pp. 395-413, Jul. 2006.
46. P. J. Werbos, "Backwards differentiation in AD and neural nets: Past links and new opportunities," in Automatic Differentiation: Applications, Theory, and Implementations, vol. 50. Berlin, Germany: Springer-Verlag, 2006, pp. 15-34.
47. D. A. White and D. A. Sofage, Eds., Handbook of Intelligent Control New York: Van Nostrand, 1992.