Ergodicity, stabilization, and singular perturbations for Bellman-Isaacs equations

Memoirs of the American Mathematical Society

Volumn 204, Issue 960, 2010, Pages 1-83

  Alvarez, Olivier ^a   Bardi, Martino ^b  

^a UNIVERSITÉ DE ROUEN   (France)

^b UNIVERSITY OF PADOVA   (Italy)

Author keywords

Averaging; Controlled diffusion processes; Differential games; Dimension reduction; Ergodic control; Hamilton Jacobi equations; Homogenization; Non resonance; Nonlinear parabolic equations; Optimal control; Oscillating initial data; Singular perturbations; Stabilization; Stochastic games; Viscosity solutions

Indexed keywords

EID: 84860612934     PISSN: 00659266     EISSN: 19476221     Source Type: Journal    
DOI: 10.1090/S0065-9266-09-00588-2     Document Type: Article

References (91)

1. V. I. Arnold and A. Avez, Problemes ergodiques de la mecanique classique, Gauthier-Villars, Paris, 1967, english translation: W. A. Benjamin, New York, 1968. MR0209436 (35:334)
2. O. Alvarez and M. Bardi, Viscosity solutions methods for singular perturbations in deterministic and stochastic control, SIAM J. Control Optim. 40(2001), 1159-1188. MR1882729 (2003a:49034)
3. -, Singular perturbations of degenerate parabolic PDEs: a general convergence result, Arch. Rational Mech. Anal. 170(2003), 17-61. MR2012646 (2004h:35012) (Pubitemid 37364563)
4. -, Ergodic problems in differential games, Advances in Dynamic Game Theory (S. Jorgensen, M. Quincampoix, and T. L. Vincent, eds.), Ann. Internat. Soc. Dynam. Games, vol. 9, Birkhauser, 2007, pp. 131-152. MR2341242 (2008k:91030)
5. O. Alvarez, M. Bardi, and C. Marchi, Multiscale problems and homogenization for second-order Hamilton-Jacobi equations, J. Differential Equations 243(2007), 349-387. MR2371792 (2009a:35030)
6. -, Multiscale singular perturbations and homogenization of optimal control problems, Geometric Control and Nonsmooth Analysis (F. Ancona, A. Bressan, P. Cannarsa, F. H. Clarke, and P. R. Wolenski, eds.), Ser. Adv. Math. Appl. Sci., 76, World Scientific Publishing, Hackensack, NJ, 2008, pp. 1-27. MR2487745
7. Z. Artstein and V. Gaitsgory, Tracking fast trajectories along a slow dynamics: a singular perturbations approach, SIAM J. Control Optim. 35(1997), 1487-1507. MR1466912 (98h:49027) (Pubitemid 127586294)
8. -, The value function of singularly perturbed control systems, Appl. Math. Optim. 41(2000), 425-445. MR1739395 (2001b:49035) (Pubitemid 30596843)
9. L. Arnold and W. Kliemann, On unique ergodicity of degenerate diffusions, Stochastics 21(1987), 41-61. MR899954 (88g:60190)
10. M. Arisawa and P.-L. Lions, On ergodic stochastic control, Comm. Partial Differential Equations 23(1998), 2187-2217. MR1662180 (2000a:49052) (Pubitemid 128361780)
11. G. Allaire, Homogenization and two-scale convergence, SIAM J. Math. Anal. 23(1992), 1482-1518. MR1185639 (93k:35022)
12. M. Arisawa, Ergodic problem for the Hamilton-Jacobi-Bellman equation I, Ann. Inst. H. Poincare Anal. Non Lineaire 14(1997), 415-438. MR1464529 (98k:49040)
13. -, Ergodic problem for the Hamilton-Jacobi-Bellman equation II, Ann. Inst. H. Poincare Anal. Non Lineaire 15(1998), 1-24. MR1614615 (99g:49029)
14. Z. Artstein, Invariant measures of differential inclusions applied to singular perturbations, J. Differential Equations 152(1999), 289-307. MR1674533 (99m:34023)
15. -, On the value function of singularly perturbed optimal control systems, Proceedings of the 43rd IEEE Conference on Decision and Control, Paradise Island, Bahamas, 2004, pp. 432-437. (Pubitemid 40291370)
16. M. Bardi, On differential games with long-time-average cost, Advances in dynamic games and their applications, Ann. Internat. Soc. Dynam. Games 10, Birkhauser Boston, Inc., Boston, MA, 2009, pp. 3-18. MR2521660
17. G. Barles, Some homogenization results for non-coercive Hamilton-Jacobi equations, Calc. Var. Partial Differential Equations 30(2007), 449-466. MR2332422 (2008f:35011)
18. F. Bagagiolo and M. Bardi, Singular perturbation of a finite horizon problem with state-space constraints, SIAM J. Control Optim. 36(1998), 2040-2060. MR1638940 (99h:49040) (Pubitemid 128492177)
19. G. K. Basak, V. S. Borkar, and M. K. Ghosh, Ergodic control of degenerate diffusions, Stochastic Anal. Appl. 15(1997), 1-17. MR1429854 (97j:93061) (Pubitemid 127383033)
20. M. Bardi and I. Capuzzo-Dolcetta, Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations, Birkhauser, Boston, 1997. MR1484411 (99e:49001)
21. A. Bensoussan, Perturbation methods in optimal control, Wiley/Gauthiers-Villars, Chichester, 1988. MR949208 (89m:93002)
22. P. Bettiol, On ergodic problem for Hamilton-Jacobi-Isaacs equations, ESAIM Control Optim. Calc. Var. 11(2005), 522-541. MR2167873 (2006k:35026) (Pubitemid 43107039)
23. V. Borkar and V. Gaitsgory, On existence of limit occupational measures set of a controlled stochastic differential equation, SIAM J. Control Optim. 44(2005), 1436-1473. MR2178039 (2006f:93103) (Pubitemid 44073974)
24. M. Bardi and F. Da Lio, On the strong maximum principle for fully nonlinear degenerate elliptic equations, Arch. Math (Basel) 73(1999), 276-285. MR1710100 (2001h:35024)
25. -, Propogation of maxima and strong maximum principle for viscosity solutions of degenerate elliptic equations. I: convex operators., Nonlinear Anal. Ser. A: Theory Methods 44(2001), 991-1006. MR1829815 (2002g:35088) (Pubitemid 32404819)
26. M. Biroli, U. Mosco, and N. Tchou, Homogenization for degenerate operators with periodical coefficients with respect to the Heisenberg group, C. R. Acad. Sci. Paris Ser. I Math. 322(1996), 439-444. MR1381780 (97e:35015)
27. S. Brahim-Otsmane, G. A. Francfort, and F. Murat, Correctors for the homogenization of the wave and heat equations, J. Math. Pures Appl. 71(1992), 197-231. MR1172450 (93d:35012)
28. J. M. Bony, Principe du maximum, inegalite de Harnack et unicite du probleme de Cauchy pour les operateurs elliptiques degeneres, Ann. Inst. Fourier (Grenoble) 19(1969), 277-304. MR0262881 (41:7486)
29. T. Bielecki and L. Stettner, On ergodic control problems for singularly perturbed Markov processes, Appl. Math. Optim. 11(1989), 131-161. MR998401 (90c:93111)
30. I. Birindelli and J. Wigniolle, Homogenization of Hamilton-Jacobi equations in the Heisenberg group, Commun. Pure Appl. Anal. 2(2003), 461-479. MR2019062 (2004i:35022)
31. X. Cabre and L. Caffarelli, Fully nonlinear elliptic equations, Amer. Math. Soc., Providence, 1995. MR1351007 (96h:35046)
32. I. Capuzzo-Dolcetta and J.-L. Menaldi, Asymptotic behavior of the first order obstacle problem, J. Differential Equations 75(1988), 303-328. MR961158 (89j:49023)
33. I. P. Cornfeld, S. V. Fomin, and Ya. G. Sinai, Ergodic theory, Springer-Verlag, Berlin, 1982. MR832433 (87f:28019)
34. M. G. Crandall, H. Ishii, and P.-L. Lions, User's guide to viscosity solutions of second order partial differential equations, Bull. Amer. Math. Soc. (N. S.) 27(1992), 1-67. MR1118699 (92j:35050)
35. F. Colonius and W. Kliemann, The dynamics of control, Birkhauser, Boston, 2000. MR1752730 (2001e:93001)
36. p-theory for fully nonlinear uniformly parabolic equations, Comm. Partial Differential Equations 25(2000), 1997-2053. MR1789919 (2003b:35093)
37. F. DaLio, Remarks on the strong maximum principle for degenerate parabolic equations, Comm. Pure Appl. Analysis 3(2004), 395-415. MR2098291 (2005i:35121) (Pubitemid 41301954)
38. A. L. Dontchev and T. Zolezzi, Well-posed optimization problems, Lecture Notes in Mathematics, no. 1543, Springer-Verlag, Berlin, 1993. MR1239439 (95a:49002)
39. L. Evans and H. Ishii, Differential games and nonlinear first order PDE on bounded domains, Manuscripta Math. 49(1984), 109-139. MR767202 (86f:35048)
40. L. Evans and P. E. Souganidis, Differential games and representation formulas for solutions of Hamilton-Jacobi-Isaacs equations, Indiana Univ. Math. J. 33(1984), 773-797. MR756158 (86d:90185)
41. L. Evans, The perturbed test function method for viscosity solutions of nonlinear P. D. E., Proc. Roy. Soc. Edinburgh Sect. A 111(1989), 359-375. MR1007533 (91c:35017)
42. -, Periodic homogenisation of certain fully nonlinear partial differential equations, Proc. Roy. Soc. Edinburgh Sect. A 120(1992), 245-265. MR1159184 (93a:35016)
43. -, A survey of partial differential equations methods in weak KAM theory, Comm. Pure Appl. Math. 57(2004), 445-480. MR2026176 (2005b:49038)
44. W. H. Fleming and W. M. McEneaney, Risk-sensitive control on an infinite time horizon, SIAM J. Control Optim. 33(1995), 1881-1915. MR1358100 (96j:93070)
45. W. H. Fleming and P. E. Souganidis, On the existence of value functions of two-players, zero-sum stochastic differential games, Indiana Univ. Math. J. 38(1989), 293-314. MR997385 (90e:93089)
46. W. H. Fleming and H. M. Soner, Controlled Markov processes and viscosity solutions, 2nd edition, Springer-Verlag, Berlin, 2006. MR2179357 (2006e:93002)
47. B. Franchi and M. C. Tesi, Two-scale homogenization in the Heisenberg group, J. Math. Pures Appl. 81(2002), 495-532. MR1912410 (2003d:35016)
48. V. Gaitsgory, Suboptimization of singularly perturbed control systems, SIAM J. Control Optim. 30(1992), 1228-1249. MR1178660 (93h:49044) (Pubitemid 23572509)
49. -, Limit Hamilton-Jacobi-Isaacs equations for singularly perturbed zero-sum differential games, J. Math. Anal. Appl. 202(1996), 862-899. MR1408359 (97i:90127) (Pubitemid 126167115)
50. -, On a representation of the limit occupational measures set of a control system with applications to singularly perturbed control systems, SIAM J. Control Optim. 43(2004), 325-340. MR2082704 (2005g:49049) (Pubitemid 40285877)
51. V. Gaitsgory and A. Leizarowitz, Limit occupational measures set for a control system and averaging of singularly perturbed control systems, J. Math. Anal. Appl. 233(1999), 461-475. MR1689645 (2000c:93064)
52. D. A. Gomes, Hamilton-Jacobi methods for vakonomic mechanics, NoDEA Nonlinear Differential Equations Appl. 14(2007), 233-257. MR2364892 (2009a:37121)
53. L. Grüne, On the relation between discounted and average optimal value functions, J. Differential Equations 148(1998), 65-99. MR1637521 (99e:49004) (Pubitemid 128343323)
54. M. K. Ghosh and K. S. M. Rao, Differential games with ergodic payoff, SIAM J. Control Optim. 43(2005), 2020-2035. MR2179475 (2006g:91026) (Pubitemid 41641374)
55. D. Gilbarg and N. S. Trudinger, Elliptic partial differential equations of second order, 2nd ed., Springer-Verlag, New York, 1983. MR737190 (86c:35035)
56. L. Hormander, Hypoelliptic second order differential equations, Acta Math. 119(1968), 147-171. MR0222474 (36:5526)
57. R. Z. Has'minskiǐ, Stochastic stability of differential equations, Sijthoff & Noordhoff, Alphen aan den Rijn, 1980. MR600653 (82b:60064)
58. K. Ichihara and H. Kunita, A classification of the second order degenerate elliptic operators and its probabilistic characterization, Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 30(1974), 235-254. MR0381007 (52:1904)
59. -, Supplements and corrections to the paper: "A classification of the second order degenerate elliptic operators and its probabilistic characterization", Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 39(1977), 81-84. MR0488328 (58:7877)
60. H. Ishii and P.-L. Lions, Viscosity solutions of fully nonlinear second-order elliptic partial differential equations, J. Differential Equations 83(1990), 26-78. MR1031377 (90m:35015)
61. V. V. Jikov, S. M. Kozlov, and O. A. Oleinik, Homogenization of differential operators and integral functionals, Springer-Verlag, Berlin, 1994. MR1329546 (96h:35003b)
62. R. Jensen and P.-L. Lions, Some asymptotic problems in fully nonlinear elliptic equations and stochastic control, Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4) 11(1984), 129-176. MR752583 (85m:49054)
63. P. V. Kokotovic, H. K. Khalil, and J. O'Reilly, Singular perturbation methods in control: analysis and design, Academic Press, London, 1986. MR950486 (89e:93002)
64. Y. Kabanov and S. Pergamenshchikov, On convergence of attainability sets for controlled two-scale stochastic linear systems, SIAM J. Control Optim. 35(1997), 134-159. MR1430286 (97m:93069) (Pubitemid 127577112)
65. -, Two-scale stochastic systems. Asymptotic analysis and control, Springer-Verlag, Berlin, 2003. MR1942740 (2004c:93003)
66. T. G. Kurtz and R. H. Stockbridge, Existence of Markov controls and characterization of optimal Markov controls, SIAM J. Control Optim. 36(1998), 609-653. MR1616514 (99b:93051) (Pubitemid 128492191)
67. H. J. Kushner, Weak convergence methods and singularly perturbed stochastic control and filtering problems, Birkhauser, Boston, 1990. MR1102242 (92d:93003)
68. A. Leizarowitz, Order reduction is invalid for singularly perturbed control problems with a vector fast variable, Math. Control Signals Systems 15(2002), 101-119. MR1910691 (2003c:49041) (Pubitemid 34725451)
69. P.-L. Lions, Generalized solutions of Hamilton-Jacobi equations, Pitman, Boston, 1982. MR0667669 (84a:49038)
70. -, Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations. Part 1: The dynamic programming principle and applications, Part 2: Viscosity solutions and uniqueness, Comm. Partial Differential Equations 8(1983), 1101-1174 and 1229-1276. MR0709164 (85i:49043a); MR0709162 (85i:49043b)
71. -, Neumann type boundary conditions for Hamilton-Jacobi equations, Duke Math. J. 52(1985), 793-820. MR816386 (87h:35055)
72. P.-L. Lions, G. Papanicolaou, and S. R. S. Varadhan, Homogeneization of Hamilton-Jacobi equations, Unpublished, 1986.
73. P.-L. Lions and P. E. Souganidis, Homogenization of degenerate second-order PDE in periodic and almost periodic environments and applications, Ann. Inst. H. Poincare Anal. Non Lineaire 22(2005), 667-677. MR2171996 (2006d:35019) (Pubitemid 41328788)
74. D. S. Naidu, Singular perturbations and time scales in control theory and applications: an overview, Dyn. Contin. Discrete Impuls. Syst. Ser. B Appl. Algorithms 9(2002), 233-278. MR1897791 (2003b:93060)
75. R. E. O'Malley, Introduction to singular perturbations, Academic Press, New York-London, 1974. MR0402217 (53:6038)
76. M. Quincampoix and F. Watbled, Averaging method for discontinuous Mayer's problem of singularly perturbed control systems, Nonlinear Anal. 54(2003), 819-837. MR1992506 (2004e:49040)
77. M. V. Safonov, On the classical solution of nonlinear elliptic equations of second-order, Math. USSR-Izv. 33(1989), 597-612
78. Engl. transl. of Izv. Akad. Nauk SSSR Ser. Mat., 52 (1988). MR984219 (90d:35104)
79. P. Soravia, Pursuit-evasion problems and viscosity solutions of Isaacs equations, SIAM J. Control Optim. 31(1993), 604-623. MR1214756 (94b:49056) (Pubitemid 23710964)
80. N. N. Subbotina, Asymptotic properties of minimax solutions of Isaacs-Bellman equations in differential games with fast and slow motions, J. Appl. Math. Mech. 60(1996), 883-890. MR1627703 (99f:90209)
81. -, Asymptotics of singularly perturbed Hamilton-Jacobi equations, J. Appl. Math. Mech. 63(1999), 213-222. MR1717733 (2000j:35035) (Pubitemid 129312610)
82. -, Singular approximations of minimax and viscosity solutions to Hamilton-Jacobi equations, Proc. Steklov Inst. Math. Suppl. 1(2000), S210-S227. MR2066026 (2005e:35033)
83. -, Asymptotics for singularly perturbed differential games, Game theory and applications, Vol. VII, Nova Sci. Publ., Huntington, NY, 2001, pp. 175-196. MR1924394 (2003k:49078)
84. A. Swiech, Another approach to the existence of value functions of stochastic differential games, J. Math. Anal. Appl. 204(1996), 884-897. MR1422779 (97j:90091) (Pubitemid 126167822)
85. G. Terrone, Singular perturbation and homogenization problems in control theory, differential games and fully nonlinear partial differential equations, Ph. D. thesis, University of Padova, 2008.
86. N. S. Trudinger, Comparison principles and pointwise estimates for viscosity solutions of nonlinear elliptic equations, Rev. Mat. Iberoamericana 4(1988), 453-468. MR1048584 (91k:35084)
87. -, On regularity and existence of viscosity solutions of nonlinear second order elliptic equations, Partial Differential Equations and the Calculus of Variations (H. Brézis, ed.), vol. 2, Birkhauser, Boston, 1989, pp. 938-957. MR1034037 (90m:35041)
88. V. Veliov, A generalization of the Tikhonov theorem for singularly perturbed differential inclusions, J. Dynam. Control Systems 3(1997), 291-319. MR1472355 (98h:34034) (Pubitemid 127722421)
89. P. Walters, An introduction to ergodic theory, Springer-Verlag, New York, 1982. MR648108 (84e:28017)
90. C.-J. Xu, Subelliptic variational problems, Bull. Soc. Math. France 118(1990), 147-169. MR1087376 (92b:49008)
91. G. G. Yin and Q. Zhang, Continuous-time Markov chains and applications. A singular perturbation approach, Springer-Verlag, New York, 1998. MR1488963 (2000a:60142)