Chapter 14 Numerical dynamic programming in economics

Handbook of Computational Economics

Volumn 1, Issue , 1996, Pages 619-729

  Rust, John ^a  

^a UNIVERSITY OF WISCONSIN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 70350192140     PISSN: 15740021     EISSN: None     Source Type: Book Series    
DOI: 10.1016/S1574-0021(96)01016-7     Document Type: Review

References (186)

1. Adams R.A. Sobolev spaces (1975), Academic Press, New York
2. Chapter 4 in this Handbook. Anderson E., McGrattan E., Hansen L.P., and Sargent T.J. Mechanics of forming and estimating dynamic linear economies (1996)
3. Archibald T.W., McKinnon K.I.M., and Thomas L.C. Serial and parallel value iteration algorithms for discounted Markov decision processes. European Journal of Operations Research 67 (1993) 188-203
4. Arrow K.J. Applications of control theory to economic growth. In: Dantzig G.B., and Veinott A.F. (Eds). Mathematics for decision sciences, Part 2 (1968), Amer. Mathematical Soc., Providence, RI
5. Arrow K.J., Harris T., and Marschak J. Optimal inventory policy. Econometrica 19 3 (1951) 250-272
6. Bakhvalov N.S. On optimal estimates of the rate of convergence of quadrature processes and integration methods of the Monte Carlo type. In the miscellany Numerical methods for solving differential and integral equations and quadrature formulae (1964), Nauka, Moscow 5-63
7. Barron A.R. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information Theory 39 3 (1993) 930-945
8. Barto A.G., Bradtke S.J., and Singh S.P. Learning to act using real-time dynamic programming. Artificial Intelligence 75 (1995) 81-138
9. Bellman R. Functional equations in the theory of dynamic programming: Positivity and quasilinearity. Proceedings of the National Academy of Sciences 41 (1955) 743-746
10. Bellman R. Dynamic programming (1957), Princeton Univ. Press, Princenton, NJ
11. Bellman R., and Dreyfus S. Applied dynamic programming (1962), Princeton Univ. Press, Princenton, NJ
12. Bellman R., Kalaba R., and Kotkin B. Polynomial approximation: A new computational technique in dynamic programming: Allocation processes. Mathematics of Computation 17 (1963) 155-161
13. Bertsekas D. Dynamic programming deterministic and stochastic models (1987), Prentice-Hall, New York
14. Bertsekas D. Monotone mappings with application in dynamic programming. SIAM Journal of Control and Optimization 15 3 (1977) 438-464
15. Bertsekas D. Convergence of discretization procedures in dynamic programming. IEEE Transactions on Automatic Control 20 (1975) 415-419
16. Bertsekas D., and Castañon D. Adaptive aggregation methods for infinite horizon dynamic programming. IEEE Transactions on Automatic Control 34 6 (1989) 589-598
17. Bertsekas D.P., and Shreve S.E. Stochastic optimal control: The discrete time case (1978), Academic Press, New York
18. Bertsekas D.P., and Tsitsiklis J.N. Simulated annealing. Statistical Science 8 (1993) 1-16
19. Bhattacharya R.N., and Majumdar M. Controlled semi-Markov models - the discounted case. Journal of Statistical Planning and Inference 21 (1989) 365-381
20. Blackwell D. Discrete dynamic programming. Annals of Mathematical Statistics 33 (1962) 719-726
21. Blackwell D. Discounted dynamic programming. Annals of Mathematical Statistics 36 (1965) 226-235
22. Blackwell D. Positive dynamic programming. Proceedings of the 5th Berkeley symposium on mathematical statistics and probability Vol. 1 (1967), University of California Press, Berkeley, CA 415-418
23. Brown P.N., and Saad Y. Hybrid Krylov methods for nonlinear systems of equations. SIAM Journal on Scientific and Statistical Computing 11 3 (1990) 450-481
24. Brock W.A. Asset prices in a production economy. In: McCall J.J. (Ed). The economics of information and uncertainty (1982), Univ. of Chicago Press, Chicago, IL
25. Brock W.A., and Mirman L.J. Optimal economic growth under uncertainty: The discounted case. Journal of Economic Theory 4 (1972) 479-513
26. Chow C.S., and Tsitsiklis J.N. The complexity of dynamic programming. Journal of Complexity 5 (1989) 466-488
27. Chow C.S., and Tsitsiklis J.N. An optimal multigrid algorithm for continuous state discrete time stochastic control. IEEE Transactions on Automatic Control 36 8 (1991) 898-914
28. Chow G.C. Analysis and control of dynamic economic systems (1975), Wiley, New York
29. Chow G.C. Optimum control of stochastic differential equation systems. Journal of Economic Dynamics and Control 1 (1979) 143-175
30. Chow G.C. Econometric analysis by control methods (1981), Wiley, New York
31. Chow G.C. Dynamic optimization without dynamic programming. Economic Modelling 3 (1992) 3-9
32. Chow G.C. Optimal control without solving the Bellman equation. Journal of Economic Dynamics and Control 17 (1993) 621-630
33. Chow G.C. Computation of optimum control functions by Lagrange multipliers. In: Belsley D.A. (Ed). Computational techniques for econometrics and economic analysis (1993), Kluwer Academic Publishers, Dordrecht 65-72
34. manuscript. Chow G.C. Dynamic economics: An exposition by the Lagrange method (1996), Princeton University
35. Chew S.H., and Epstein L.G. The structure of preferences and attitudes towards the timing and resolution of uncertainty. International Economic Review 30 1 (1989) 103-118
36. Chou A.W. On the optimality of the Krylov information. Journal of Complexity 3 (1987) 545-559
37. Christiano L.J. Linear-quadratic approximation and value function iteration: A comparison. Journal of Business and Economic Statistics 8 1 (1990) 99-113
38. manuscript. Christiano L.J., and Fisher J.D.M. Algorithms for solving dynamic models with occasionally binding constraints (1994), Department of Economics, University of Western Ontario
39. Coleman W.J. Solving the stochastic growth model by policy-function iteration. Journal of Business and Economic Statistics 8 (1990) 27-29
40. Coleman W.J. Solving nonlinear dynamic models on parallel computers. Journal of Business and Economic Statistics 11 3 (1993) 325-330
41. manuscript. Coleman W.J., and Liu M. Incomplete markets and inequality constraints: Some theoretical and computational issues (1994), Fuqua School of Business, Duke University
42. Cryer C.W. Numerical functional analysis (1982), Clarendon Press, Oxford
43. Daniel J.W. Splines and efficiency in dynamic programming. Journal of Mathematical Analysis and Applications 54 (1976) 402-407
44. Dantzig G.B., Harvey R.P., Landowne Z.F., and McKnight R.D. DYGAM - A computer system for the solution of dynamic programs (1974), Control Analysis Corporation, Palo Alto, CA
45. Davis P.J., and Rabinowitz P. Methods of numerical integration (1975), Academic Press, New York
46. de Boor C. A practical guide to splines (1978), Springer, New York
47. Denardo E.V. Contraction mappings underlying the theory of dynamic programming. SIAM Review 9 (1967) 165-177
48. Doshi B.T. Continuous time control of Markov processes on an arbitrary state space: Discounted rewards. Annals of Statistics 4 6 (1976) 1219-1235
49. Dudley R.M. Real analysis and probability (1989), Wadsworth, Pacific Groves, CA
50. Dutta P. What do discounted optima converge to?: A theory of discount rate asymptotics in economic models. Journal of Economic Theory 55 1 (1991) 64-94
51. Dynkin E.B., and Juskevic A.A. Controlled Markov processes (1979), Springer, New York
52. Eckstein Z., and Wolpin K. The specification and estimation of dynamic stochastic discrete choice models. Journal of Human Resources 24 4 (1989) 562-598
53. Epstein L.G., and Zin S.E. Substitution, risk aversion, and the temporal behavior of consumption and asset returns: A theoretical framework. Econometrica 57 4 (1989) 937-970
54. Epstein L.G., and Zin S.E. First-order risk aversion and the equity premium puzzle. Journal of Monetary Economics 26 (1990) 387-407
55. Evtushenko Y.G. Numerical optimization techniques. Optimization Software Division (1985), Springer, New York
56. Fleming W.H., and Mete Soner H. Controlled Markov processes and viscosity solutions (1993), Springer, New York
57. Fox B.L. Discretizing dynamic programming. Journal of Optimization Theory and Its Applications 11 (1973) 228-234
58. Garey M.R., and Johnson D.S. Computers and intractibility: A guide to the theory of NP-completeness (1983), Freeman, New York
59. Geman S., and Hwang C. Diffusions for global optimization. SIAM Journal on Control and Optimization 24 5 (1986) 1031-1043
60. manuscript. Geweke J., Slonim R., and Zarkin G. Econometric solution methods for dynamic discrete choice problems (1992), University of Minnesota
61. Gihman I.I., and Skorohod A.V. The theory of stochastic processes Volumes I, II, III (1979), Springer, New York
62. Goldfarb D., and Todd M.J. Linear programming. In: Rinooy Kan A.H.G., and Todd M.J. (Eds). Handbooks in operations research Optimization Volume I (1989), North-Holland, Amsterdam
63. Gihman I.I., and Skorohod A.V. Controlled stochastic processes (1979), Springer, New York
64. Goffe W.L., Ferrier G.D., and Rogers J. Global optimization of statistical functions with simulated annealing. Computational Economics 5 2 (1992) 133-146
65. Hackbusch W. Multi-grid methods and applications (1985), Springer, New York
66. Hakansson N. Optimal investment and consumption strategies under risk for a class of utility functions. Econometrica 38 (1970) 587-607
67. Hammersley J.J., and Handscomb D.C. Monte Carlo methods (1992), Chapman & Hall, London
68. Hansen L.P., and Sargent T.J. Formulating and estimating dynamic linear rational expectations models. Journal of Economic Dynamics and Control 2 1 (1980) 7-46
69. Hansen L.P., and Sargent T.J. Linear rational expectations models for dynamically interrelated variables. In: Lucas Jr. R.E., and Sargent T.J. (Eds). Rational expectations and econometric practice (1980), Univ. of Minnesota Press, Minneapolis, MN
70. Hansen L.P., and Sargent T.J. Discounted linear exponential quadratic Gaussian control. IEEE Transactions on Automatic Control 40 5 (1995) 968-971
71. manuscript. Hansen L.P., and Sargent T.J. Recursive models of dynamic linear economies (1996), Hoover Institution
72. Hansen L.P., and Singleton K. Generalized instrumental variables estimation of nonlinear rational expectations models. Econometrica 50 (1982) 1269-1281
73. Hansen L.P., and Singleton K. Stochastic consumption, risk aversion, and the temporal behavior of asset returns. Journal of Political Economy 91 2 (1983) 249-265
74. Hawkins D., and Simon H.A. Note: Some conditions for macroeconomic stability. Econometrica 17 (1949) 245-248
75. Hinderer K. Foundations of nonstationary dynamic programming with discrete time parameter (1970), Springer, Berlin
76. Hornik K.M., Stinchcombe M., and White H. Universal approximation of an unknown function and its derivatives using multilayer feedforward networks. Neural Networks 3 (1990) 551-560
77. Hornik K., Stinchcombe M., White H., and Auer P. Degree of approximation results for feedforward networks approximating unknown mappings and their derivatives. Neural Computation 6 (1994) 1262-1274
78. Howard R. Dynamic programming and Markov processes (1960), Wiley, New York
79. Howard R. Dynamic probabilistic systems, Volume II: Semi-Markov and decision processes (1971), Wiley, New York
80. Hubner G. Improved procedures for eliminating suboptimal actions in Markov programming by the use of contraction properties. Transactions of the seventh Prague conference on information theory, statistical decision functions, random processes (1977), D. Reidel, Dordrecht 257-263
81. Hwang C. Laplace's method revisited: Weak convergence of probability measures. Annals of Probability 8 6 (1980) 1177-1182
82. Imrohoroǧlu A., and Imrohoroǧlu S. A numerical algorithm for solving models with incomplete markets. International Journal of Supercomputer Applications 7 3 (1993) 212-230
83. Johnson S.A., et al. Numerical solution of continuous-state dynamic programs using linear and spline interpolation. Operations Research 41 3 (1993) 484-500
84. Judd K. Projection methods for solving aggregate growth models. Journal of Economic Theory 58 (1993) 410-452
85. Judd K. Comments on Marcet, Rust and Pakes. In: Sims C. (Ed). Advances in econometrics sixth world congress Vol. II (1994), Cambridge Univ. Press 261-276
86. manuscript. Judd K., and Solnick A. Numerical dynamic programming with shape-preserving splines (1994), Hoover Institution
87. Chapter 12 in this Handbook. Judd K. Approximation methods and projection methods in economic analysis (1996)
88. manuscript. Judd K. Numerical methods in economics (1996), Hoover Institution
89. (Translated in Solviet Phys. Dokl., 7:595-596). Khachian L.C. A polynomial time algorithm for linear programming. Dokl. Acad. Nauk SSSR 145 2 (1979) 293-294
90. Kantorovich L.V., and Akilov G.P. Functional analysis (1982), Pergamon, Oxford
91. Keane M.P., and Wolpin K.I. The solution and estimation of discrete choice dynamic programming models by simulation: Monte Carlo evidence. Review of Economics and Statistics 76 4 (1994) 648-672
92. Klee V., and Minty G.J. How good is the simplex algorithm?. In: Shisha O. (Ed). Inequalities III (1972), Academic Press, New York 159-175
93. manuscript. Kortum S. Value function approximaton in an estimation routine (1992), Boston University
94. Translated by D. Louvish. Krasnoselskii M.A., Vainikko G.M., Zabreiko P.P., Rutitskii Ya.B., and Stetsenko V.Ya. Approximate solution of operator equations (1972), Wolters-Noordhoff, Groningen
95. Kreps D.M., and Porteus E.L. Temporal resolution of uncertainty and dynamic choice theory. Econometrica 46 1 (1978) 185-200
96. Kreps D.M., and Porteus E.L. Dynamic choice theory and dynamic programming. Econometrica 47 1 (1979) 91-100
97. Kronsjö L. Computational complexity of sequential and parallel algorithms (1985), Wiley, New York
98. Kushner H.J. Numerical methods for stochastic control problems in continuous time. SIAM Journal on Control and Optimization 28 5 (1990) 999-1048
99. Kushner H.J., and Kleinman A.J. Accelerated procedures for the solution of discrete Markov control problems. IEEE Transactions on Automatic Control 16 2 (1971) 147-152
100. Kushner H.J., and Dupuis P.G. Numerical methods for stochastic control problems in continuous time (1992), Springer, New York
101. Kydland F., and Prescott E.C. Time to build and aggregate fluctuations. Econometrica 50 (1982) 1345-1370
102. Leland H. Optimal growth in a stochastic in a stochastic environment. Review of Economic Studies 41 (1974) 75-86
103. Leonteif W. Input-output economics (1966), Oxford Univ. Press, London
104. Lettau M., and Uhlig H. Rules of thumb and dynamic programming. Discussion paper 9527 (1995), Center, Tilburg University
105. Levhari D., and Srinivasan T. Optimal savings under uncertainty. Review of Economic Studies 36 (1969) 153-163
106. Long J.B., and Plosser C. Real business cycles. Journal of Political Economy 91 1 (1983) 39-69
107. Lucas Jr. R.E. Asset prices in an exchange economy. Econometrica 46 (1978) 1426-1446
108. Lucas Jr. R.E., and Prescott E.C. Investment under uncertainty. Econometrica 39 5 (1971) 659-681
109. Luenberger D.G. Optimization by vector space methods (1969), Wiley, New York
110. Machina M.J. Choice under uncertainty: Problems solved and unsolved. Journal of Economic Perspectives 1 1 (1987) 121-154
111. Marcet A. Simulation analysis of dynamic stochastic models: Applications to theory and estimation. In: Sims C. (Ed). Advances in econometrics sixth world congress Vol. II (1994), Cambridge Univ. Press 91-118
112. Marcet A., and Marshall D.A. Solving nonlinear rational expectations models by parameterized expectations: Convergence to stationary solutions. Working paper WP-94-20 (1994), Federal Reserve Bank of Chicaog
113. McKelvey R.D., and Palfrey T.R. An experimental study of the centipede game. Econometrica 60 4 (1992) 803-836
114. Merton R.C. Lifetime portfolio selection under uncertainty: The continuous-time case. Review of Economics and Statistics 51 (1969) 247-257
115. Miranda M., and Schnitkey G. Estimation of dynamic agricultural decision models: The case of dairy cow replacement. Journal of Applied Econometrics 10 5 (1995) 41-56
116. Nelder J., and Mead R. A simplex method for function minimization. Computational Journal 7 (1965) 308-313
117. Nemirovsky A.S., and Yudin D.B. Problem complexity and method efficiency in optimization (1983), Wiley, New York
118. Niederreiter H. Random number generation and quasi-Monte Carlo methods. SIAM CBMS-NSF Vol. 63 (1992), SIAM, Philadelphia, PA
119. Novak E. Deterministic and stochastic error bounds in numerical analysis. Lecture Notes in Mathematics Vol. 1349 (1988), Springer, Berlin
120. Ortega J.M. Introduction to parallel and vector solution of linear systems (1988), Plenum, New York
121. Ortega J.M., and Rheinboldt W.C. Iterative solution on nonlinear equations in several variables (1970), Academic Press, New York
122. Ortega J.M., and Voigt R.G. Solution of partial differential equations on vector and parallel processors. SIAM Review 27 2 (1985) 149-240
123. forthcoming. Ozaki H., and Streufert P. Nonlinear dynamic programming for non-additive stochastic objectives. Journal of Mathematical Economics (1996)
124. Pakes A., and McGuire P. Computing Markov-perfect Nash equilibria: Numerical implications of a dynamic differentiated products model. Rand Journal of Economics 25 4 (1994) 555-589
125. Pan V., and Reif J. Efficient parallel solution of linear systems. Transactions of the ACM 18 2 (1985) 143-152
126. Papadimitriou C.H., and Tsitsiklis J.N. The complexity of Markov decision processes. Mathematics of Operations Research 12 3 (1987) 441-450
127. Paskov S.H. Average case complexity of multivariate integration for smooth functions. Journal of Complexity 9 (1993) 291-312
128. Paskov S.H. New methodologies for valuing securities. In: Pliska S., and Dempster M. (Eds). Mathematics of Derivative Securities (1996), Isaac Newton Institute
129. Phelps E. Accumulation of risky capital. Econometrica 30 (1962) 729-743
130. Porteus E.L. Overview of iterative methods for discounted finite Markov and semi-Markov decision chains. In: Hartley R., et al. (Ed). Recent developments in Markov decision processes (1980), Academic Press, New York
131. Powell M.J.D. Approximation theory and methods (1981), Cambridge Univ. Press
132. Press W., Flannery B.P., Teukolsky S.A., and Vetterling W.T. Numerical recipies (1992), Cambridge Univ. Press, Cambridge
133. Puterman M.L. Markov decision processes. In: Heyman D.P., and Sobel M.J. (Eds). Handbooks in operations research and management science Vol. 2 (1990), North-Holland, Amsterdam
134. Puterman M.L. Markov decision processes (1994), Wiley, New York
135. Puterman M.L., and Shin M.C. Modified policy iteration algorithms for discounted Markov decision problems. Management Science 24 (1978) 1127-1137
136. Rivlin T.J. An introduction to the approximation of functions (1969), Blaisdell, Waltham, MA
137. Rust J. Stationary equilibrium in a Market for durable assets. Econometrica 53 4 (1985) 783-806
138. Rust J. When is it optimal to kill off the market for used durable goods?. Econometrica 54 1 (1986) 65-86
139. Rust J. Optimal replacement of GMC bus engines: An empirical model of Harold Zurcher. Econometrica 55 5 (1987) 999-1033
140. Rust J. Maximum likelihood estimation of discrete control processes. SIAM Journal on Control and Optimization 26 5 (1988) 1006-1023
141. Rust J. A dynamic programming model of retirement behavior. In: Wise D. (Ed). The economics of aging (1989), Univ. of Chicago Press 359-398
142. Rust J. Comment on "optimal collection of information by partially informed agents". Econometric Reviews 7 2 (1989) 155-160
143. Rust J. Structural estimation of Markov decision processes. In: McFadden D., and Engle R. (Eds). Handbook of econometics Vol. 4 (1994), North-Holland, Amsterdam 3081-3143
144. Rust J. Estimation of dynamic structural models, problems and prospects: Discrete decision process. In: Sims C. (Ed). Advances in econometrics sixth world congress Vol. II (1994), Cambridge Univ. Press 119-170
145. manuscript. Rust J. Do people behave according to Bellman's principle of optimality? (1995), University of Wisconsin
146. forthcoming. Rust J. Using randomization to break the curse of dimensionality. Econometrica (1995)
147. manuscript. Rust J. Stochastic decision processes: Theory, computation, and estimation (1996), University of Wisconsin
148. manuscript. Rust J. Is dynamic programming inherently sequential? (1996), University of Wisconsin
149. Saad Y. Practical use of some Krylov subspace methods for solving indefinite and nonsymmetric linear systems. SIAM Journal on Scientific and Statistical Computing 5 1 (1984) 203-228
150. Saad Y. Krylov subspace methods on supercomputers. SIAM Journal on Scientific and Statistical Computing 10 6 (1989) 1200-1232
151. Saad Y., and Schultz M.H. GMRES: A generalized minimum residual algorithm for solving nonsymmetric linear systems. SIAM Journal on Scientific and Statistical Computing 7 3 (1986) 856-869
152. Samuelson P.A. Lifetime portfolio selection by dynamic stochastic programming. Review of Economics and Statistics 51 (1969) 239-246
153. manuscript. Santos M., and Vigo J. Accuracy estimates for a numerical approach to stochastic growth models (1995), ITAM
154. manuscript. Santos M., and Vigo J. Error bounds for a numerical solution for dynamic economic models (1995), ITAM, Mexico
155. Sargent T.J. Estimation of dynamic labor demand schedules under rational expectations. Journal of Political Economy 86 6 (1978) 1009-1044
156. Sargent T.J. Interpreting economic time series. Journal of Political Econmy 89 2 (1981) 213-248
157. Schumacher L.L. On shape preserving quadratic spline interpolation. SIAM Journal of Numerical Analysis 20 4 (1983) 854-864
158. Schweitzer P.J., and Seidmann A. Generalized polynomial approximations in Markovian decision processes. Journal of Mathematical Analysis and Applications 110 (1985) 568-582
159. Semmler W. Solving nonlinear dynamic models by iterative dynamic programming. Computational Economics 8 2 (1995) 127-154
160. Sikorski K. Optimal solution of nonlinear equations satisfying a Lipschitz condition. Numerische Matematik 43 (1984) 225-240
161. Sikorski K. Optimal solution of nonlinear equations. Journal of Complexity 1 (1985) 197-209
162. Simon H.A. Dynamic programming under uncertainty with a quadratic criterion function. Econometrica 24 (1956) 74-81
163. Sims, C.A. 'Solving the stochastic growth model by backsolving with a particular nonlinear form for the decision rule', Journal of Business and Economic Statistics, 8(1):45-47.
164. Smith Jr. A.A. Solving stochastic dynamic programming problems using rules of thumb. Discussion Paper 816, Institute for Economic Research (1991), Queen's University, Mexico
165. Solis F.J., and Wets R.J. Minimization by random search techniques. Mathematics of Operations Research 6 1 (1981) 19-30
166. Stokey N.L., Lucas Jr. R.E., and Prescott E.C. Recursive methods in economic dynamics (1989), Harvard Univ. Press, Cambridge, MA
167. Tapiero C.S., and Sulem A. Computational aspects in applied stochastic control. Computational Economics 7 2 (1994) 109-146
168. Tauchen G. Solving the stochastic growth model by using quadrature methods and value function iterations. Journal of Business & Economic Statistics 8 1 (1990) 49-51
169. Tauchen G., and Hussey R. Quadrature-based methods for obtaining approximate solutions to nonlinear asset pricing models. Econometrica 59 2 (1991) 371-396
170. Traub J.F. Breaking intractability. Scientific American 270 1 (1994) 102-107
171. Traub J.F., Wasilkowski G.W., and Wozniakowski H. Information-based complexity (1988), Academic Press, New York
172. Traub J.F., and Wozniakowski H. A general theory of optimal algorithms (1980), Academic Press, New York
173. Traub J.F., and Wozniakowski H. On the optimal solution of large linear systems. Journal of the Association for Computing Machinery 31 3 (1984) 545-559
174. Traub J.F., and Wozniakowski H. Information-based complexity: New questions for mathematicians. The Mathematical Intelligencer 13 2 (1991) 34-43
175. Traub J.F., and Wozniakowski H. The Monte Carlo algorithm with a pseudorandom generator. Mathematics of Computation 58 197 (1992) 323-339
176. manuscript. Trick M.A., and Zin S.E. A linear programming approach to solving stochastic dynamic programs (1993), Carnegie-Mellon University
177. Tsitsiklis J.N. Asynchronous stochastic approximation and Q-learning. Machine Learning 16 (1994) 185-202
178. manuscript. Tsitsiklis J.N. Complexity-theoretic aspects of problems in control theory (1994), Center for Intelligent Control Systems, MIT Press
179. van Dijk N.M. Controlled Markov processes: Time discretization. CWI Tract II (1984), Mathematische Centrum, Amsterdam
180. Wasilkowski G.W. On the average complexity of global optimization problems. Mathematical Programming 57 (1992) 313-324
181. Werschulz A.G. The computational complexity of differential and integral equations (1991), Oxford Univ. Press, New York
182. Wheeler Jr. R.M., and Narendra K.S. Decentralized learning in finite Markov chains. IEEE Transactions on Automatic Control AC-31 (1986) 519-526
183. Whitt W. Approximations of dynamic programs I. Mathematics of Operations Research 3 (1978) 231-243
184. Whittle P. Optimization over time: Dynamic programming and stochastic control Vol. I and II (1982), Wiley, New York
185. Wozniakowski H. Average case complexity of multivariate integration. Bulletin of the American Mathematical Society 24 (1991) 185-194
186. Wozniakowski H. Average case complexity of linear multivariate problems. Journal of Complexity 8 (1992) 372-392