Chapter 51 Structural estimation of markov decision processes

Handbook of Econometrics

Volumn 4, Issue , 1994, Pages 3081-3143

  Rust, John ^a  

^a UNIVERSITY OF WISCONSIN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 70350107178     PISSN: 15734412     EISSN: None     Source Type: Book Series    
DOI: 10.1016/S1573-4412(05)80020-0     Document Type: Review

References (122)

1. manuscript. Ahn M.Y. Duration Dependence, Search Effort, and Labor Market Outcomes: A Structural Model of Labor Market History (1993), Duke University
2. manuscript. Ahn M.Y. Econometric Analysis of Sequential Discrete Choice Behavior with Duration Dependence (1993), Duke University
3. Ahn H., and Manski C. Distribution Theory for the Analysis of Binary Choice Under Uncertainty with Nonparametric Estimation of Expectations. Journal of Econometrics 56 3 (1993) 270-291
4. Amemiya T. On A Two-Step Estimation of a Multivariate Logit Model. Journal of Econometrics 8 (1976) 13-21
5. Andrews D.W.K. Chi-Square Diagnostic Tests for Econometric Models. Journal of Econometrics 37 (1988) 135-156
6. Andrews D.W.K. Chi-Square Tests for Econometric Models: Theory. Econometrica 56 (1988) 1414-1453
7. Arrow K.J., Harris T., and Marschak J. Optimal Inventory Policy. Econometrica 19 3 (1951) 250-272
8. forthcoming. Bansal R., Gallant A.R., Hussey R., and Tauchen G. Nonparametric Estimation of Structural Models for High Frequency Currency Market Data. Journal of Econometrics (1993)
9. Basawa I.V., and Prakasa Rao B.L.S. Statistical Inference for Stochastic Processes (1980), Academic Press, New York
10. Bellman R. Dynamic Programming (1957), Princeton University Press, Princeton
11. Berkovec J., and Stern S. Job Exit Behavior of Older Men. Econometrica 59 1 (1991) 189-210
12. Berndt E., Hall B., Hall R., and Hausman J. Estimation and Inference in Nonlinear Structural Models. Annals of Economic and Social Measurement 3 (1974) 653-665
13. Bertsekas D. Dynamic Programming Deterministic and Stochastic Models (1987), Prentice Hall, Princeton
14. Bertsekas D., and Castañon D. Adaptive Aggregation Methods for Infinite Horizon Dynamic Programming. IEEE Transactions on Automatic Control 34 6 (1989) 589-598
15. Bhattacharya R.N., and Majumdar M. Controlled Semi-Markov Models - The Discounted Case. Journal of Statistical Planning and Inference 21 (1989) 365-381
16. Billingsley P. Statistical Inference for Markov Processes (1961), University of Chicago Press, New York
17. Blackwell D. Discrete Dynamic Programming. Annals of Mathematical Statistics 33 (1962) 719-726
18. Blackwell D. Discounted Dynamic Programming. Annals of Mathematical Statistics 36 (1965) 226-235
19. Blackwell D. Positive Dynamic Programming. Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability 1 (1967) 415-418
20. Block H., and Marschak J. Random Orderings and Stochastic Theories of Response. In: Olkin I. (Ed). Contributions to Probability and Statistics (1960), Stanford University Press, Chicago
21. Boldrin M., and Montrucchio L. On the Indeterminacy of Capital Accumulation Paths. Journal of Economic Theory 40 1 (1986) 26-39
22. Brock W.A. Asset Prices in a Production Economy. In: McCall J.J. (Ed). The Economics of Information and Uncertainty (1982), University of Chicago Press, Chicago
23. Brock W.A., and Mirman L.J. Optimal Economic Growth Under Uncertainty: The Discounted Case. Journal of Economic Theory 4 (1972) 479-513
24. Chamberlain G. Panel Data. In: Griliches Z., and Intrilligator M.D. (Eds). Handbook of Econometrics Volume 2 (1984), North-Holland, Amsterdam 1247-1318
25. Chew S.H., and Epstein L.G. The Structure of Preferences and Attitudes Towards the Timing and Resolution of Uncertainty. International Economic Review 30 1 (1989) 103-118
26. Christensen B.J., and Kiefer N.M. The Exact Likelihood Function for an Empirical Job Search Model. Econometric Theory (1991)
27. manuscript. Christensen B.J., and Kiefer N.M. Measurement Error in the Prototypical Job Search Model (1991), Cornell University, Amsterdam
28. Cox D.R. Partial Likelihood. Biometrika 62 2 (1975) 269-276
29. Cox D.R., and Hinkley D.V. Theoretical Statistics (1974), Chapman and Hall, London
30. Dagsvik J. Discrete Dynamic Choice: An Extension of the Choice Models of Luce and Thurstone. Journal of Mathematical Psychology 27 (1983) 1-43
31. manuscript. Dagsvik J. A Note on the Theoretical Content of the GEU Model for Discrete Choice (1991), Central Bureau of Statistics, London
32. Teakfield, Hampshire. Daly A.J., and Zachary S. Improved Multiple Choice Models. In: Hensher D.A., and Dalvi Q. (Eds). Determinants of Travel Choice (1978) 335-357
33. Das M. A Micro Econometric Model of Capital Utilization and Retirement: The Case of the Cement Industry. Review of Economic Studies 59 2 (1992) 287-298
34. Daula T., and Moffitt R. Estimating a Dynamic Programming Model of Army Reenlistment Behavior. Military Compensation and Personnel Retention (1991)
35. Debreu G. Review of R.D. Luce Individual Choice Behavior. American Economic Review 50 (1960) 186-188
36. Denardo E.V. Contraction Mappings Underlying the Theory of Dynamic Programming. SIAM Review 9 (1967) 165-177
37. Dongarra J.J., and Hewitt T. Implementing Dense Linear Algebra Algorithms Using Multitasking on the Cray X-MP-4. SIAM Journal on Scientific and Statistical Computing 7 1 (1986) 347-350
38. Eckstein Z., and Wolpin K. The Specification and Estimation of Dynamic Stochastic Discrete Choice Models. Journal of Human Resources 24 4 (1989) 562-598
39. Eckstein Z., and Wolpin K. Dynamic Labour Force Participation of Married Women and Endogenous Work Experience. Review of Economic Studies 56 (1989) 375-390
40. Eckstein Z., and Wolpin K. Estimating a Market Equilibrium Search Model from Panel Data on Individuals. Econometrica 58 4 (1990) 783-808
41. Engle R. Wald, Likelihood Ratio and Lagrange Multiplier Tests in Econometrics. In: Griliches Z., and Intrilligator M.D. (Eds). Handbook of Econometrics Vol 2 (1984), North-Holland, Oslo
42. Epstein L.G., and Zin S.E. Substitution, Risk Aversion, and the Temporal Behavior of Consumption and Asset Returns: A Theoretical Framework. Econometrica 57 4 (1989) 937-970
43. Flinn C., and Heckman J.J. New Methods for Analyzing Laborforce Dynamics. Journal of Econometrics 18 (1982) 115-168
44. Gihman I.I., and Skorohod A.V. Controlled Stochastic Processes (1979), Springer-Verlag New York, Amsterdam
45. Gotz G.A., and McCall J.J. Estimation in Sequential Decisionmaking Models: A Methodological Note. Economics Letters 6 (1980) 131-136
46. Gotz G.A., and McCall J.J. A Dynamic Retention Model for Air Force Officers. Report R-3028-AF (1984), The RAND Corporation, Santa Monica, California
47. Haavelmö T. The Probability Approach in Econometrics. Econometrica Supplement 12 (1944) 1-115
48. Hakansson N. Optimal Investment and Consumption Strategies Under Risk for a Class of Utility Functions. Econometrica 38 (1970) 587-607
49. Hansen L.P. Large Sample Properties of Method of Moments Estimators. Econometrica 50 (1982) 1029-1054
50. Hansen L.P. In: Engle R., and McFadden D. (Eds). Handbook of Econometrics Vol. 4 (1994), North-Holland, Santa Monica, California
51. Hansen L.P., and Sargent T.J. Formulating and Estimating Dynamic Linear Rational Expectations Models. Journal of Economic Dynamics and Control 2 1 (1980) 7-46
52. Hansen L.P., and Sargent T.J. Linear Rational Expectations Models for Dynamically Interrelated Variables. In: Lucas Jr. R.E., and Sargent T.J. (Eds). Rational Expectations and Econometric Practice (1980), University of Minnesota Press, Minneapolis
53. manuscript. Hansen L.P., and Sargent T.J. Recursive Models of Dynamic Economics (1993), Hoover Institution
54. Hansen L.P., and Singleton K. Generalized Instrumental Variables Estimation of Nonlinear Rational Expectations Models. Econometrica 50 (1982) 1269-1281
55. Hansen L.P., and Singleton K. Stochastic Consumption, Risk Aversion, and the Temporal Behavior of Asset Returns. Journal of Political Economy 91 2 (1983) 249-265
56. Heckman J.J. Statistical Models of Discrete Panel Data. In: Manski C.F., and McFadden D. (Eds). Structural Analysis of Discrete Data (1981), MIT Press, Cambridge, Massachusetts
57. Heckman J.J. The Incidental Parameters Problem and the Problem of Initial Conditions in Estimating Discrete-Time, Discrete-Data Stochastic Processes. In: Manski C.F., and McFadden D. (Eds). Structural Analysis of Discrete Data (1981), MIT Press, Cambridge, Massachusetts
58. Heckman J.J. Randomization and Social Policy Evaluation. NBER Technical Working paper 107 (1991)
59. Heckman J.J. Alternative Approaches to the Evaluation of Social Programs: Econometric and Experimental Methods. In: Laffont J., and Sims C. (Eds). Advances in Econometrics: Sixth World Congress. Econometric Society Monographs (1994), Cambridge University Press, Cambridge, Massachusetts
60. forthcoming. Hotz V.J., and Miller R.A. Conditional Choice Probabilities and the Estimation of Dynamic Models. Review of Economic Studies (1993)
61. Hotz V.J., Miller R.A., Sanders S., and Smith J. A Simulation Estimator for Dynamic Models of Discrete Choice. Review of Economic Studies 60 (1993) 397-429
62. Howard R. Dynamic Programming and Markov Processes (1960), J. Wiley, New York
63. Howard R. Dynamic Probabilistic Systems: Volume 2 - Semi-Markov and Decision Processes (1971), J. Wiley, New York
64. Hurwicz L., and Uzawa H. On the Integrability of Demand Functions. In: Chipman J., et al. (Ed). Preferences, Utility, and Demand (1971), Harcourt, Brace and Jovanovich, New York
65. manuscript. Judd K. Numerical Methods in Economics (1994), Hoover Institution
66. manuscript. Keane M., and Wolpin K. The Solution and Estimation of Discrete Choice Dynamic Programming Models by Simulation: Monte Carlo Evidence (1994), University of Minnesota
67. forthcoming. Kennet M. A Structural Model of Aircraft Engine Maintenance. Journal of Applied Econometrics (1994)
68. Kreps D., and Porteus E. Temporal Resolution of Uncertainty and Dynamic Choice Theory. Econometrica 46 (1978) 185-200
69. Kushner H.J. Numerical Methods for Stochastic Control Problems in Continuous Time. SIAM Journal on Control and Optimization 28 5 (1990) 999-1048
70. Kydland F., and Prescott E.C. Time to Build and Aggregate Fluctuations. Econometrica 50 (1982) 1345-1370
71. Lancaster T. The Econometric Analysis of Transition Data (1990), Cambridge University Press
72. Leland H. Optimal Growth in a Stochastic Environment. Review of Economic Studies 41 (1974) 75-86
73. Levhari D., and Srinivasan T. Optimal Savings Under Uncertainty. Review of Economic Studies 36 (1969) 153-163
74. Long J.B., and Plosser C. Real Business Cycles. Journal of Political Economy 91 1 (1983) 39-69
75. Carnegie-Rochester Conference on Public Policy. Lucas Jr. R.E. Econometric Policy Evaluation: A Critique. In: Brunner K., and Meltzer A.K. (Eds). The Phillips Curve and Labour Markets (1976), North-Holland, Amsterdam
76. Lucas Jr. R.E. Asset Prices in an Exchange Economy. Econometrica 46 (1978) 1426-1446
77. Lucas Jr. R.E., and Prescott C.E. Investment Under Uncertainty. Econometrica 39 5 (1971) 659-681
78. Lumsdaine R., Stock J., and Wise D. Three Models of Retirement: Computational Complexity vs. Predictive Validity. In: Wise D. (Ed). Topics in the Economics of Aging (1992), University of Chicago Press, Chicago
79. Machina M.J. Expected Utility without the Independence Axiom. Econometrica 50-52 (1982) 277-324
80. Machina M.J. Choice Under Uncertainty: Problems Solved and Unsolved. Journal of Economic Perspectives 1 1 (1987) 121-154
81. Mantel R. On the Characterization of Excess Demand. Journal of Economic Theory 7 (1974) 348-353
82. Marcet A. Simulation Analysis of Stochastic Dynamic Models: Applications to Theory and Estimation. In: Sims C., and Laffont J. (Eds). Advances in Econometrics: Proceedings of the 1990 Meetings of the Econometric Society (1994)
83. Marschak T. Economic Measurements for Policy and Prediction. In: Hood W.C., and Koopmans T.J. (Eds). Studies in Econometric Method (1953), Wiley, New York
84. McFadden D. Conditional Logit Analysis of Qualitative Choice Behavior. In: Zarembka P. (Ed). Frontiers of Econometrics (1973), Academic Press, New York
85. McFadden D. Econometric Models of Probabilistic Choice. In: Manski C.F., and McFadden D. (Eds). Structural Analysis of Discrete Data (1981), MIT Press, New York
86. McFadden D. Econometric Analysis of Qualitative Response Models. In: Griliches Z., and Intriligator M.D. (Eds). Handbook of Econometrics Vol. 2 (1984), North-Holland, Cambridge, Massachusetts 1395-1457
87. McFadden D. A Method of Simulated Moments for Estimation of Discrete Response Models without Numerical Integration. Econometrica 57 5 (1989) 995-1026
88. Merton R.C. Lifetime Portfolio Selection Under Uncertainty: The Continuous-time Case. Review of Economics and Statistics 51 (1969) 247-257
89. Miller R. Job Matching and Occupational Choice. Journal of Political Economy 92 6 (1984) 1086-1120
90. Pakes A. Patents as Options: Some Estimates of the Value of Holding European Patent Stocks. Econometrica 54 (1986) 755-785
91. Pakes A. Estimation of Dynamic Structural Models: Problems and Prospects Part II: Mixed Continuous-Discrete Models and Market Interactions. In: Sims C., and Laffont J.J. (Eds). Proceedings of the 6th World Congress of the Econometric Society. Barcelona, Spain (1994), Cambridge University Press, Amsterdam
92. Pakes A., and Pollard D. Simulation and the Asymptotics of Optimization Estimators. Econometrica 57 5 (1989) 1027-1057
93. Phelps E. Accumulation of Risky Capital. Econometrica 30 (1962) 729-743
94. Pollard D. General Chi-Square Goodness of Fit Tests with Data-Dependent Cells. Z. Wahrscheinlichkeitstheorie verw. Gebeite 50 (1979) 317-331
95. Polland D. Convergence of Stochastic Processes (1984), Springer Verlag
96. Puterman M.L. Markov Decision Processes. In: Heyman D.P., and Sobel M.J. (Eds). Handbooks in Operations Research and Management Science Volume 2 (1990), North-Holland/Elsevier, Amsterdam
97. Rust J. Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher. Econometrica 55 5 (1987) 999-1033
98. Rust J. Statistical Models of Discrete Choice Processes. Transportation Research 22B 2 (1988) 125-158
99. Rust J. Maximum Likelihood Estimation of Discrete Control Processes. SIAM Journal on Control and Optimization 26 5 (1988) 1006-1023
100. Rust J. A Dynamic Programming Model of Retirement Behavior. In: Wise D. (Ed). The Economics of Aging (1989), University of Chicago Press, Amsterdam 359-398
101. Rust J. Do People Behave According to Bellman's Principal of Optimality?. Hoover Institution Working Paper, E-92-10 (1992)
102. manuscript. Rust J. How Social Security and Medicare Affect Retirement Behavior in a World of Incomplete Markets (1993), University of Wisconsin, Chicago
103. Rust J. Estimation of Dynamic Structural Models: Problems and Prospects Part I: Discrete Decision Processes. In: Sims C., and Laffont J.J. (Eds). Proceedings of the 6th World Congress of the Econometric Society. Barcelona, Spain (1994), Cambridge University Press
104. forthcoming. Rust J. Numerical Dynamic Programming in Economics. In: Amman H., Kendrick D., and Rust J. (Eds). Handbook of Computational Economics (1995), North-Holland
105. manuscript. Rust J. Using Randomization to Break the Curse of Dimensionality (1995), University of Wisconsin
106. manuscript. Rust J. Stochastic Decision Processes: Theory, Computation, and Estimation (1996), University of Wisconsin
107. Samuelson P.A. Lifetime Portfolio Selection by Dynamic Stochastic Programming. Review of Economics and Statistics 51 (1969) 239-246
108. Sargent T.J. Estimation of Dynamic Labor Demand Schedules Under Rational Expectations. Journal of Political Economy 86 6 (1978) 1009-1044
109. Sargent T.J. Interpreting Economic Time Series. Journal of Political Economy 89 2 (1981) 213-248
110. Smith Jr. A.A. Solving Stochastic Dynamic Programming Problems Using Rules of Thumb. Discussion Paper 818 (1991), Institute for Economic Research, Queen's University, Ontario, Canada
111. Sonnenschein H. Do Walras Law and Continuity Characterize the Class of Community Excess Demand Functions?. Journal of Economic Theory 6 (1973) 345-354
112. Stern S. A Method for Smoothing Simulated Moments of Discrete Probabilities in Multinomial Probit Models. Econometrica 60 4 (1992) 943-952
113. forthcoming. Stern S. Approximate Solutions to Stochastic Dynamic Programming Problems. Econometric Theory (1995)
114. Stock J., and Wise D. Pensions, The Option Value of Work and Retirement. Econometrica 58 5 (1990) 1151-1180
115. Stokey N.L., Lucas Jr. R.E., and Prescott E.C. Recursive Methods in Economic Dynamics (1989), Harvard University Press, Ontario, Canada
116. Theil H. Principles of Econometrics (1971), Wiley, Cambridge, Massachusetts
117. van Dijk N.M. Controlled Markov Processes: Time Discretization. CWI Tract 11 (1984), Center for Mathematics and Computer Science, New York
118. Verim H. The Nonparametric Approach to Demand Analysis. Econometrica 52 3 (1982) 945-972
119. White H. Maximum Likelihood Estimation of Misspecified Models. Econometrica 50 (1982) 1-26
120. Williams H.C. On the Formation of Travel Demand Models and Economic Evaluation of User Benefit. Environment and Planning A-9 (1977) 285-344
121. Wolpin K. An Estimable Dynamic Stochastic Model of Fertility and Child Mortality. Journal of Political Economy 92 5 (1984) 852-874
122. Wolpin K. Estimating a Structural Search Model: The Transition from Schooling to Work. Econometrica 55 (1987) 801-818