Approximate policy optimization and adaptive control in regression models

Computational Economics

Volumn 27, Issue 4, 2006, Pages 433-452

  Han, Jiarui ^a   Lai, Tze Leung ^b   Spivakovsky, Viktor ^c  

^a Federal Service for Surveillance on Consumer Rights Protection and Human Well Being   (United States)

^b STANFORD UNIVERSITY   (United States)

^c Citadel Investment Group   (United States)

Author keywords

Dynamic programming; Learning by doing; Monte Carlo; Policy iteration; Rollout

Indexed keywords

EID: 33745450286     PISSN: 09277099     EISSN: 15729974     Source Type: Journal    
DOI: 10.1007/s10614-005-9007-1     Document Type: Article

References (27)

1. Aghion, P., Bolton, P. and Harris, C. (1991). Optimal learning byexperimentation. Review of Economic Studies 58, 621-654.
2. Anderson, T.W. and Taylor, J. (1976). Some experimental resultson the statistical properties of least squares estimates in controlproblems. Econometrica 44, 1289-1302.
3. Åstrom, K.J. (1983). Theory and applications of adaptive control - a survey. Automatica 19, 471-486.
4. Bellman, R. (1957). Dynamic Programming. Princeton, NJ:PRinceton University Press.
5. Bertsekas, D.P. (2000). Dynamic Programming and Optimal Control, 2nd edition. Belmont, MA: Athena Scientific.
6. Bertsekas, D.P. and Tsitsiklis, J.N. (1996). Neuro-Dynamic Programming. Belmont, MA: Athena Scientific.
7. Bertsekas, D.P. Tsitsiklis, J.N. and Wu, C. (1997). Rollout algorithms for combinatorial optimization. Journal ofHeuristics 3, 245-262.
8. Blume, L. and Easley, D. (1984). Rational expectationsequilibrium: An alternative approach. Journal of EconomicTheory 34, 116-129.
9. Box, G.E.P. and Tiao, G.C. (1973). Bayesian Inference in Statistical Analysis. New York: Wiley.
10. Easley, D. and Kiefer, N.M. (1989). Optimal learning withendogenous data. International Economic Review 30, 963-978.
11. Kendrick, D. (1981). Stochastic Control for Economic Models. New York: McGraw-Hill.
12. Kiefer, N.M. (1989). A value function arising in the economics ofinformation. Journal of Economic Dynamics and Control 13, 201-223.
13. Lai, T.L. and Robbins, H. (1979). Adaptive design and stochastic approximation. Annals of Statistics 7, 1196-1221.
14. Lai, T.L. and Robbins, H. (1982). Iterated least squares inmultiperiod control. Advances in Applied Mathematics 3, 50-73.
15. Lai, T.L. and Wong, S.P. (2004). Valuation of American options via basis functions. IEEE Transactions onAutomatic Control 49 374-385.
16. Longstaff, F.A. and Schwartz, E.S. (2001). Valuing American options by simulation: A simple least-squaresapproach. Review of Financial Studies 14, 113-147.
17. MacGovern, A., Moss, E. and Barto, A. (2002). Building a basic block instruction scheduler using reinforcement learning and rollouts. Machine Learning 49, 141-160.
18. Prescott, E. (1972). The multi-period control problem under uncertainty. Econometrica 40, 1043-1058.
19. Rust, J. (1997). Using randomization to break the curse of dimensionality. Econometrica 65, 487-516.
20. Secomandi, N. (2001). A rollout policy for the vehicle routing problem with stochastic demands. Operations Research, 49, 796-802.
21. Secomandi, N. (2003). Analysis of a rollout approach to sequencing problems with stochastic routing applications. Journal of Heuristics 9, 321-352.
22. Stokey, N. and Lucas, R.E. (1989). Recursive Methods in Economic Dynamics. Cambridge, MA: HarvardUniversity Press.
23. Tesauro, G. and Galperin, G. (1996). On-line policy improvement using Monte-Carlo search. In Advances inNeural Information Processing Systems 9, 1068-1074. Cambridge, MA: MIT Press.
24. Tsitsiklis, J.N. and Van Roy, B. (2001). Regression methods for pricing complex American-style options. IEEE Transactions on Neural Networks 12, 694-703.
25. Wieland, V. (1995). Optimal control with unknown parameters - a study of optimal learning strategies with anapplication to monetary policy. Ph.D. Thesis, Stanford University.
26. Wieland, V. (2000). Learning by doing and the value of optimal experimentation. Journal of Economic Dynamics and Control 24, 501-534.
27. Yan, X., Diaconis, P., Rusmevichientong, P. and Van Roy, B. (2005). Solitaire: Man versus machine. In Advances in Neural Information Processing Systems 17, in press. Cambridge, MA: MIT Press