Q-learning algorithms for optimal stopping based on least squares

2007 European Control Conference, ECC 2007

Volumn , Issue , 2007, Pages 2368-2375

  Yu, Huizhen ^a   Bertsekas, Dimitri P ^b  

^a UNIVERSITY OF HELSINKI   (Finland)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION ALGORITHMS; APPROXIMATION THEORY; ITERATIVE METHODS; REINFORCEMENT LEARNING; STOCHASTIC CONTROL SYSTEMS; STOCHASTIC SYSTEMS;

ALTERNATIVE ALGORITHMS; LINEAR FUNCTIONS; OPTIMAL STOPPING; OPTIMAL STOPPING PROBLEM; Q-LEARNING ALGORITHMS; STOCHASTIC APPROXIMATIONS; TEMPORAL DIFFERENCES; VALUE ITERATION;

LEARNING ALGORITHMS;

EID: 84927748655     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.23919/ecc.2007.7068523     Document Type: Conference Paper

References (18)

1. J. N. Tsitsiklis and B. Van Roy, "Optimal stopping of Markov processes: Hilbert space theory, approximation algorithms, and an application to pricing financial derivatives," IEEE Trans. Automat. Contr., vol. 44, pp. 1840-1851, 1999.
2. J. Barraquand and D. Martineau, "Numerical valuation of high dimensional multivariate American securities," Journal of Financial and Quantitative Analysis, vol. 30, pp. 383-405, 1995.
3. F. A. Longstaff and E. S. Schwartz, "Valuing American options by simulation: A simple least-squares approach," Review of Financial Studies, vol. 14, pp. 113-147, 2001.
4. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
5. D. P. Bertsekas, Dynamic Programming and Optimal Control, Vol. II, 3rd ed. Belmont, MA: Athena Scientific, 2007.
6. R. S. Sutton, "Learning to predict by the methods of temporal differences," Machine Learning, vol. 3, pp. 9-44, 1988.
7. D. P. Bertsekas and S. Ioffe, "Temporal differences-based policy iteration and applications in neuro-dynamic programming," MIT, LIDS Tech. Report LIDS-P-2349, 1996.
8. A. Nedíc and D. P. Bertsekas, "Least squares policy evaluation algorithms with linear function approximation," Discrete Event Dyn. Syst., vol. 13, pp. 79-110, 2003.
9. H. Yu and D. P. Bertsekas, "Convergence results for some temporal difference methods based on least squares," MIT, LIDS Tech. Report 2697, 2006.
10. -, "A least squares Q-learning algorithm for optimal stopping problems," MIT, LIDS Tech. Report 2731, 2006.
11. S. J. Bradtke and A. G. Barto, "Linear least-squares algorithms for temporal difference learning," Machine Learning, vol. 22, no. 2, pp. 33-57, 1996.
12. J. A. Boyan, "Least-squares temporal difference learning," in Proc. The 16th Int. Conf. Machine Learning, 1999.
13. D. S. Choi and B. Van Roy, "A generalized Kalman filter for fixed point approximation and efficient temporal-difference learning," Discrete Event Dyn. Syst., vol. 16, no. 2, pp. 207-239, 2006.
14. D. P. Bertsekas, V. S. Borkar, and A. Nedíc, "Improved temporal difference methods with linear function approximation," MIT, LIDS Tech. Report 2573, 2003, also appears in "Learning and Approximate Dynamic Programming," by A. Barto, W. Powell, J. Si, (Eds.), IEEE Press, 2004.
15. C. J. C. H. Watkins, "Learning from delayed rewards," Doctoral dissertation, University of Cambridge, Cambridge, United Kingdom, 1989.
16. C. J. C. H. Watkins and P. Dayan, "Q-learning," Machine Learning, vol. 8, pp. 279-292, 1992.
17. J. N. Tsitsiklis, "Asynchronous stochastic approximation and Qlearning," Machine Learning, vol. 16, pp. 185-202, 1994.
18. J. N. Tsitsiklis and B. Van Roy, "An analysis of temporal-difference learning with function approximation," IEEE Trans. Automat. Contr., vol. 42, no. 5, pp. 674-690, 1997.