Regret bounds for the adaptive control of Linear Quadratic systems

Journal of Machine Learning Research

Volumn 19, Issue , 2011, Pages 1-26

  Abbasi Yadkori, Yasin ^a   Szepesvaŕi, Csaba ^a  

^a UNIVERSITY OF ALBERTA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS;

ADAPTIVE CONTROL; CONTROL COMMUNITY; LEAST-SQUARES ESTIMATION; LINEAR QUADRATIC; LINEAR QUADRATIC CONTROL PROBLEM; LQ CONTROL PROBLEM; MODEL PARAMETERS; REGRET BOUNDS;

ADAPTIVE CONTROL SYSTEMS;

EID: 84867882483     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (33)

1. Y. Abbasi-Yadkori, D. Pal, and Cs. Szepesvári. Online least squares estimation with self-normalized processes: An application to bandit problems. Arxiv preprint http://arxiv.org/abs/1102.2670, 2011.
2. B. D. O. Anderson and J. B. Moore. Linear Optimal Control. Prentice-Hall, 1971.
3. P. Auer. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research, 3:397-422, 2003. ISSN 1533-7928.
4. P. Auer, N. Cesa-Bianchi, and P. Fischer. Finite time analysis of the multiarmed bandit problem. Machine Learning, 47(2-3):235-256, 2002. (Pubitemid 34126111)
5. P. Auer, R. Ortner, and Cs. Szepesvári. Improved rates for the stochastic continuumarmed bandit problem. In Proceedings of the 20th Annual Conference on Learning Theory (COLT-07), pages 454-468, 2007.
6. P. Auer, T. Jaksch, and R. Ortner. Near-optimal regret bounds for reinforcement learning. Journal of Machine Learning Research, 11:1563-1600, 2010.
7. P. L. Bartlett and A. Tewari. REGAL: A regularization based algorithm for reinforcement learning in weakly communicating MDPs. In Proceedings of the 25th Annual Conference on Uncertainty in Artificial Intelligence, 2009.
8. A. Becker, P. R. Kumar, and C. Z. Wei. Adaptive control with the stochastic approximation algorithm: Geometry and convergence. IEEE Trans, on Automatic Control, 30(4) :330-338, 1985.
9. D. Bertsekas. Dynamic Programming. Prentice-Hall, 1987.
10. D. P. Bertsekas. Dynamic Programming and Optimal Control. Athena Scientific, 2nd edition, 2001.
11. S. Bittanti and M. C. Campi. Adaptive control of linear time invariant systems: the "bet on the best" principle. Communications in Information and Systems, 6(4):299-320, 2006.
12. R. I. Brafman and M. Tennenholtz. R-MAX - a general polynomial time algorithm for near-optimal reinforcement learning. Journal of Machine Learning Research, 3:213-231, 2002.
13. M. C. Campi and P. R. Kumar. Adaptive linear quadratic Gaussian control: the cost-biased approach revisited. SIAM Journal on Control and Optimization, 36(6):1890-1907, 1998.
14. H. Chen and L. Guo. Optimal adaptive control and consistent parameter estimates for armax model with quadratic cost. SIAM Journal on Control and Optimization, 25(4): 845-867, 1987. (Pubitemid 17599082)
15. H. Chen and J. Zhang. Identification and adaptive control for systems with unknown orders, delay, and coefficients. Automatic Control, IEEE Transactions on, 35(8):866-877, August 1990. (Pubitemid 20738736)
16. V. Dani and T. P. Hayes. Robbing the bandit: Less regret in online geometric optimization against an adaptive adversary. In 16th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 937-943, 2006.
17. V. Dani, T. P. Hayes, and S. M. Kakade. Stochastic linear optimization under bandit feedback. COLT-2008, pages 355-366, 2008.
18. C. Fiechter. Pac adaptive control of linear systems. In in Proceedings of the 10th Annual Conference on Computational Learning Theory, ACM, pages 72-80. Press, 1997.
19. S. M. Kakade. On the sample complexity of reinforcement learning. PhD thesis, Gatsby Computational Neuroscience Unit, University College London, 2003.
20. S. M. Kakade, M. J. Kearns, and J. Langford. Exploration in metric state spaces. In T. Fawcett and N. Mishra, editors, ICML 2003, pages 306-312. AAAI Press, 2003.
21. M. Kearns and S. P. Singh. Near-optimal performance for reinforcement learning in polynomial time. In J. W. Shavlik, editor, ICML 1998, pages 260-268. Morgan Kauffmann, 1998.
22. R. Kleinberg. Nearly tight bounds for the continuum-armed bandit problem. In Advances in Neural Information Processing Systems, pages 697-704, 2005.
23. R. Kleinberg, A. Slivkins, and E. Upfal. Multi-armed bandits in metric spaces. In Proceedings of the 40th annual ACM symposium on Theory of computing, pages 681-690, 2008.
24. T. L. Lai and H. Robbins. Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6:4-22, 1985.
25. T. L. Lai and C. Z. Wei. Least squares estimates in stochastic regression models with applications to identification and control of dynamic systems. The Annals of Statistics, 10(1):pp. 154-166, 1982a.
26. T. L. Lai and C. Z. Wei. Least squares estimates in stochastic regression models with applications to identification and control of dynamic systems. The Annals of Statistics, 10(1) : 154-166, 1982b.
27. T. L. Lai and C. Z. Wei. Asymptotically efficient self-tuning regulators. SIAM Journal on Control and Optimization, 25:466-481, March 1987.
28. T. L. Lai and Z. Ying. Efficient recursive estimation and adaptive control in stochastic regression and armax models. Statistica Sinica, 16:741-772, 2006. (Pubitemid 44744348)
29. P. Rusmevichientong and J. N. Tsitsiklis. Linearly parameterized bandits. Mathematics of Operations Research, 35(2):395-411, 2010.
30. H. A. Simon, dynamic programming under uncertainty with a quadratic criterion function. Econometrica, 24(1):741, 1956.
31. A. L. Strehl and M. L. Littman. Online linear regression and its application to model-based reinforcement learning. In NIPS, pages 1417-1424, 2008.
32. A. L. Strehl, L. Li, E. Wiewiora, J. Langford, and M. L. Littman. PAC model-free reinforcement learning. In ICML, pages 881-888, 2006.
33. I. Szita and Cs. Szepesvári. Model-based reinforcement learning with nearly tight exploration complexity bounds. In ICML 2010, pages 1031-1038, 2010.