On the possibility of learning in reactive environments with arbitrary dependence

Theoretical Computer Science

Volumn 405, Issue 3, 2008, Pages 274-284

  Ryabko, Daniil ^a,b   Hutter, Marcus ^c  

^a DALLE MOLLE INSTITUTE FOR ARTIFICIAL INTELLIGENCE IDSIA   (Switzerland)

^b INRIA   (France)

^c AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

(non) Markov decision processes; Asymptotic average value; Reinforcement learning; Self optimizing policies

Indexed keywords

MARKOV PROCESSES; STOCHASTIC SYSTEMS;

AVERAGE VALUES; MARKOV DECISION PROCESSES; MIXING CONDITIONS; PROBABILISTIC ASSUMPTIONS; SELF-OPTIMIZING; STOCHASTIC DEPENDENCE;

REINFORCEMENT LEARNING;

EID: 77949509398     PISSN: 03043975     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tcs.2008.06.039     Document Type: Article

References (18)

1. D. Bosq, Nonparametric Statistics for Stochastic Processes, Springer, 1996.
2. R.I. Brafman, M. Tennenholtz, A general polynomial time algorithm for near-optimal reinforcement learning, in: Proc. 17th International Joint Conference on Artificial Intelligence, IJCAI-01, 1999, pp. 734-739.
3. N. Cesa-Bianchi, G. Lugosi, Prediction, Learning, and Games, Cambridge University Press, New York, 2006.
4. I. Csiszar, P.C. Shields, Notes on information theory and statistics, in: Foundations and Trends in Communications and Information Theory, 2004.
5. D. Pucci de Farias, N. Megiddo, How to combine expert (and novice) advice when actions impact the environment? in: Sebastian Thrun, Lawrence Saul, Bernhard Schölkopf (Eds.), Advances in Neural Information Processing Systems, vol. 16, MIT Press, Cambridge, MA, 2004.
6. J.L. Doob, Stochastic Processes, John Wiley & Sons, New York, 1953.
7. E. Even-Dar, S.M. Kakade, Y. Mansour, Reinforcement learning in POMDPs without resets, in: IJCAI, 2005, pp. 690-695.
8. M. Hutter, J. Poland, Prediction with expert advice by following the perturbed leader for general weights, in: Proc. 15th International Conf. on Algorithmic Learning Theory, ALT'04, in: LNAI, vol. 3244, Springer, Padova, Berlin, 2004, pp. 279-293. (Pubitemid 41050298)
9. M. Hutter, Self-optimizing and Pareto-optimal policies in general environments based on Bayes-mixtures, in: Proc. 15th Annual Conference on Computational Learning Theory, COLT 2002, in: Lecture Notes in Artificial Intelligence, Springer, Sydney, Australia, July 2002, pp. 364-379.
10. M. Hutter, Optimality of universal Bayesian prediction for general loss and alphabet, Journal of Machine Learning Research 4 (2003) 971-1000.
11. M. Hutter, Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability, Springer, Berlin, 2005, 300 pages http://www.hutter1.net/ai/uaibook.htm.
12. M. Hutter, General discounting versus average reward, in: Proc. 17th International Conf. on Algorithmic Learning Theory, ALT'06, in: LNAI, vol. 4264, Springer, Barcelona, Berlin, 2006, pp. 244-258. (Pubitemid 44705632)
13. P.R. Kumar, P.P. Varaiya, Stochastic Systems: Estimation, Identification, and Adaptive Control, Prentice Hall, Englewood Cliffs, NJ, 1986.
14. J. Poland, M. Hutter, Defensive universal learning with experts, in: Proc. 16th International Conf. on Algorithmic Learning Theory, ALT'05, in: LNAI, vol. 3734, Springer, Singapore, Berlin, 2005, pp. 356-370.
15. J. Poland, M. Hutter, Universal learning of repeated matrix games, in: Annual Machine Learning Conference of Belgium and The Netherlands, Benelearn'06, Ghent, 2006.
16. D. Ryabko, M. Hutter, Predicting Non-Stationary Processes, Applied Mathematics Letters 21 (5) (2008) 477-482. (Pubitemid 351424908)
17. S.J. Russell, P. Norvig, Artificial Intelligence. A Modern Approach, Prentice-Hall, Englewood Cliffs, 1995.
18. R. Sutton, A. Barto, Reinforcement Learning: An Introduction, MIT Press, Cambridge, MA, 1998.