Convergence and loss bounds for Bayesian sequence prediction

IEEE Transactions on Information Theory

Volumn 49, Issue 8, 2003, Pages 2061-2067

  Hutter, Marcus ^a  

^a DALLE MOLLE INSTITUTE FOR ARTIFICIAL INTELLIGENCE IDSIA   (Switzerland)

Author keywords

Bayesian sequence prediction; Convergence; General loss function and bounds; Mixture distributions

Indexed keywords

ALGORITHMS; BOUNDARY CONDITIONS; CONVERGENCE OF NUMERICAL METHODS; DECISION THEORY; FUNCTION EVALUATION; SET THEORY;

BAYESIAN SEQUENCE PREDICTION; GENERAL LOSS FUNCTION AND BOUNDS; MIXTURE DISTRIBUTIONS;

PROBABILITY DISTRIBUTIONS;

EID: 0043268070     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2003.814488     Document Type: Letter

References (18)

1. N. Merhav and M. Feder, "Universal prediction," IEEE Trans. Inform. Theory, vol. 44, pp. 2124-2147, Oct. 1998.
2. J. L. Doob, Stochastic Processes. New York: Wiley, 1953.
3. M. Li and P. M. B. Vitányi, An Introduction to Kolmogorov Complexity and Its Applications, 2nd ed. New York: Springer-Verlag, 1997.
4. V. G. Vovk, "On a randomness criterion," DOKLADY: Russian Acad. Sciences Dokl. Math., vol. 35, 1987. Formerly, Sov. Math.-Dokl.
5. P. M. Vitányi and M. Li, "Minimum description length induction, Bayesianism, and Kolmogorov complexity," IEEE Trans. Inform. Theory, vol. 46, pp. 446-464, Mar. 2000.
6. M. Hutter, "Convergence and error bounds of universal prediction for general alphabet," in Proc. 12th European Conf. Machine Learning (ECML-2001), 2001, [Online]. Available: ftp://ftp.idsia.ch/pub/techrep/IDSIA-07-01.ps.gz, pp. 239-250.
7. ____, "General loss bounds for universal sequence prediction," in Proc. 18th Int. Conf. Machine Learning (ICML-2001), 2001, [Online]. Available: ftp://ftp.idsia.ch/pub/techrep/IDSIA-03-01.ps.gz, pp. 210-217.
8. ____, (2002) Optimality of universal Bayesian prediction for general loss and alphabet. Istituto Dalle Molle di Studi sull'Intelligenza Artificiale (IDSIA), Manno (Lugano), Switzerland. [Online]. Available: ftp://ftp.idsia.ch/pub/techrep/IDSIA-02-02.ps.gz. Tech. Rep. IDSIA-02-02
9. R. J. Solomonoff, "Complexity-based induction systems: Comparisons and convergence theorems," IEEE Trans. Inform. Theory, vol. IT-24, pp. 422-432, July 1978.
10. P. D. Grünwald, "The minimum description length principle and reasoning under uncertainty," Ph.D. dissertation, Univ. Amsterdam, Amsterdam, The Netherlands, 1998.
11. T. M. Cover and J. A. Thomas, Elements of Information Theory, ser. Wiley Series in Telecommunications. New York: Wiley, 1991.
12. A. A. Borovkov and A. Moullagaliev, Mathematical Statistics. New York: Gordon & Breach, 1998.
13. A. R. Barron, "Limits of information, Markov chains, and projection," in Proc. IEEE Int. Symp. Information Theory (ISIT), Sorrento, Italy, June 2000, p. 25.
14. M. Hutter, "New error bounds for Solomonoff prediction," J. Comput. Syst. Sci., vol. 62, no. 4, pp. 653-667, 2001. [Online]. Available: ftp://ftp.idsia.ch/pub/techrep/IDSIA-11-00.ps.gz.
15. ____, "An open problem regarding the convergence of universal a priori probability," in Proc. 16th Conf. Computational Learning Theory (COLT 2003), 2003, Available: [Online] http://www.hutter1.de/ai/mlconv.htm.
16. N. Cesa-Bianchi et al., "How to use expert advice," J. ACM, vol. 44, no. 3, pp. 427-485, 1997.
17. A. R. Barron, B. S. Clarke, and D. Haussler, "Information bounds for the risk of Bayesian predictions and the redundancy of universal codes," in Proc. IEEE Int. Symp. Information Theory (ISIT), 1993, pp. 54-54.
18. M. Hutter, "Self-optimizing and Pareto-optimal policies in general environments based on Bayes-mixtures," in Proc. 15th Annu. Conf. Computational Learning Theory (COLT 2002) (Lecture Notes in Artificial Intelligence). Sydney, Australia: Springer, 2002, pp. 364-379. [Online]. Available: ftp://ftp.idsia.ch/pub/techrep/IDSIA-04-02.ps.gz.