Strong entropy concentration, game theory, and algorithmic randomness

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 2111, Issue , 2001, Pages 320-336

  Grünwald, Peter ^a  

^a EURANDOM   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

GAME THEORY;

ALGORITHMIC RANDOMNESS; CONCENTRATION PHENOMENA; EXPONENTIAL FAMILY; GENERAL LOSS FUNCTIONS; LOGARITHMIC LOSS; MAXIMUM ENTROPY DISTRIBUTION; RELATIVE ENTROPY; SEQUENTIAL PREDICTION;

ENTROPY;

EID: 84943263119     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/3-540-44581-1_21     Document Type: Conference Paper

References (25)

1. K. Azoury and M. Warmuth. Relative loss bounds for on-line density estimation with the exponential family of distributions. In Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence (UAI’99), pages 31–40. Morgan Kaufmann, 1999.
2. A. Barron, J. Rissanen, and B. Yu. The minimum description length principle in codingand modeling. IEEE Transactions on Information Theory, 44(6):2743–2760, 1998.
3. P. Billingsley. Convergence of Probability Measures. Wiley, 1968.
4. T.M. Cover and J.A. Thomas. Elements of Information Theory. Wiley Interscience, New York, 1991.
5. I. Csiszár. I-divergence geometry of probability distributions and minimization problems. The Annals of Probability, 3(1):146–158, 1975.
6. I. Csiszár. Sanov property, generalized i-projection and a conditional limit theorem. The Annals of Probability, 12(3):768–793, 1984.
7. I. Csiszár. Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems. The Annals of Statistics, 19(4):2032–2066, 1991.
8. M. Feder. Maximum entropy as a special case of the minimum description length criterion. IEEE Transactions on Information Theory, 32(6):847–849, 1986.
9. W. Feller. An Introduction to Probability Theory and Its Applications, volume 2. Wiley, 1968. Third edition.
10. P. D. Grünwald. Maximum entropy and the glasses you are looking through. In Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence (UAI 2000). Morgan Kaufmann Publishers, 2000.
11. P.D. Grünwald. The Minimum Description Length Principle and Reasoning under Uncertainty. PhD thesis, University of Amsterdam, The Netherlands, October 1998. Available as ILLC Dissertation Series 1998-03; see www.cwi.nl/˜pdg.
12. P.D. Grünwald: Strong entropy concentration, coding, game theory and randomness. Technical Report 010, EURANDOM, 2001.
13. E.T. Jaynes. Where do we stand on maximum entropy? In R.D. Levine and M. Tribus, editors, The Maximum Entropy Formalism, pages 15–118. MIT Press, Cambridge, MA, 1978.
14. E.T. Jaynes. On the rationale of maximum-entropy methods. Proceedings of the IEEE, 70 (939-951), 1982.
15. E.T. Jaynes. Papers on Probability, Statistics and Statistical Physics. Kluwer Academic Publishers, second edition, 1989.
16. E.T. Jaynes. Probability theory: the logic of science. Available at ftp://bayes.wustl.edu/Jaynes.book/, 1996.
17. J. N. Kapur and H. K Kesavan. Entropy Optimization Principles with Applications. Academic Press, Inc., 1992.
18. R.E. Kass and P.W. Voss. Geometrical Foundations of Asymptotic Inference. Wiley Interscience, 1997.
19. J. Lafferty. Additive models, boosting and inference for generalized divergences. In Proceedings of the Twelfth Annual Workshop on Computational Learning Theory (COLT’99), 1999.
20. M. Li and P.M.B. Vitányi. An Introduction to Kolmogorov Complexity and Its Applications. Springer-Verlag, New York, revised and expanded second edition, 1997.
21. N. Merhav and M. Feder. A strongv ersion of the redundancy-capacity theorem of universal coding. IEEE Transactions on Information Theory, 41(3):714–722, 1995.
22. J. Rissanen. Stochastic Complexity in Statistical Inquiry. World Scientific Publishing Company, 1989.
23. J. Rissanen. Strongoptimalit y of the normalized ML models as universal codes, 2001. To appear in IEEE Transactions on Information Theory.
24. F. Topsøe. Information theoretical optimization techniques. Kybernetika, 15(1), 1979.
25. J. van Campenhout and T. Cover. Maximum entropy and conditional probability. IEEE Transactions on Information Theory, IT-27(4):483–489, 1981.