Predicting a binary sequence almost as well as the optimal biased coin

Information and Computation

Volumn 182, Issue 2, 2003, Pages 73-94

  Freund, Yoav ^a  

^a HEBREW UNIVERSITY OF JERUSALEM   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; OPTIMIZATION; PREDICTIVE CONTROL SYSTEMS; PROBABILITY;

BAYES ALGORITHM;

BINARY SEQUENCES;

EID: 0037959556     PISSN: 08905401     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0890-5401(02)00033-0     Document Type: Article

References (42)

1. Abramowitz, Stegun, Handbook of Mathematical Functions, National Bureau of Standards, 1970.
2. A. Barron, J. Rissanen, B. Yu, The minimum description length principle in coding and modeling, IEEE Trans. Inform. Theory 44 (6) (1998) 2743-2760; Inform. Theory: 1948-1998.
3. A. Barron, J. Rissanen, B. Yu, The minimum description length principle in coding and modeling, IEEE Trans. Inform. Theory 44 (6) (1998) 2743-2760; Inform. Theory: 1948-1998.
4. J.M. Bernardo, Reference posterior distributions for Bayesian inference, J. Roy. Stat. Soc. B. 41 (1979) 113-147.
5. D. Blackwell, An analog of the minimax theorem for vector payoffs, Pacific J. Math. 6 (1) (1956) 1-8.
6. D. Blackwell, M.A. Girshick, Theory of Games and Statistical Decisions, Dover, New York, 1954.
7. N. Cesa-Bianchi, Y. Freund, D.P. Helmbold, D. Haussler, R.E. Schapire, M. K. Warmuth. How to use expert advice, in: Proceedings of the Twenty-Fifth Annual ACM Symposium on the Theory of Computing, 1993, pp 382-391.
8. N. Cesa-Bianchi, G. Lugosi, On prediction of individual sequences, Ann. Stat. 27 (6) (1999) 1865-1895.
9. B.S. Clarke, A.R. Barron, Jeffrey's prior is asymptotically least favorable under entropic risk, J. Stat. Planning Inference 41 (1994) 37-60.
10. T.M. Cover, E. Ordentlich, Universal portfolios with side information, IEEE Trans. Inform. Theory (March) (1996).
11. T.M. Cover, Behavior of sequential predictors of binary sequences, in: Transactions of the Fourth Prague Conference on Information Theory, Statistical Decision Functions, Random Processes (Prague, 1965), Academia, Prague, 1967, pp. 263-272.
12. T.M. Cover, Universal portfolios, Math. Finance 1 (1) (1991) 1-29.
13. L.D. Davisson, Universal noiseless coding, IEEE Trans. Inform. Theory 19 (1973) 783-795.
14. N.G. de Buijn, Asymptotic Methods in Analysis. Dover, New York, 1958, 1981. A good introduction to Laplace Method and the Saddle Point method for approximating integrals.
15. M. Feder, N. Merhav, M. Gutman, Universal prediction of individual sequences, IEEE Trans. Inform. Theory 38 (1992) 1258-1270.
16. T.S. Ferguson, Mathematical Statistics: A Decision Theoretic Approach, Academic Press, New York, 1967.
17. D.P. Foster, Prediction in the worst case, Ann. Stat. 19 (2) (1991) 1084-1090.
18. D.P. Foster, R. Vohra, Regret in the on-line decision problem. Games Econ. Behavior 29 (1-2) (1999) 7-35, Learning in games: a symposium in honor of David Blackwell.
19. Y. Freund, Predicting a binary sequence almost as well as the optimal biased coin, in: Proceedings of the Ninth Annual Conference on Computational Learning Theory, 1996.
20. J. Hannan, Approximation to Bayes risk in repeated play, in: M. Dresher, A.W. Tucker, P. Wolfe (Eds.), Contributions to the Theory of Games, vol. III, Princeton University Press, Princeton, NJ, 1957, pp. 97-139.
21. S. Hart, A. Mas-Colell, A simple adaptive procedure leading to correlated equilibrium, Econometrica 68 (5) (2000) 1127-1150.
22. D. Haussler, A general minimax result for relative entropy, IEEE Trans. Inform. Theory 43 (4) (1997) 1276-1280.
23. D. Haussler, J. Kivinen, M.K. Warmuth, Tight worst-case loss bounds for predicting with expert advice, in: Computational Learning Theory: Second European Conference, EuroCOLT '95, Springer, Berlin, 1995, pp. 69-83.
24. D. Haussler, M. Opper, Worst case prediction over sequences under log loss, in: D. O'Leary, G. Cybenko, J. Rissanen (Eds.), The Mathematics of Information Coding, Extraction and Distribution, Springer, Berlin, 1998.
25. M. Li, P. Vitányi, in: An Introduction to Kolmogorov Complexity and its Applications, Texts and Monogaraphs in Computer Science, Springer, Berlin, 1993.
26. N. Littlestone, Learning when irrelevant attributes abound, in: 28th Annual Symposium on Foundations of Computer Science, October 1987, pp. 68-77.
27. N. Littlestone, M.K. Warmuth, The weighted majority algorithm, Inform. Comput. 108 (1994) 212-261.
28. J.D. Murray, Asymptotic Analysis, Springer, Berlin, 1973.
29. J. Rissanen, in: Stochastic Complexity in Statistical Inquiry, Series in Computer Science, vol. 15, World Scientific, Singapore, 1989.
30. J. Rissanen, G.G. Langdon Jr, Universal modeling and coding, IEEE Trans. Inform. Theory IT-27 (1) (1981) 12-23.
31. J.J. Rissanen, Fisher information and stochastic complexity, IEEE Trans. Inform. Theory 42 (1) (1996) 40-47.
32. Y.M. Shtar'kov, Universal sequential coding of single messages, Problems Inform. Transmission (Trans. Russian) 23 (July-September) (1987) 175-186.
33. Yu.M. Shtarkov, Coding of descrete sources with unknown statistics, in: I. Csiszar, P. Elias (Eds.), Topics in Information Theory, North-Holland, Amsterdam, 1975, pp. 559-574.
34. J. Suzuki, Some notes on universal noiseless coding, IEICE Trans. Fundam. E78-A (12) (1995).
35. V.G. Vovk, A game of prediction with expert advice, J. Comp. Syst. Sci, 56 (2) (1998) 153-173.
36. V.G. Vovk, Aggregating strategies, in: Proceedings of the Third Annual Workshop on Computational Learning Theory, 1990, pp. 371-383.
37. G.N. Watson, Theory of Bessel functions, Cambridge University Press, Cambridge, 1952.
38. F.M.J. Willems, Y.M. Shtarkov, T.J. Tjalkens, The context tree weighting method: basic properties, IEEE Trans. Inform. Theory 41 (3) (1995) 653-664.
39. Q. Xie, A. Barron, Minimax redundancy for the class of memoryless sources, IEEE Trans. Inform. Theory 43 (1997) 446-657.
40. Q. Xie, A.R. Barron, Asymptotic minimax regret for data compression, gambling, and prediction, IEEE Trans. Inform. Theory 46 (2) (2000) 431-445.
41. K. Yamanishi, A decision-theoretic extension of stochastic complexity and its applications to learning, IEEE Trans. Inform. Theory 44 (4) (1998) 1424-1439.
42. Z. Zhang, Discrete noninformative prioirs, Ph.D. Thesis, Yale University, 1994.