Worst-case bounds for the logarithmic loss of predictors

Machine Learning

Volumn 43, Issue 3, 2001, Pages 247-264

  Cesa Bianchi, Nicolò ^a   Lugosi, Gábor ^b  

^a UNIVERSITY OF MILAN   (Italy)

^b UNIVERSITAT POMPEU FABRA   (Spain)

Author keywords

Empirical processes; Metric entropy; On line learning; Universal coding; Universal prediction

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; MAXIMUM LIKELIHOOD ESTIMATION; ONLINE SYSTEMS; PROBABILITY DENSITY FUNCTION; PROBABILITY DISTRIBUTIONS;

METRIC ENTROPY; UNIVERSAL PREDICTIONS;

LEARNING SYSTEMS;

EID: 0035371298     PISSN: 08856125     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1010848128995     Document Type: Article

References (21)

1. Azuma, K. (1967). Weighted sums of certain dependent random variables. Tohoku Mathematical Journal, 68, 357-367.
2. Barron, A. & Xie, Q. (1996). Asymptotic minimax regret for data compression, gambling, and prediction. Unpublished manuscript presented at an informal meeting on prediction of individual sequences held at the University of California (Santa Cruz).
3. Cover, T. (1991). Universal portfolios. Mathematical Finance, 1, 1-29.
4. Cover, T. & Ordentlich, E. (1996). Universal portfolios with side information. IEEE Transactions on Information Theory, 42:2, 348-363.
5. Cover, T. & Thomas, J. (1991). Elements of Information Theory. New York: John Wiley and Sons.
6. Feder, M. (1991). Gambling using a finite state machine. IEEE Transactions on Information Theory, 37, 1459-1465.
7. Freund, Y. (1996). Predicting a binary sequence almost as well as the optimal biased coin. In Proceedings of the 9th Annual Conference on Computational Learning Theory (pp. 89-98).
8. Haussler, D. & Barron, A. (1993). How well does the Bayes method work in on-line predictions of {+1, -1} Values? In Proceedings of 3rd NEC Symposium (pp. 74-100).
9. Haussler, D., Kivinen, J., & Warmuth, M. (1998). Sequential prediction of individual sequences under general loss functions. IEEE Transactions on Information Theory, 44, 1906-1925.
10. Merhav, N. & Feder, M. (1998). Universal prediction. IEEE Transactions on Information Theory, 44:6, 2124-2147.
11. Opper, M. & Haussler, D. (1997). Worst case prediction over sequences under log loss. The Mathematics of Information Coding, Extraction, and Distribution. Springer Verlag.
12. Rissanen, J. (1976). Generalized Kraft's inequality and arithmetic coding. IBM Journal of Research and Development, 20, 198-203.
13. Rissanen, J. (1996). Fischer information and stochastic complexity. IEEE Transactions on Information Theory, 42, 40-47.
14. Santis, A. D., Markowski, G., & Wegman, M. (1988). Learning probabilistic prediction functions. In Proceedings of the 1st Annual Workshop on Computational Learning Theory (pp. 312-328).
15. Shtarkov, Y. (1987). Universal sequential coding of single messages. Translated from: Problems in Information Transmission, 23:3, 3-17.
16. Talagrand, M. (1996). Majorizing measures: The generic chaining. Annals of Probability, 24, 1049-1103. (Special Invited Paper).
17. Vovk, V. (1990). Aggregating strategies. In Proceedings of the 3rd Annual Workshop on Computational Learning Theory, 372-383.
18. Vovk, V. (1998). A game of prediction with expert advice. Journal of Computer and System Sciences, 56:2, 153-173.
19. Weinberger, M., Merhav, N., & Feder, M. (1994). Optimal sequential probability assignment for individual sequences. IEEE Transactions on Information Theory, 40, 384-396.
20. Yamanishi, K. (1995). A loss bound model for on-line stochastic algorithms. Information and Computation, 119:1, 39-54.
21. Yamanishi, K. (1998). A decision-theoretic extension of stochastic complexity and its application to learning. IEEE Transactions on Information Theory, 44, 1424-1440.