Follow the leader if you can, hedge if you must

Journal of Machine Learning Research

Volumn 15, Issue , 2014, Pages 1281-1316

  De Rooij, Steven ^a   Van Erven, Tim ^b   Grünwald, Peter D ^c   Koolen, Wouter M ^c  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

^b UNIVERSITÉ PARIS SUD   (France)

^c LEIDEN UNIVERSITY   (Netherlands)

Author keywords

Hedge; Learning rate; Mixability; Online learning; Prediction with expert advice

Indexed keywords

INVESTMENTS; LEARNING ALGORITHMS; STOCHASTIC SYSTEMS;

HEDGE; LEARNING RATES; MIXABILITY; ONLINE LEARNING; PREDICTION WITH EXPERT ADVICE;

WOODEN FENCES;

EID: 84901638727     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (28)

1. Jean-Yves Audibert. PAC-Bayesian statistical learning theory. PhD thesis, Universit Paris VI, 2004.
2. Peter Auer, Nicol Cesa-Bianchi, and Claudio Gentile. Adaptive and self-confident on-line learning algorithms. Journal of Computer and System Sciences, 64:48-75, 2002. (Pubitemid 34473696)
3. Olivier Catoni. PAC-Bayesian Supervised Classification. Lecture Notes-Monograph Series. IMS, 2007.
4. Nicol Cesa-Bianchi and Gbor Lugosi. Prediction, learning, and games. Cambridge University Press, 2006.
5. Nicol Cesa-Bianchi, Yoav Freund, David Haussler, David P. Helmbold, Robert E. Schapire, and Manfred K.Warmuth. How to use expert advice. Journal of the ACM, 44(3):427-485, 1997. (Pubitemid 127634586)
6. Nicol Cesa-Bianchi, Yishay Mansour, and Gilles Stoltz. Improved second-order bounds for prediction with expert advice. Machine Learning, 66(2/3):321-352, 2007. (Pubitemid 46360617)
7. Kamalika Chaudhuri, Yoav Freund, and Daniel Hsu. A parameter-free hedging algorithm. In Advances in Neural Information Processing Systems 22 (NIPS 2009), pages 297-305, 2009.
8. Alexey V. Chernov and Vladimir Vovk. Prediction with advice of unknown number of experts. In Peter Grnwald and Peter Spirtes, editors, UAI, pages 117-125. AUAI Press, 2010.
9. Marie Devaine, Pierre Gaillard, Yannig Goude, and Gilles Stoltz. Forecasting electricity consumption by aggregating specialized experts; a review of the sequential aggregation of specialized experts, with an application to Slovakian and French country-wide one-dayahead (half-)hourly predictions. Machine Learning, 90(2):231-260, February 2013.
10. Yoav Freund and Robert E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55:119-139, 1997. (Pubitemid 127433398)
11. Yoav Freund and Robert E. Schapire. Adaptive game playing using multiplicative weights. Games and Economic Behavior, 29:79-103, 1999.
12. Sbastien Gerchinovitz. Prdiction de suites individuelles et cadre statistique classique: tude de quelques liens autour de la rgression parcimonieuse et des techniques dagrgation. PhD thesis, Universit Paris-Sud, 2011.
13. Peter Grnwald. Safe learning: bridging the gap between Bayes, MDL and statistical learning theory via empirical convexity. In Proceedings of the 24th International Conference on Learning Theory (COLT 2011), pages 551-573, 2011.
14. Peter Grnwald. The safe Bayesian: learning the learning rate via the mixability gap. In Proceedings of the 23rd International Conference on Algorithmic Learning Theory (ALT 2012), 2012.
15. Peter Grnwald and John Langford. Suboptimal behavior of Bayes and MDL in classification under misspecification. Machine Learning, 66(2-3):119-149, 2007. DOI 10.1007/s10994-007-0716-7. (Pubitemid 46360609)
16. Lszl Gyrfi and Gyrgy Ottucsk. Sequential prediction of unbounded stationary time series. IEEE Transactions on Information Theory, 53(5):1866-1872, 2007.
17. Elad Hazan and Satyen Kale. Extracting certainty from uncertainty: Regret bounded by variation in costs. In Proceedings of the 21st Annual Conference on Learning Theory (COLT), pages 57-67, 2008.
18. Marcus Hutter and Jan Poland. Adaptive online prediction by following the perturbed leader. Journal of Machine Learning Research, 6:639-660, 2005.
19. Adam Kalai and Santosh Vempala. Efficient algorithms for online decision. In Proceedings of the 16st Annual Conference on Learning Theory (COLT), pages 506-521, 2003.
20. Yuri Kalnishkan and Michael V. Vyugin. The weak aggregating algorithm and weak mixability. In Proceedings of the 18th Annual Conference on Learning Theory (COLT), pages 188-203, 2005.
21. Wouter M. Koolen, Manfred K. Warmuth, and Jyrki Kivinen. Hedging structured concepts. In A.T. Kalai and M. Mohri, editors, Proceedings of the 23rd Annual Conference on Learning Theory (COLT 2010), pages 93-105, 2010.
22. Nick Littlestone and Manfred K. Warmuth. The weighted majority algorithm. Information and Computation, 108(2):212-261, 1994.
23. Koji Tsuda, Gunnar Rtsch, and Manfred K. Warmuth. Matrix exponentiated gradient updates for on-line learning and Bregman projection. Journal of Machine Learning Research, 6:995-1018, 2005.
24. Tim van Erven, Peter Grnwald, Wouter M. Koolen, and Steven de Rooij. Adaptive hedge. In Advances in Neural Information Processing Systems 24 (NIPS 2011), pages 1656-1664, 2011.
25. Vladimir Vovk. A game of prediction with expert advice. Journal of Computer and System Sciences, 56(2):153-173, 1998. (Pubitemid 128415329)
26. Vladimir Vovk. Competitive on-line statistics. International Statistical Review, 69(2):213-248, 2001.
27. Vladimir Vovk, Akimichi Takemura, and Glenn Shafer. Defensive forecasting. In Proceedings of AISTATS 2005, 2005. Archive version available at http://www.vovk.net/df.
28. Tong Zhang. Information theoretical upper and lower bounds for statistical estimation. IEEE Transactions on Information Theory, 52(4):1307-1321, 2006.