A second-order bound with excess losses

Journal of Machine Learning Research

Volumn 35, Issue , 2014, Pages 176-196

  Gaillard, Pierre ^a,b   Stoltz, Gilles ^b   Van Erven, Tim ^c  

^a EDF R AND D   (France)

^b Centre National de la Recherche Scientifique   (France)

^c UNIVERSITÉ PARIS SACLAY   (France)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; SOFTWARE ENGINEERING;

INSTANTANEOUS LOSS; LEARNING RATES; ONLINE AGGREGATIONS; REGRET BOUNDS; SECOND ORDERS; SECOND-ORDER BOUNDS; STANDARD SETTING; WEIGHTED AVERAGES;

ALGORITHMS;

EID: 84937940324     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (25)

1. Dmitry Adamskiy, Wouter M. Koolen, Alexey Chernov, and Vladimir Vovk. A closer look at adaptive regret. In Algorithmic Learning Theory (ALT), pages 290-304, 2012.
2. Peter Auer, Nicolò Cesa-Bianchi, and Claudio Gentile. Adaptive and self-confident on-line learning algorithms. Journal of Computer and System Sciences, 64:48-75, 2002.
3. David Blackwell. An analog of the minimax theorem for vector payoffs. Pacific Journal of Mathematics, 6:1-8, 1956.
4. Avrim Blum. Empirical support for winnow and weighted-majority algorithms: Results on a calendar scheduling domain. Machine Learning, 26:5-23, 1997.
5. Avrim Blum and Yishay Mansour. From external to internal regret. Journal of Machine Learning Research, 8:1307-1324, 2007.
6. N. Cesa-Bianchi and G. Lugosi. Potential-based algorithms in on-line prediction and game theory. Machine Learning, 3(51):239-261, 2003.
7. Nicolò Cesa-Bianchi and Gábor Lugosi. Prediction, Learning, and Games. Cambridge University Press, 2006.
8. Nicolò Cesa-Bianchi, Gábor Lugosi, and Gilles Stoltz. Minimizing regret with label efficient prediction. IEEE Trans. Inform. Theory, 51:77-92, 2005.
9. Nicolò Cesa-Bianchi, Yishay Mansour, and Gilles Stoltz. Improved second-order bounds for prediction with expert advice. Machine Learning, 66(2/3):321-352, 2007.
10. Alexey V. Chernov and Vladimir Vovk. Prediction with advice of unknown number of experts. CoRR, abs/1006.0475, 2010.
11. Chao-Kai Chiang, Tianbao Yang, Chia-Jung Lee, Mehrdad Mahdavi, Chi-Jen Lu, Rong Jin, and Shenghuo Zhu.. Online optimization with gradual variations. In Proceedings of the 25th Annual Conference on Learning Theory (COLT), pages 6.1-6.20, 2012.
12. Steven De Rooij, Tim Van Erven, Peter D. Grünwald, and Wouter M. Koolen. Follow the leader if you can, hedge if you must. Available at arXiv:1301.0534 [cs. LG], 2013.
13. Marie Devaine, Pierre Gaillard, Yannig Goude, and Gilles Stoltz. Forecasting electricity consumption by aggregating specialized experts. Machine Learning, 90(2):231-260, 2013.
14. John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. 12(Jul):2121-2159, 2011.
15. David Freedman. On tail probabilities for martingales. Annals of Probability, 3:100-118, 1975.
16. Yoav Freund and Robert E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55:119-139, 1997.
17. Yoav Freund, Robert E. Schapire, Yoram Singer, and Manfred K. Warmuth. Using and combining predictors that specialize. In Proceedings of the twenty-ninth annual ACM symposium on Theory of computing, pages 334-343, 1997.
18. Pierre Gaillard, Gilles Stoltz, and Tim Van Erven. A second-order bound with excess losses. Extended version available at arXiv:1301.0534 [cs. LG], 2014.
19. László Györfi and György Ottucsák. Sequential prediction of unbounded stationary time series. IEEE Transactions on Information Theory, 53(5):1866-1872, 2007.
20. Elad Hazan and Satyen Kale. Extracting certainty from uncertainty: regret bounded by variation in costs. Machine Learning, 80(2-3):165-188, 2010.
21. Wouter M. Koolen, Dmitry Adamskiy, and Manfred K. Warmuth. Putting Bayes to sleep. In Advances in Neural Information Processing Systems 25 (NIPS 2012), pages 135-143, 2013.
22. Nick Littlestone and Manfred K. Warmuth. The weighted majority algorithm. Information and Computation, 108(2):212-261, 1994.
23. Tim Van Erven, Peter Grünwald, Wouter M. Koolen, and Steven De Rooij. Adaptive Hedge. In Advances in Neural Information Processing Systems 25 (NIPS 2012), 2011.
24. Vladimir Vovk. A game of prediction with expert advice. Journal of Computer and System Sciences, 56(2):153-173, 1998.
25. Olivier Wintenberger. Optimal learning with bernstein online aggregation. Extended version available at arXiv:1404.1356 [stat. ML], 2014.