Sparsity regret bounds for individual sequences in online linear regression

Journal of Machine Learning Research

Volumn 19, Issue , 2011, Pages 377-395

  Gerchinovitz, Sébastien ^a  

^a ECOLE NORMALE SUPÉRIEURE   (France)

Author keywords

Individual Sequences; Online Linear Regression; Regret Bounds; Sparsity

Indexed keywords

ARTIFICIAL INTELLIGENCE; SOFTWARE ENGINEERING;

EXPONENTIAL WEIGHTING; GAUSSIANS; INDIVIDUAL SEQUENCES; ONLINE LINEAR REGRESSION; REGRET BOUNDS; RISK BOUNDS; SPARSITY; STOCHASTIC SETTINGS;

RISK PERCEPTION;

EID: 84898026009     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (28)

1. J.-Y. Audibert. Fast learning rates in statistical inference through aggregation. Ann. Statist., 37(4):1591-1646, 2009.
2. P. Auer, N. Cesa-Bianchi, and C. Gentile. Adaptive and self-confident on-line learning algorithms. J. Comp. Sys. Sci, 64:48-75, 2002. (Pubitemid 34473696)
3. K. S. Azoury and M. K. Warmuth. Relative loss bounds for on-line density estimation with the exponential family of distributions. Mach. Learn., 43(3):211-246, 2001.
4. G. Biau, K. Bleakley, L. Györfi, and G. Ottucsák. Nonparametric sequential prediction of time series. J. Nonparametr. Stat., 22(3-4):297-317, 2010.
5. L. Birgé and P. Massart. Gaussian model selection. J. Eur. Math. Soc., 3:203-268, 2001.
6. F. Bunea, A. B. Tsybakov, and M. H. Wegkamp. Aggregation for regression learning. Technical report, 2004. Available at http://arxiv.org/abs/math/ 0410214.
7. 1 penalized least squares. In Proceedings of the 19th Annual Conference on Learning Theory (COLT'06), pages 379-391, 2006.
8. F. Bunea, A. B. Tsybakov, and M. H. Wegkamp. Aggregation for Gaussian regression. Ann. Statist., 35(4):1674-1697, 2007.
9. O. Catoni. Statistical learning theory and stochastic optimization. Springer, New York, 2004.
10. N. Cesa-Bianchi and G. Lugosi. Prediction, Learning, and Games. Cambridge University Press, 2006.
11. N. Cesa-Bianchi, Y. Mansour, and G. Stoltz. Improved second-order bounds for prediction with expert advice. Mach. Learn., 66(2/3):321, 2007.
12. A. Dalalyan and A. B. Tsybakov. Aggregation by exponential weighting and sharp oracle inequalities. In Proceedings of the 20th Annual Conference on Learning Theory (COLT'07), pages 97-111, 2007.
13. A. Dalalyan and A. B. Tsybakov. Aggregation by exponential weighting, sharp PACBayesian bounds and sparsity. Mach. Learn., 72(1-2):391, 2008.
14. A. Dalalyan and A. B. Tsybakov. Sparse regression learning by aggregation and Langevin Monte-Carlo. In Proceedings of the 22nd Annual Conference on Learning Theory (COLT'09), pages 83-92, 2009.
15. A. Dalalyan and A. B. Tsybakov. Mirror averaging with sparsity priors. Bernoulli, 2011. To appear. Available at http://hal.archives-ouvertes.fr/hal- 00461580/.
16. J. C. Duchi, S. Shalev-Shwartz, Y. Singer, and A. Tewari. Composite objective mirror descent. In Proceedings of the 23rd Annual Conference on Learning Theory (COLT'10), pages 14-26, 2010.
17. Y. Freund, R. E. Schapire, Y. Singer, and M. K. Warmuth. Using and combining predictors that specialize. In Proceedings of the 29th annual ACM Symposium on Theory of Computing (STOC'97), pages 334-343, 1997.
18. S. Gerchinovitz. Sparsity regret bounds for individual sequences in online linear regression. Technical report, 2011. Available at http://arxiv.org/abs/1101.1057.
19. L. Györfi and G. Ottucsák. Sequential prediction of unbounded stationary time series. IEEE Trans. Inform. Theory, 53(5):1866-1872, 2007.
20. L. Györfi, M. Kohler, A. Krzyżak, and H. Walk. A distribution-free theory of nonparametric regression. Springer Series in Statistics. Springer-Verlag, New York, 2002.
21. S. M. Kakade and A. Tewari. On the generalization ability of online strongly convex programming algorithms. In Advances in Neural Information Processing Systems 21 (NIPS'08), pages 801-808. 2009.
22. J. Kivinen and M. K. Warmuth. Averaging expert predictions. In Proceedings of the 4th European Conference on Computational Learning Theory (EuroCOLT'99), pages 153-167, 1999.
23. J. Langford, L. Li, and T. Zhang. Sparse online learning via truncated gradient. J. Mach. Learn. Res., 10:777-801, 2009.
24. lq-ball sparsity. In Proceedings of the 47th annual Allerton conference on communication, control, and computing (Allerton'09), pages 251-257, 2009.
25. M. W. Seeger. Bayesian inference and optimal design for the sparse linear model. J. Mach. Learn. Res., 9:759-813, 2008.
26. 1-regularized loss minimization. In Proceedings of the 26th Annual International Conference on Machine Learning (ICML'09), pages 929-936, 2009.
27. V. Vovk. Competitive on-line statistics. Internat. Statist. Rev., 69:213-248, 2001.
28. L. Xiao. Dual averaging methods for regularized stochastic learning and online optimization. J. Mach. Learn. Res., 11:2543-2596, 2010.