Generalization error bounds for aggregation by mirror descent with averaging

Advances in Neural Information Processing Systems

Volumn , Issue , 2005, Pages 603-610

  Juditsky, Anatoli ^a   Nazin, Alexander ^b   Tsybakov, Alexandre ^c   Vayatis, Nicolas ^c  

^a UNIV GRENOBLE ALPES   (France)

^b TRAPEZNIKOV INSTITUTE OF CONTROL SCIENCES   (Russian Federation)

^c University of Paris VII LPSM partially supported by ERC Starting Grant 680275 MALIG   (France)

Author keywords

[No Author keywords available]

Indexed keywords

BASE FUNCTION; CONVERGENCE RATES; DUAL SPACES; GENERALIZATION ERROR; GENERALIZATION ERROR BOUNDS; GRADIENT DESCENT; HIGH-DIMENSIONAL PROBLEMS; ON-LINE SETTING; STOCHASTIC PROCEDURE; UPPER BOUND;

ERROR ANALYSIS;

MIRRORS;

EID: 77952686979     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. Beck, A. & Teboulle, M. (2003) Mirror descent and nonlinear projected subgradient methods for convex optimization. Operations Research Letters, 31:167-175.
2. Ben-Tal, A., Margalit, T. & Nemirovski, A. (2001) The Ordered Subsets Mirror Descent optimization method and its use for the Positron Emission Tomography reconstruction problem. SIAM J. on Optimization, 12:79-108.
3. Ben-Tal, A. & Nemirovski, A.S. (1999) The conjugate barrier mirror descent method for non-smooth convex optimization. MINERVA Optimization Center Report, Technion Institute of Technology. Available at http://iew3.technion.ac. il/Labs/Opt/opt/Pap/CP MD.pdf
4. Cesa-Bianchi, N. & Gentile, C. (2005) Improved risk tail bounds for on-line algorithms. Submitted.
5. Cesa-Bianchi, N., Conconi, A. & Gentile, C. (2004) On the generalization ability of on-line learning algorithms. IEEE Transactions on Information Theory, 50(9):2050-2057.
6. Helmbold, D.P., Kivinen, J. & Warmuth, M.K. (1999) Relative loss bounds for single neurons. IEEE Trans. on Neural Networks, 10(6):1291-1304.
7. Juditsky, A. & Nemirovski, A. (2000) Functional aggregation for nonparametric estimation. Annals of Statistics, 28(3): 681-712.
8. Juditsky, A.B., Nazin, A.V., Tsybakov, A.B. & Vayatis N. (2005) Recursive Aggregation of Estimators via the Mirror Descent Algorithm with Averaging. Technical Report LPMA, Université Paris 6. Available at http://www.proba.jussieu.fr/pageperso/vayatis/publication.html
9. Kiwiel, K.C. (1997) Proximal minimization methods with generalized Bregman functions. SIAM J. Control Optim., 35:1142-1168.
10. Kivinen J. & Warmuth M.K. (1997) Additive versus exponentiated gradient updates for linear prediction. Information and Computation, Vol.132(1): 1-64.
11. Lugosi, G. & Vayatis, N. (2004) On the Bayes-risk consistency of regularized boosting methods (with discussion). Annals of Statitics, 32(1): 30-55.
12. Nemirovski, A.S. & Yudin, D.B. (1983) Problem Complexity and Method Efficiency in Optimization. Wiley-Interscience.
13. Polyak, B.T. & Juditsky, A.B. (1992) Acceleration of stochastic approximation by averaging. SIAM J. Control Optim., 30:838-855.
14. Scovel, J.C. & Steinwart, I. (2005) Fast Rates for Support Vector Machines. In Proceedings of the 18th Conference on Learning Theory (COLT 2005), Bertinoro, Italy.
15. 1 Penalty. Preprint, University of Leiden.
16. Tsybakov, A. (2003) Optimal Rates of Aggregation. Proceedings of COLT'03, LNCS, Springer, Vol. 2777:303-313.
17. Tsybakov, A. (2004) Optimal aggregation of classifiers in statistical learning. Annals of Statistics, 32(1):135-166.
18. Zhang, T. (2004) Statistical behavior and consistency of classification methods based on convex risk minimization (with discussion). Annals of Statistics, 32(1):56-85.
19. Zhang, T. (2004) Solving large scale linear prediction problems using stochastic gradient descent algorithms. In Proceedings of ICML'04.