Optimization with first-order surrogate functions

30th International Conference on Machine Learning, ICML 2013

Volumn , Issue PART 3, 2013, Pages 1820-1828

  Mairal, Julien ^a  

^a INRIA Team Lear Grenoble   (France)

Author keywords

[No Author keywords available]

Indexed keywords

LEARNING SYSTEMS; OPTIMIZATION;

ALGORITHMIC VARIANTS; BLOCK COORDINATE DESCENTS; CONVERGENCE ANALYSIS; FRANK-WOLFE ALGORITHMS; LARGE-SCALE OPTIMIZATION; OBJECTIVE FUNCTIONS; OPTIMIZATION METHOD; OPTIMIZATION TECHNIQUES;

ITERATIVE METHODS;

EID: 84897534825     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (36)

1. Beck, A. and Teboulle, M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci., 2(1):183-202, 2009.
2. Bertsekas, D.P. Nonlinear programming. Athena Scientific Belmont, 1999. 2nd edition.
3. Böhning, D. and Lindsay, B. G. Monotonicity of quadratic- approximation algorithms. Ann. I. Stat. Math., 40(4):-641-663, 1988.
4. Borwein, J.M. and Lewis, A.S. Convex analysis and nonlinear optimization: theory and examples. Springer, 2006.
5. Bottou, L. Large-scale machine learning with stochastic gradient descent. In Proc. COMPSTAT, 2010.
6. Bradley, J.K., Kyrola, A., Bickson, D., and Guestrin, C. Parallel coordinate descent for l1-regularized loss minimization. In Proc. ICML, 2011.
7. 1 minimization. J. Fourier Anal. Appl., 14(5):877-905, 2008.
8. Collins, M., Schapire, R.E., and Singer, Y. Logistic regression, AdaBoost and Bregman distances. Mach. Learn., 48(1):253-285, 2002. (Pubitemid 34247580)
9. Daubechies, I., Defrise, M., and De Mol, C. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pur. Appl. Math., 57(11):1413-1457, 2004. (Pubitemid 39427442)
10. Della Pietra, S., Della Pietra, V., and Lafferty, J. Duality and auxiliary functions for Bregman distances. Technical report, CMU-CS-01-109, 2001.
11. Fan, R.-E., Chang, K.-W., Hsieh, C.-J., Wang, X.-R., and Lin, C.-J. LIBLINEAR: A library for large linear classification. J. Mach. Learn. Res., 9:1871-1874, 2008.
12. Gasso, G., Rakotomamonjy, A., and Canu, S. Recovering sparse signals with non-convex penalties and DC programming. IEEE T. Signal Process., 57(12):4686-4698, 2009.
13. Harchaoui, Z., Juditsky, A., and Nemirovski, A. Conditional gradient algorithms for norm-regularized smooth convex optimization. preprint arXiv:1302.2325v4, 2013.
14. Hazan, E. and Kale, S. Projection-free online learning. In Proc. ICML, 2012.
15. Horst, R. and Thoai, N.V. DC programming: overview. J. Optim. Theory App., 103(1):143, 1999.
16. Juditsky, A. and Nemirovski, A. First order methods for nonsmooth convex large-scale optimization, I: General purpose methods. In Optimization for Machine Learning. MIT Press, 2011.
17. Khan, E., Marlin, B., Bouchard, G., and Murphy, K. Variational bounds for mixed-data factor analysis. In Adv. NIPS, 2010.
18. Lacoste-Julien, S., Jaggi, M., Schmidt, M., and Pletscher, P. Block-coordinate Frank-Wolfe optimization for structural SVMs. In Proc. ICML, 2013.
19. Lange, K., Hunter, D.R., and Yang, I. Optimization transfer using surrogate objective functions. J. Comput. Graph. Stat., 9(1):1-20, 2000.
20. Le Roux, N., Schmidt, M., and Bach, F. A stochastic gradient method with an exponential convergence rate for finite training sets. In Adv. NIPS, 2012.
21. Lee, D.D. and Seung, H.S. Algorithms for non-negative matrix factorization. In Adv. NIPS, 2001.
22. Mairal, J., Bach, F., Ponce, J., and Sapiro, G. Online learning for matrix factorization and sparse coding. J. Mach. Learn. Res., 11:19-60, 2010.
23. Neal, R.M. and Hinton, G.E. A view of the EM algorithm that justifies incremental, sparse, and other variants. Learning in graphical models, 89:355-368, 1998.
24. Nesterov, Y. Introductory lectures on convex optimization. Kluwer Academic Publishers, 2004.
25. Nesterov, Y. Gradient methods for minimizing composite objective functions. Technical report, CORE Discussion Paper, 2007.
26. Nesterov, Y. Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM J. Optimiz., 22(2):341-362, 2012.
27. Nesterov, Y. and Polyak, B.T. Cubic regularization of Newton method and its global performance. Math. Program., 108(1):177-205, 2006.
28. Richtárik, P. and Takáč, M. Iteration complexity of randomized block coordinate descent methods for minimizing a composite function. Math. Program., 2012.
29. Seeger, M.W. and Wipf, D.P. Variational Bayesian inference techniques. IEEE Signal Proc. Mag., 27(6):81-91, 2010.
30. Shalev-Schwartz, S. and Zhang, T. Proximal stochastic dual coordinate ascent, preprint arXiv 1211.2717v1, 2012.
31. 1 regularized loss minimization. In Proc. ICML, 2009.
32. Tseng, P. and Yun, S. A coordinate gradient descent method for nonsmooth separable minimization. Math. Program., 117:387-423, 2009.
33. Wainwright, M.J. and Jordan, M.I. Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn., 1(1-2):1-305, 2008.
34. Wright, S., Nowak, R., and Figueiredo, M. Sparse reconstruction by separable approximation. IEEE T. Signal Process., 57(7):2479-2493, 2009.
35. Zhang, T. Sequential greedy approximation for certain convex optimization problems. IEEE T. Inform. Theory, 49(3):682-691, 2003.
36. Zhang, X., Yu, Y., and Schuurmans, D. Accelerated training for matrix-norm regularization: a boosting approach. In Adv. NIPS, 2012.