Revisiting Frank-Wolfe: Projection-free sparse convex optimization

30th International Conference on Machine Learning, ICML 2013

Volumn , Issue PART 1, 2013, Pages 427-435

  Jaggi, Martin ^a  

^a CENTRE DE MATHÉMATIQUES APPLIQUÉES   (France)

Author keywords

[No Author keywords available]

Indexed keywords

CONVEX OPTIMIZATION; ITERATIVE METHODS; LEARNING SYSTEMS;

BROAD APPLICATION; CONDITIONAL GRADIENT; CONSTRAINED CONVEX OPTIMIZATIONS; CONVERGENCE RESULTS; GREEDY METHOD; LOW-RANK MATRICES; MATRIX FACTORIZATIONS; PERMUTATION MATRIX;

OPTIMIZATION;

EID: 84897524603     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (52)

1. Alon, N and Naor, A. Approximating the Cut-Norm via Grothendieck's Inequality. SIAM J. Computing, 2006.
2. Arora, S, Hazan, E, and Kale, S. Fast algorithms for approximate semidefinite programming using the multiplicative weights update method. FOCS, 2005.
3. Bach, F. Learning with Submodular Functions: A Convex Optimization Perspective. 2011.
4. Bach, F, Mairal, J, and Ponce, J. Convex Sparse Matrix Factorizations. Technical report, 2008.
5. Bach, F, Lacoste-Julien, S, and Obozinski, G. On the Equivalence between Herding and Conditional Gradient Algorithms. In ICML, 2012.
6. Boyd, S and Vandenberghe, L. Convex optimization. 2004.
7. Cai, J-F, Candes, E J, and Shen, Z. A Singular Value Thresholding Algorithm for Matrix Completion. SIAM Journal on Optimization, 20(4):1956-1982, 2010.
8. Canon, M D and Cullum, C D. A Tight Upper Bound on the Rate of Convergence of Frank-Wolfe Algorithm. SIAM Journal on Control, 6(4):509-516, 1968.
9. Chandrasekaran, V, Recht, B, Parrilo, P A, and Willsky, A S. The Convex Geometry of Linear Inverse Problems. Found. Comp. Math., 12(6):805-849, 2012.
10. Clarkson, K L. Coresets, Sparse Greedy Approximation, and the Frank-Wolfe Algorithm. ACM Transactions on Algorithms, 6(4), 2010.
11. Demyanov, V F and Rubinov, A M. Approximate methods in optimization problems. Elsevier, 1970.
12. Dudik, M, Harchaoui, Z, and Malick, J. Lifted coordinate descent for learning with trace-norm regularization. In AISTATS, 2012.
13. Dunn, J C and Harshbarger, S. Conditional gradient algorithms with open loop step size rules. Journal of Mathematical Analysis and Applications, 62(2):432-444, 1978.
14. Edmonds, J. Submodular Functions, Matroids, and Certain Polyhedra. In Comb. Struct. and Appl., 69-87, 1970.
15. Frank, M and Wolfe, P. An algorithm for quadratic programming. Naval Res. Logis. Quart., 3:95-110, 1956.
16. Gärtner, B and Jaggi, M. Coresets for polytope distance. ACM SCG, 2009.
17. Giesen, J, Jaggi, M, and Laue, S. Regularization Paths with Guarantees for Convex Semidefinite Optimization. AISTATS, 2012.
18. Goemans, M and Williamson, D. Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J. ACM, 42(6), 1995.
19. GuéLat, J and Marcotte, P. Some comments on Wolfe's 'away step'. Mathematical Programming, 35(1), 1986.
20. Harchaoui, Z, Juditsky, A, and Nemirovski, A. Conditional gradient algorithms for machine learning. In NIPS Workshop on Optimization for ML, December 2012.
21. Hazan, E. Sparse Approximate Solutions to Semidefinite Programs. In LATIN, pp. 306-316, 2008.
22. Hazan, E and Kale, S. Projection-free Online Learning. In ICML, 2012.
23. Hazan, E, Kale, S, and Warmuth, M.K. Learning rotations with little regret. In COLT, pp. 144-154, 2010.
24. Jaggi, M. Sparse Convex Optimization Methods for Machine Learning. PhD thesis, ETH Zürich, 2011.
25. Jaggi, M and Sulovský, M. A Simple Algorithm for Nuclear Norm Regularized Problems. ICML, 2010.
26. Jenatton, R, Audibert, J-Y, and Bach, F. Structured Variable Selection with Sparsity-Inducing Norms. JMLR, 12:2777-2824, 2011.
27. Jones, L K. A Simple Lemma on Greedy Approximation in Hilbert Space and Convergence Rates for Projection Pursuit Regression and Neural Network Training. The Annals of Statistics, 20(1):608-613, 1992.
28. Kuczyński, J and Woźniakowski, H. Estimating the Largest Eigenvalue by the Power and Lanczos Algorithms with a Random Start. SIAM Journal on Matrix Analysis and Applications, 13(4):1094-1122, 1992.
29. Lacoste-Julien, S, Jaggi, M, Schmidt, M, and Pletscher, P. Block-Coordinate Frank-Wolfe Optimization for Structural SVMs. In ICML, 2013.
30. Lee, J, Recht, B, Salakhutdinov, R, Srebro, N, and Tropp, J A. Practical Large-Scale Optimization for Max-Norm Regularization. NIPS, 2010.
31. Levitin, E S and Polyak, B T. Constrained minimization methods. USSR Comp. Math. & M. Phys., 6(5), 1966.
32. Li, J and Barron, A. Mixture density estimat. NIPS, 2000.
33. Lovász, L. Submodular functions and convexity. Mathematical programming: the state of the art, 1983.
34. Lovász, L and Plummer, M D. Matching Theory. American Mathematical Society, 2009.
35. Mallat, S G and Zhang, Z. Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing, 41(12):3397-3415, 1993.
36. Murty, K G and Kabadi, S N. Some NP-complete problems in quadratic and nonlinear programming. Mathematical Programming, 39(2):117-129, 1987.
37. Nesterov, Y. Introductory Lectures on Convex Optimization. A Basic Course. Kluwer, 2004.
38. Obozinski, G, Jacob, L, and Vert, JP. Group Lasso with Overlaps: the Latent Group Lasso approach. arXiv, 2011.
39. Orabona, F, Argyriou, A, and Srebro, N. PRISMA: PRoximal Iterative SMoothing Algorithm. arXiv.org, 2012.
40. Ouyang, H. and Gray, A. Fast Stochastic Frank-Wolfe Algorithms for Nonlinear SVMs. SDM, 2010.
41. Patriksson, M. Partial linearization methods in nonlinear programming. Journal of Optimization Theory and Applications, 78(2):227-246, 1993.
42. Rockafellar, R T. Convex analysis. 1997.
43. Shalev-Shwartz, S, Srebro, N, and Zhang, T. Trading Accuracy for Sparsity in Optimization Problems with Sparsity Constraints. SIAM J. on Optimization, 20, 2010.
44. Srebro, N and Shraibman, A. Rank, Trace-Norm and Max-Norm. In COLT, 545-560, 2005.
45. Temlyakov, V N. Greedy approximation in convex optimization. arXiv.org, stat.ML, 2012.
46. Tewari, A, Ravikumar, P, and Dhillon, I S. Greedy Algorithms for Structurally Constrained High Dimensional Problems. In NIPS, 2011.
47. Tibshirani, R. Regression Shrinkage and Selection via the Lasso. J. Royal Statistical Society. Series B, 1996.
48. Tropp, J A and Gilbert, A. Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit. IEEE Trans. on Information Theory, 53(12):4655-4666, 2007.
49. Yuan, M and Lin, Y. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B, 68(1):49-67, 2006. (Pubitemid 43415335)
50. Yuan, X-T and Yan, S. Forward Basis Selection for Sparse Approximation over Dictionary. In AISTATS, 2012.
51. Zhang, T. Sequential greedy approximation for certain convex optimization problems. IEEE Transactions on Information Theory, 49(3):682-691, 2003.
52. Zhang, X, Yu, Y, and Schuurmans, D. Accelerated Training for Matrix-norm Regularization: A Boosting Approach. In NIPS, 2012.