Conditional gradient method for stochastic submodular maximization: Closing the gap

International Conference on Artificial Intelligence and Statistics, AISTATS 2018

Volumn , Issue , 2018, Pages 1886-1895

  Mokhtari, Aryan ^a   Hassani, Hamed ^b   Karbasi, Amin ^c  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b UNIVERSITY OF PENNSYLVANIA   (United States)

^c YALE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; GRADIENT METHODS; OPTIMIZATION;

CONDITIONAL GRADIENT METHOD; CONTINUOUS OPTIMIZATION; CONVEX BODY; HARDNESS RESULT; OBJECTIVE FUNCTIONS; STOCHASTIC GRADIENT; SUBMODULAR FUNCTIONS; SUBMODULAR SET FUNCTIONS;

STOCHASTIC SYSTEMS;

EID: 85057229439     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (52)

1. A. A. Abbasi and M. Younis. A survey on clustering algorithms for wireless sensor networks. Computer communications, 30(14):2826–2841, 2007.
2. A. Anandkumar and R. Ge. Efficient approaches for escaping higher order saddle points in non-convex optimization. In Proceedings of the 29th Conference on Learning Theory, pages 81–102, 2016.
3. A. Anandkumar, R. Ge, D. Hsu, S. M. Kakade, and M. Telgarsky. Tensor decompositions for learning latent variable models. Journal of Machine Learning Research, 15: 2773–2832, 2014.
4. S. Arora, R. Ge, R. Kannan, and A. Moitra. Computing a nonnegative matrix factorization - provably. SIAM J. Comput., 45(4):1582–1611, 2016.
5. F. Bach. Submodular functions: from discrete to continuous domains. arXiv preprint arXiv:1511.00394, 2015.
6. A. Badanidiyuru and J. Vondrák. Fast algorithms for maximizing submodular functions. In Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1497–1514, 2014.
7. Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle. Greedy layer-wise training of deep networks. In Advances in Neural Information Processing Systems 19, pages 153–160, 2007.
8. A. A. Bian, B. Mirzasoleiman, J. M. Buhmann, and A. Krause. Guaranteed non-convex optimization: Submodular maximization over continuous domains. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics.
9. N. Buchbinder, M. Feldman, J. Naor, and R. Schwartz. Submodular maximization with cardinality constraints. In Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1433–1452, 2014.
10. N. Buchbinder, M. Feldman, J. Naor, and R. Schwartz. A tight linear time (1/2)-approximation for unconstrained submodular maximization. SIAM J. Comput., 44(5): 1384–1402, 2015.
11. G. Calinescu, C. Chekuri, M. Pál, and J. Vondrák. Maximizing a monotone submodular function subject to a matroid constraint. SIAM Journal on Computing, 40 (6):1740–1766, 2011.
12. D. Chakrabarty, Y. T. Lee, A. Sidford, and S. C. Wong. Subquadratic submodular function minimization. In Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing, pages 1220–1231, 2017.
13. C. Chekuri, J. Vondrák, and R. Zenklusen. Submodular function maximization via the multilinear relaxation and contention resolution schemes. SIAM Journal on Computing, 43(6):1831–1879, 2014.
14. C. Chekuri, T. Jayram, and J. Vondrák. On multiplicative weight updates for concave and submodular function maximization. In Proceedings of the 2015 Conference on Innovations in Theoretical Computer Science, pages 201–210. ACM, 2015.
15. N. R. Devanur and K. Jain. Online matching with concave returns. In Proceedings of the 44th Symposium on Theory of Computing Conference, pages 137–144, 2012.
16. R. Eghbali and M. Fazel. Designing smoothing functions for improved worst-case competitive ratio in online optimization. In Advances in Neural Information Processing Systems 29, pages 3279–3287, 2016.
17. A. Ene, H. L. Nguyen, and L. A. Végh. Decomposable submodular function minimization: Discrete and continuous. In Advances in Neural Information Processing Systems 30, pages 2874–2884, 2017.
18. U. Feige. A threshold of ln n for approximating set cover. Journal of the ACM, 45(4):634–652, 1998.
19. U. Feige, V. S. Mirrokni, and J. Vondrak. Maximizing non-monotone submodular functions. SIAM Journal on Computing, 40(4):1133–1153, 2011.
20. M. Feldman, J. Naor, and R. Schwartz. A unified continuous greedy algorithm for submodular maximization. In IEEE 52nd Annual Symposium on Foundations of Computer Science, pages 570–579, 2011.
21. M. Feldman, C. Harshaw, and A. Karbasi. Greed is good: Near-optimal submodular maximization via greedy optimization. In Proceedings of the 30th Conference on Learning Theory, pages 758–784, 2017.
22. S. Fujishige. Submodular functions and optimization, volume 58. Elsevier, 2005.
23. R. Ge, F. Huang, C. Jin, and Y. Yuan. Escaping from saddle points–online stochastic gradient for tensor decomposition. In Proceedings of The 28th Conference on Learning Theory, pages 797–842, 2015.
24. R. Ge, J. D. Lee, and T. Ma. Matrix completion has no spurious local minimum. In Advances in Neural Information Processing Systems 29, pages 2973–2981, 2016.
25. S. O. Gharan and J. Vondrák. Submodular maximization by simulated annealing. In Proceedings of the Twenty-Second Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1098–1116, 2011.
26. D. Golovin, A. Krause, and M. Streeter. Online submodular maximization under a matroid constraint with application to learning assignments. arXiv preprint arXiv:1407.1082, 2014.
27. S. H. Hassani, M. Soltanolkotabi, and A. Karbasi. Gradient methods for submodular maximization. In Advances in Neural Information Processing Systems 30, pages 5843–5853, 2017.
28. E. Hazan and H. Luo. Variance-reduced and projection-free stochastic optimization. In Proceedings of the 33nd International Conference on Machine Learning, pages 1263–1271, 2016.
29. E. Hazan, K. Y. Levy, and S. Shalev-Shwartz. On graduated optimization for stochastic non-convex problems. In Proceedings of the 33nd International Conference on Machine Learning, pages 1833–1841, 2016.
30. C. Jin, R. Ge, P. Netrapalli, S. M. Kakade, and M. I. Jordan. How to escape saddle points efficiently. In Proceedings of the 34th International Conference on Machine Learning, pages 1724–1732, 2017.
31. M. R. Karimi, M. Lucic, S. H. Hassani, and A. Krause. Stochastic submodular maximization: The case of coverage functions. In Advances in Neural Information Processing Systems 30, pages 6856–6866, 2017.
32. L. Lovász. Submodular functions and convexity. In Mathematical Programming The State of the Art, pages 235–257. Springer, 1983.
33. A. Mehta, A. Saberi, U. Vazirani, and V. Vazirani. Ad-words and generalized online matching. Journal of the ACM, 54(5):22, 2007.
34. S. Mei, Y. Bai, and A. Montanari. The landscape of empirical risk for non-convex losses. arXiv preprint arXiv:1607.06534, 2016.
35. A. Mokhtari, A. Koppel, G. Scutari, and A. Ribeiro. Large-scale nonconvex stochastic optimization by doubly stochastic successive convex approximation. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 4701–4705, 2017.
36. K. G. Murty and S. N. Kabadi. Some np-complete problems in quadratic and nonlinear programming. Mathematical programming, 39(2):117–129, 1987.
37. G. L. Nemhauser, L. A. Wolsey, and M. L. Fisher. An analysis of approximations for maximizing submodular set functions–I. Mathematical Programming, 14(1):265–294, 1978.
38. P. Netrapalli, U. N. Niranjan, S. Sanghavi, A. Anandkumar, and P. Jain. Non-convex robust PCA. In Advances in Neural Information Processing Systems 27, pages 1107–1115, 2014.
39. S. Paternain, A. Mokhtari, and A. Ribeiro. A second order method for nonconvex optimization. arXiv preprint arXiv:1707.08028, 2017.
40. S. J. Reddi, S. Sra, B. Póczos, and A. Smola. Stochastic Frank-Wolfe methods for nonconvex optimization. In 54th Annual Allerton Conference on Communication, Control, and Computing, pages 1244–1251. IEEE, 2016.
41. A. Ruszczyński. Feasible direction methods for stochastic programming problems. Mathematical Programming, 19 (1):220–229, 1980.
42. A. Ruszczyński. A merit function approach to the subgradient method with averaging. Optimisation Methods and Software, 23(1):161–172, 2008.
43. J. Snoek, H. Larochelle, and R. P. Adams. Practical bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems 25, pages 2951–2959, 2012.
44. T. Soma, N. Kakimura, K. Inaba, and K. Kawarabayashi. Optimal budget allocation: Theoretical guarantee and efficient algorithm. In Proceedings of the 31th International Conference on Machine Learning, pages 351–359, 2014.
45. M. Staib and S. Jegelka. Robust budget allocation via continuous submodular functions. In Proceedings of the 34th International Conference on Machine Learning, pages 3230–3240, 2017.
46. S. Stan, M. Zadimoghaddam, A. Krause, and A. Karbasi. Probabilistic submodular maximization in sub-linear time. In Proceedings of the 34th International Conference on Machine Learning, pages 3241–3250, 2017.
47. M. Sviridenko, J. Vondrák, and J. Ward. Optimal approximation for submodular and supermodular optimization with bounded curvature. In Proceedings of the Twenty-Sixth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1134–1148, 2015.
48. J. Vondrák. Submodularity in combinatorial optimization. 2007.
49. J. Vondrák. Optimal approximation for the submodular welfare problem in the value oracle model. In Proceedings of the 40th Annual ACM Symposium on Theory of Computing, pages 67–74. ACM, 2008.
50. J. Vondrák. Submodularity and curvature: The optimal algorithm (combinatorial optimization and discrete algorithms). RIMS Kokyuroku Bessatsu, B23:253–266, 2010.
51. L. A. Wolsey. An analysis of the greedy algorithm for the submodular set covering problem. Combinatorica, 2(4): 385–393, 1982.
52. Y. Yang, G. Scutari, D. P. Palomar, and M. Pesavento. A parallel decomposition method for nonconvex stochastic multi-agent optimization problems. IEEE Transactions on Signal Processing, 64(11):2949–2964, 2016.