Variance-reduced and projection-free stochastic optimization

33rd International Conference on Machine Learning, ICML 2016

Volumn 3, Issue , 2016, Pages 1926-1936

  Hazan, Elad ^a   Luo, Haipeng ^a  

^a PRINCETON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLASSIFICATION (OF INFORMATION); LEARNING SYSTEMS; STOCHASTIC SYSTEMS;

MACHINE LEARNING APPLICATIONS; OBJECTIVE FUNCTIONS; OPTIMIZATION ALGORITHMS; REAL-WORLD DATASETS; STOCHASTIC GRADIENT; STOCHASTIC LEARNING; STOCHASTIC OPTIMIZATIONS; VARIANCE REDUCTION TECHNIQUES;

OPTIMIZATION;

EID: 84998842249     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (16)

1. Deng, Jia, Dong, Wei, Socher, Richard, Li, Li-Jia, Li, Kai, and Fei-Fei, Li. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp. 248-255. IEEE, 2009.
2. Dudik, Miro, Harchaoui, Zaid, and Malick, Jérôme. Lifted coordinate descent for learning with trace-norm regularization. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, volume 22, pp. 327-336, 2012.
3. Frank, Marguerite and Wolfe, Philip. An algorithm for quadratic programming. Naval research logistics quarterly, 3(1-2):95-110, 1956.
4. Frostig, Roy, Ge, Rong, Kakade, Sham M, and Sidford, Aaron. Competing with the empirical risk minimizer in a single pass. In Proceedings of the 28th Annual Conference on Learning Theory, 2015.
5. Garber, Dan and Hazan, Elad. A linearly convergent conditional gradient algorithm with applications to online and stochastic optimization. arXiv preprint arXiv. 1301.4666, 2013.
6. Harchaoui, Zaid, Juditsky, Anatoli, and Nemirovski, Arkadi. Conditional gradient algorithms for normregularized smooth convex optimization. Mathematical Programming, 152(1-2):75-112, 2015.
7. Hazan, Elad and Kale, Satyen. Projection-free online learning. In Proceedings of the 29th International Conference on Machine Learning, 2012.
8. Hazan, Elad, Kale, Satyen, and Shalev-Shwartz, Shai. Near-optimal algorithms for online matrix prediction. In COLT 2012 - The 25th Annual Conference on Learning Theory, June 25-27, 2012, Edinburgh, Scotland, pp. 38.1-38.13, 2012.
9. Jaggi, Martin. Revisiting frank-wolfe: Projection-free sparse convex optimization. In Proceedings of the 30th International Conference on Machine Learning, pp. 427-435, 2013.
10. Johnson, Rie and Zhang, Tong. Accelerating stochastic gradient descent using predictive variance reduction. In Advances in Neural Information Processing Systems 27, pp. 315-323, 2013.
11. Lacoste-Julien, Simon and Jaggi, Martin. On the global linear convergence of frank-wolfe optimization variants. In Advances in Neural Information Processing Systems 29, pp. 496-504, 2015.
12. Lan, Guanghui and Zhou, Yi. Conditional gradient sliding for convex optimization. Optimization-Online preprint (4605), 2014.
13. Mahdavi, Mehrdad, Zhang, Lijun, and Jin, Rong. Mixed optimization for smooth functions. In Advances in Neural Information Processing Systems, pp. 674-682, 2013.
14. Moritz, Philipp, Nishihara, Robert, and Jordan, Michael I. A linearly-convergent stochastic 1-bfgs algorithm. In Proceedings of the Nineteenth International Conference on Artificial Intelligence and Statistics, 2016.
15. 2). In Soviet Mathematics Doklady, volume 27, pp. 372-376, 1983.
16. Zhang, Xinhua, Schuurmans, Dale, and Yu, Yao-liang. Accelerated training for matrix-norm regularization: A boosting approach. In Advances in Neural Information Processing Systems 26, pp. 2906-2914, 2012.