SDN A: Stochastic dual Newton ascent for empirical risk minimization

33rd International Conference on Machine Learning, ICML 2016

Volumn 4, Issue , 2016, Pages 2707-2725

  Qu, Zheng ^a   Richtárik, Peter ^b   Takáč, Martin ^c   Fereoq, Olivier ^d  

^a UNIVERSITY OF HONG KONG   (Hong Kong)

^b UNIVERSITY OF EDINBURGH   (United Kingdom)

^c Lehigh University   (United States)

^d UNIVERSITÉ PARIS SACLAY   (France)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; ITERATIVE METHODS; LEARNING SYSTEMS;

COORDINATE ASCENT; EMPIRICAL RISK MINIMIZATION; LOCAL CURVATURE; ORDERS OF MAGNITUDE; QUADRATIC FUNCTION; QUADRATIC MAXIMIZATIONS; RANDOM SUBSETS; THEORY AND PRACTICE;

STOCHASTIC SYSTEMS;

EID: 84998772498     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (34)

1. Bordes, Antoine, Bottou, Leon, and Gallinari, Patrick. SGD-QN: Careful quasi-Newton stochastic gradient descent. JMLR, 10:1737-1754, 2009.
2. Byrd, R.H., Hansen, S.L., Nocedal, Jorge, and Singer, Yoram. A stochastic quasi-Newton method for large- scale optimization. arXiv: 1401.7020, 2014.
3. Defazio, Aaron, Bach, Francis, and Lacoste-Julien, Simon. SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives. In Advances in Neural Information Processing Systems, pp. 1646-1654, 2014.
4. Fercoq, Olivier and Richtarik, Peter. Smooth minimization of nonsmooth functions by parallel coordinate descent. arXiv: 1309.5885, 2013.
5. Fercoq, Olivier and Richtarik, Peter. Accelerated, parallel, and proximal coordinate descent. SIAM Journal on Optimization, 25(4): 1997-2023, 2015.
6. Fercoq, Olivier, Qu, Zheng, Richtarik, Peter, and Takac, Martin. Fast distributed coordinate descent for minimizing non-strongly convex losses. IEEE International Workshop on Machine Learning for Signal Processing, 2014.
7. Fountoulakis, Kimon and Tappenden, Rachael. A flexible coordinate descent method for big data applications. arXiv preprint arXiv:1507.03713, 2015.
8. Johnson, Rie and Zhang, Tong. Accelerating stochastic gradient descent using predictive variance reduction. In NIPS, 2013.
9. Konecny, Jakub and Richtarik, Peter. S2GD: Semi- stochastic gradient descent methods. arXiv: 1312.1666, 2014.
10. Konecny, Jakub, Lu, Jie, Richtarik, Peter, and Takac, Martin. mS2GD: Mini-batch semi-stochastic gradient descent in the proximal setting. arXiv:1410.4744, 2014a.
11. Konecny, Jakub, Qu, Zheng, and Richtarik, Peter. Semi- stochastic coordinate descent. arXiv:1412.6293, 2014b.
12. Lin, Qihang, Lu, Zhaosong, and Xiao, Lin. An accelerated proximal coordinate gradient method and its application to regularized empirical risk minimization. Technical Report MSR-TR-2014-94, Microsoft Research, July 2014.
13. Mairal, Julien. Incremental majorization-minimization optimization with application to large-scale machine learning. SIAM Journal on Optimization, 25(2):829-855, 2015.
14. Pilanci, Mert and Wainwright, Martin J. Iterative Hessian sketch: fast and accurate solution approximation for constrained least-squares. arXiv:1411.0347, 2014.
15. Qu, Zheng and Richtarik, Peter. Coordinate descent with arbitrary sampling I: Algorithms and complexity. arXiv:1412.8060, 2014a.
16. Qu, Zheng and Richtarik, Peter. Coordinate descent methods with arbitrary sampling II: Expected separable over- Approximation. arXiv: 1412.8063, 2014b.
17. Qu, Zheng, Richtarik, Peter, and Zhang, Tong. Randomized dual coordinate ascent with arbitrary sampling. arXiv:1411.5873, 2014.
18. Richtarik, Peter and Takac, Martin. Distributed coordinate descent method for learning with big data. arXiv: 1310.2059, 2013.
19. Richtarik, Peter and Takac, Martin. On optimal probabilities in stochastic coordinate descent methods. Optimization Letters, pp. 1-11, 2015.
20. Richtarik, Peter and Takac, Martin. Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function. Mathematical Programming, 144(2): 1-38, 2014.
21. Richtarik, Peter and Takac, Martin. Parallel coordinate descent methods for big data optimization. Mathematical Programming, pp. 1-52, 2015. ISSN 0025-5610. doi:10.1007/s10107-015-0901-6.
22. Schmidt, Mark, Le Roux, Nicolas, and Bach, Francis. Minimizing finite sums with the stochastic average gradient. arXiv: 1309.2388, 2013.
23. Schraudolph, Nicol N., Yu, Jin, and Giinter, Simon. A stochastic quasi-Newton method for online convex optimization. In AISTATS, pp. 433-440, 2007.
24. Shalev-Shwartz, Shai and Ben-David, Shai. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014.
25. Shalev-Shwartz, Shai and Zhang, Tong. Accelerated mini- batch stochastic dual coordinate ascent. In NIPS, pp. 378-385, 2013a.
26. Shalev-Shwartz, Shai and Zhang, Tong. Stochastic dual coordinate ascent methods for regularized loss. JMLR, 14(1):567-599, 2013b.
27. Shalev-Shwartz, Shai, Singer, Yoram, Srebro, Nati, and Cotter, Andrew. Pegasos: Primal estimated sub-gradient solver for SVM. Mathematical Programming, pp. 3-30, 2011.
28. Sohl-Dickstein, Jascha, Poole, Ben, and Ganguli, Surya. Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods. In ICML, 2014.
29. Takac, Martin, Bijral, Avleen, Richtarik, Peter, and Srebro, Nathan. Mini-batch primal and dual methods for SVMs. In ICML, 2013.
30. Tappenden, Rachael, Richtarik, Peter, and Gondzio, Jacek. Inexact block coordinate descent method: complexity and preconditioning. arXiv: 1304.5530, 2013.
31. Tappenden, Rachael, Richtarik, Peter, and Biike, Burak. Separable approximations and decomposition methods for the augmented Lagrangian. Optimization Methods and Software, 2014.
32. Tseng, Paul and Yun, Sangwoon. A coordinate gradient descent method for nonsmooth separable minimization. Mathematical Programming, 117(1-2):387-423, 2009.
33. Xiao, Lin and Zhang, Tong. A proximal stochastic gradient method with progressive variance reduction. SIAM Journal on Optimization, 24(4):2057-2075, 2014.
34. Zhao, Peilin and Zhang, Tong. Stochastic optimization with importance sampling. arXiv: 1401.2753, 2014.