Un-regularizing: Approximate proximal point and faster stochastic algorithms for empirical risk minimization

32nd International Conference on Machine Learning, ICML 2015

Volumn 3, Issue , 2015, Pages 2530-2538

  Frostig, Roy ^a   Ge, Rong ^b   Kakade, Sham M ^b   Sidford, Aaron ^c  

^a STANFORD UNIVERSITY   (United States)

^b MICROSOFT RESEARCH   (United States)

^c MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMPUTATION THEORY; FUNCTIONS; LEARNING SYSTEMS; STOCHASTIC SYSTEMS;

CONVEX FUNCTIONS; CONVEX REGULARIZATIONS; EMPIRICAL RISK MINIMIZATION; LINEAR LEAST SQUARES; PROXIMAL POINT ALGORITHM; PROXIMAL POINTS; RUNNING TIME; STOCHASTIC ALGORITHMS;

ALGORITHMS;

EID: 84970024762     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. Bottou, L. and Bousquet, O. The tradeoffs of large scale learning. In Advances in Neural Information Processing Systems (NIPS), 2008.
2. Cohen, M. B., Lee, Y. T., Musco, C., Musco, C., Peng, R., and Sidford, A. Uniform sampling for matrix approximation. In Innovations in Theoretical Computer Science (ITCS), 2015.
3. Defazio, A., Bach, F., and Lacoste-Julien, S. Saga: A fast incremental gradient method with support for non-strongly convex composite objectives. In Advances in Neural Information Processing Systems (NIPS), 2014.
4. Gtiler, O. New proximal point algorithms for convex minimization. SIAM Journal on Optimization, 2(4):649-664, 1992.
5. Johnson, R. and Zhang, T. Accelerating stochastic gradient descent using predictive variance reduction. In Advances in Neural Information Processing Systems (NIPS), 2013.
6. Lee, Y. T. and Sidford, A. Efficient accelerated coordinate descent methods and faster algorithms for solving linear systems. In Foundations of Computer Science (FOCS), 2013.
7. Li, M., Miller, G. L., and Peng, R. Iterative row sampling. In Foundations of Computer Science (FOCS), 2013.
8. Lin, Q., Lu, Z., and Xiao, L. An accelerated proximal coordinate gradient method. In Advances in Neural Information Processing Systems (NIPS), 2014.
9. Needell, D., Srebro, N., and Ward, R. Stochastic gradient descent, weighted sampling, and the randomized kacz-marz algorithm. In Advances in Neural Information Processing Systems (NIPS), 2014.
10. Nelson, J. and Nguyen, H. L. Osnap: Faster numerical linear algebra algorithms via sparser subspace embeddings. In Foundations of Computer Science (FOCS), 2013.
11. 2). Soviet Mathematics Doklady, 27(2):372-376, 1983.
12. Nesterov, Y. Introductory Lectures on Convex Optimization: A Basic Course. Springer, 2004.
13. Parikh, N. and Boyd, S. Proximal algorithms. Foundations and Trends in Optimization, 1(3):123-231, 2014.
14. Rahimi, A. and Recht, B. Random features for large-scale kernel machines. In Advances in Neural Information Processing Systems (NIPS), 2007.
15. Rockafellar, R. T. Monotone operators and the proximal point algorithm. SIAM Journal on Control and Optimization, 14(5):877-898, 1976.
16. Roux, N. L., Schmidt, M., and Bach, F. A stochastic gradient method with an exponential convergence rate for finite training sets. In Advances in Neural Information Processing Systems (NIPS), 2012.
17. Shalev-Shwartz, S. and Zhang, T. Stochastic dual coordinate ascent methods for regularized loss minimization. Journal of Machine Learning Research (JMLR), 14:567-599, 2013.
18. Shalev-Shwartz, S. and Zhang, T. Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization. Mathematical Programming, pp. 1-41, 2014.
19. Strohmer, T. and Vershynin, R. A randomized kaczmarz algorithm with exponential convergence. Journal of Fourier Analysis and Applications, 15:262-278, 2009.
20. Williams, V. V. Multiplying matrices faster than Coppersmith-Winograd. In Symposium on Theory of Computing (STOC), 2012.
21. Xiao, L. and Zhang, T. A proximal stochastic gradient method with progressive variance reduction. SIAM Journal on Optimization, 24(4):2057-2075, 2014.