A stochastic PCA and SVD algorithm with an exponential convergence rate

32nd International Conference on Machine Learning, ICML 2015

Volumn 1, Issue , 2015, Pages 144-152

  Shamir, Ohad ^a  

^a WEIZMANN INSTITUTE OF SCIENCE   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; CONVEX OPTIMIZATION; LEARNING SYSTEMS; OPTIMIZATION; SINGULAR VALUE DECOMPOSITION; STOCHASTIC SYSTEMS;

DATA SIZE; EXPONENTIAL CONVERGENCE RATE; NONCONVEX PROBLEM; OPTIMAL SOLUTIONS; SIMPLE ALGORITHM; SLOW CONVERGENCES; STOCHASTIC GRADIENT TECHNIQUES; STOCHASTIC ITERATION;

PRINCIPAL COMPONENT ANALYSIS;

EID: 84969523896     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (14)

1. Arora, R., Cotter, A., Livescu, K., and Srebro, N. Stochastic optimization for PCA and PLS. In 2072 50th Annual Allerton Conference on Communication, Control, and Computing, 2012.
2. Arora, R., Cotter, A., and Srebro, N. Stochastic optimization of PCA with capped MSG. In NIPS, 2013.
3. Balsubramani, A., Dasgupta, S., and Freund, Y. The fast convergence of incremental PCA. In NIPS, 2013.
4. De Sa, C, Olukotun, K., and Re, C. Global convergence of stochastic gradient descent for some nonconvex matrix problems. arXiv preprint arXiv:1411.1134, 2014.
5. Frostig, R., Ge, R., Kakade, S., and Sidford, A. Competing with the empirical risk minimizer in a single pass. CoRR, abs/1412.6606, 2014.
6. Golub, G. and van Loan, C. Matrix computations (4. ed.). Johns Hopkins University Press, 2013.
7. Hoeffding, W. Probability inequalities for sums of bounded random variables. Journal of the American statistical association, 58(301): 13-30, 1963.
8. Johnson, R. and Zhang, T. Accelerating stochastic gradient descent using predictive variance reduction. In NIPS, 2013.
9. Konecny, J. and Richtarik, P. Semi-stochastic gradient descent methods. CoRR, abs/1312.1666, 2013.
10. Krasulina, T.P. The method of stochastic approximation for the determination of the least eigenvalue of a symmetrical matrix. USSR Computational Mathematics and Mathematical Physics, 9(6): 189-195, 1969.
11. Mahdavi, M., Zhang, L., and Jin, R. Mixed optimization for smooth functions. In NIPS, 2013.
12. Mitliagkas, I., Caramanis, C, and Jain, P. Memory limited, streaming PCA. In NIPS, 2013.
13. Oja, E. Simplified neuron model as a principal component analyzer. Journal of mathematical biology, 15(3):267-273, 1982.
14. Oja, E. and Karhunen, J. On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix. Journal of mathematical analysis and applications, 106(l):69-84, 1985.