Stochastic optimization for PCA and PLS

2012 50th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2012

Volumn , Issue , 2012, Pages 861-868

  Arora, Raman ^a   Cotter, Andrew ^a   Livescu, Karen ^a   Srebro, Nathan ^a  

^a TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EMPIRICAL PERFORMANCE; STOCHASTIC APPROXIMATIONS; STOCHASTIC OPTIMIZATION PROBLEMS; STOCHASTIC OPTIMIZATIONS;

COMMUNICATION; STOCHASTIC SYSTEMS;

OPTIMIZATION;

EID: 84875700547     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/Allerton.2012.6483308     Document Type: Conference Paper

References (30)

1. L. Bottou and O. Bousquet, "The tradeoffs of large scale learning," in NIPS'07, 2007, pp. 161-168.
2. S. Shalev-Shwartz and N. Srebro, "SVM optimization: Inverse dependence on training set size," in ICML'08, 2008, pp. 928-935.
3. A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro, "Robust stochastic approximation approach to stochastic programming," SIAM Journal on Optimization, vol. 19, no. 4, pp. 1574-1609, January 2009.
4. S. Shalev-Shwartz, Y. Singer, and N. Srebro, "Pegasos: Primal Estimated sub-GrAdient SOlver for SVM," in ICML'07, 2007, pp. 807-814.
5. M. Collins, A. Globerson, T. Koo, X. Carreras, and P. L. Bartlett, "Exponentiated gradient algorithms for conditional random fields and max-margin markov networks," J. Mach. Learn. Res., vol. 9, pp. 1775-1822, Jun. 2008.
6. S. Shalev-Shwartz and A. Tewari, "Stochastic methods for l1 regularized loss minimization," in Proceedings of the 26th Annual International Conference on Machine Learning, ser. ICML'09. New York, NY, USA: ACM, 2009, pp. 929-936.
7. T. D. Sanger, "Optimal unsupervised learning in a single-layer linear feedforward neural network," Neural Networks, vol. 12, pp. 459-473, 1989.
8. M. K.Warmuth and D. Kuzmin, "Randomized PCA algorithms with regret bounds that are logarithmic in the dimension," in NIPS'06, 2006.
9. M. Gu, "Single- and multiple-vector iterations," in Templates for the solution of algebraic eigenvalue problems: a practical guide, Bai, Zhaojun, Demmel, James, Dongarra, Jack, Ruhe, Axel, and van der Vorst, Henk, Eds. Philadelphia, PA: Society for Industrial and Applied Mathematics, 2000, ch. 4.3.
10. A. Ruhe, "Lanczos method," in Templates for the solution of algebraic eigenvalue problems: a practical guide, Bai, Zhaojun, Demmel, James, Dongarra, Jack, Ruhe, Axel, and van der Vorst, Henk, Eds. Philadelphia, PA: Society for Industrial and Applied Mathematics, 2000, ch. 4.4.
11. J. A. Tropp, "User-friendly tail bounds for sums of random matrices," Foundations of Computational Math., Aug 2011.
12. E. Oja and J. Karhunen, "On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix," Journal of Mathematical Analysis and Applications, vol. 106, pp. 69-84, 1985. (Pubitemid 15476174)
13. K. I. Kim, M. O. Franz, and B. Schölkopf, "Iterative kernel principal component analysis for image modeling," IEEE Trans. PAMI, vol. 27, no. 9, pp. 1351-1366, 2005. (Pubitemid 41387769)
14. M. Brand, "Incremental singular value decomposition of uncertain data with missing values," in ECCV, 2002.
15. J. R. Bunch, C. P. Nielsen, and D. C. Sorensen, "Rankone modification of the symmetric eigenproblem," Numerische Mathematik, vol. 31, no. 1, pp. 31-48, 1978.
16. T.-J. Chin, K. Schindler, and D. Suter, "Incremental kernel SVD for face recognition with image sets," in Proc. 7th Intl. Conf. Automatic Face and Gesture Recog. (FG06), 2002.
17. B. Sarwar, G. Karypis, J. Konstan, and J. Riedl, "Incremental singular value decomposition algorithms for highly scalable recommender systems," in Fifth International Conference on Computer and Information Science, 2002.
18. Y. N. Rao and J. C. Principe, "A fast on-line generalized eigendecomposition algorithm for time series segmentation," IEEE Transactions on Signal Processing, 2000.
19. A. Beck and M. Teboulle, "Mirror descent and nonlinear projected subgradient methods for convex optimization," Operations Research Letters, vol. 31, no. 3, pp. 167-175, 2003.
20. K. Tsuda, G. Rätsch, and M. K. Warmuth, "Matrix exponentiated gradient updates for on-line learning and bregman projection," J. Mach. Learn. Res., pp. 995-1018, Dec. 2005.
21. K. Choukri, G. Chollet, and Y. Grenier, "Spectral transformations through canonical correlation analysis for speaker adptation in ASR," in ICASSP, 1986.
22. K. Chaudhuri, S. M. Kakade, K. Livescu, and K. Sridharan, "Multi-view clustering via canonical correlation analysis," in ICML'09, 2009.
23. K. Livescu and M. Stoehr, "Multi-view learning of acoustic features for speaker recognition," in ASRU, 2009.
24. F. Rudzicz, "Adaptive kernel canonical correlation analysis for estimation of task dynamics from acoustics," in ICASSP, 2010.
25. S. Bharadwaj, R. Arora, K. Livescu, and M. Hasegawa-Johnson, "Multiview acoustic feature learning using articulatory measurements," in Intl. Workshop on Stat. Machine Learning for Speech Recognition (IWSML), 2012.
26. R. Arora and K. Livescu, "Kernel CCA for multi-view learning of acoustic features using articulatory measurements," in Symp. on Machine Learning in Speech and Language Processing (MLSLP), 2012.
27. A. Haghighi, P. Liang, T. Berg-Kirkpatrick, and D. Klein, "Learning bilingual lexicons from monolingual corpora," in ACL, 2008.
28. D. R. Hardoon, S. Szedmak, and J. Shawe-Taylor, "Canonical correlation analysis: An overview with application to learning methods," Neural Computation, vol. 16, no. 12, pp. 2639-2664, 2004. (Pubitemid 39604012)
29. M. B. Blaschko and C. H. Lampert, "Correlational spectral clustering," in CVPR, 2008.
30. J. R. Westbury, X-ray microbeam speech production database user's handbook, Waisman Center on Mental Retardation & Human Development, University of Wisconsin, Madison, WI, USA, June 1994.