Online learning in the embedded manifold of low-rank matrices

Journal of Machine Learning Research

Volumn 13, Issue , 2012, Pages 429-458

  Shalit, Uri ^a,b   Weinshall, Daphna ^a   Chechik, Gal ^b,c  

^a HEBREW UNIVERSITY OF JERUSALEM   (Israel)

^b BAR ILAN UNIVERSITY   (Israel)

^c GOOGLE INC   (United States)

Author keywords

Low rank; Metric learning; Multitask learning; Online learning; Retractions; Riemannian manifolds

Indexed keywords

LOW RANK; METRIC LEARNING; MULTITASK LEARNING; ONLINE LEARNING; RETRACTIONS; RIEMANNIAN MANIFOLD;

ALGORITHMS; OPTIMIZATION;

E-LEARNING;

EID: 84857807897     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (43)

1. P.-A. Absil and J. Malick. Projection-like retractions on matrix manifolds. Technical Report UCLINMA-2010.038, Department of Mathematical Engineering, Université catholique de Louvain, July 2010.
2. P.-A. Absil, R.Mahony, and R. Sepulchre. Optimization Algorithms onMatrixManifolds. Princeton Univ Press, 2008.
3. B. Bai, J. Weston, R. Collobert, and D. Grangier. Supervised semantic indexing. Advances in Information Retrieval, pages 761-765, 2009.
4. A. Bar-Hillel, T. Hertz, N. Shental, and D. Weinshall. Learning a mahalanobis metric from equivalence constraints. Journal of Machine Learning Research, 6(1):937-965, 2006.
5. J. Briët, F.M. de Oliveira Filho, and F. Vallentin. The Grothendieck problem with rank constraint. In Proceedings of the 19th International Symposium on Mathematical Theory of Networks and Systems, MTNS, 2010.
6. G. Chechik, V. Sharma, U. Shalit, and S. Bengio. Large scale online learning of image similarity through ranking. Journal of Machine Learning Research, 11:1109-1135, 2010.
7. K. Crammer, O. Dekel, J. Keshet, S. Shalev-Shwartz, and Y. Singer. Online passive-aggressive algorithms. Journal of Machine Learning Research, 7:551-585, 2006. (Pubitemid 43668115)
8. J. Deng, W. Dong, R. Socher, L.J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of the 22nd IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, 2009.
9. J. Deng, A. Berg, and L. Fei-Fei. Hierarchical Semantic Indexing for Large Scale Image Retrieval. In Proceedings of the 24th IEEE Conference on Computer Vision and Pattern Recognition, pages 785-792, 2011.
10. M.P. Do Carmo. Riemannian Geometry. Birkhauser, 1992.
11. L. Eldén and B. Savas. A Newton-Grassmann method for computing the best multi-linear rank-(r1, r2, r3) approximation of a tensor. SIAM Journal on Matrix Analysis and applications, 31(2): 248-271, 2009.
12. M. Fazel. Matrix Rank Minimization with Applications. PhD thesis, Electrical Engineering Department, Stanford University, 2002.
13. M. Fazel, H. Hindi, and S. Boyd. Rank minimization and applications in system theory. In Proceedings of the 2004 American Control Conference, pages 3273-3278. IEEE, 2005.
14. A. Globerson and S. Roweis. Metric learning by collapsing classes. In Advances in Neural Information Processing Systems, volume 18, page 451, 2006.
15. J. Goldberger, S. Roweis, G. Hinton, and R. Salakhutdinov. Neighbourhood components analysis. In Advances in Neural Information Processing Systems, volume 17, 2005.
16. D. Grangier and S. Bengio. A discriminative kernel-based model to rank images from text queries. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30:1371-1384, 2008.
17. M. Ishteva, L. De Lathauwer, P.-A. Absil, and S. Van Huffel. Best low multilinear rank approximation of higher-order tensors, based on the Riemannian trust-region scheme. SIAM Journal on Matrix Analysis and Applications, 32(1):115-132, 2011.
18. P. Jain, B. Kulis, I.S. Dhillon, and K. Grauman. Online metric learning and fast similarity search. In Advances in Neural Information Processing Systems, volume 20, pages 761-768, 2008.
19. P. Jain, R. Meka, and I. Dhillon. Guaranteed rank minimization via singular value projection. In Advances in Neural Information Processing Systems, volume 24, pages 937-945, 2011.
20. M. Journée, F. Bach, P.-A. Absil, and R. Sepulchre. Low-Rank Optimization on the Cone of Positive Semidefinite Matrices. SIAM Journal on Optimization, 20(5):2327-2351, 2010a.
21. M. Journée, Y. Nesterov, P. Richtárik, and R. Sepulchre. Generalized power method for sparse principal component analysis. The Journal of Machine Learning Research, 11:517-553, 2010b.
22. S.M. Kakade, S. Shalev-Shwartz, and A. Tewari. Regularization techniques for learning with matrices, 2010. Arxiv preprint arXiv:0910.0610v2.
23. R.H. Keshavan, A. Montanari, and S. Oh. Matrix completion from noisy entries. The Journal of Machine Learning Research, 99:2057-2078, 2010.
24. B. Kulis,M.A. Sustik, and I.S. Dhillon. Low-rank kernel learning with bregman matrix divergences. The Journal of Machine Learning Research, 10:341-376, 2009.
25. K. Lang. Learning to filter netnews. In Proceeding of the 12th Internation Conference on Machine Learning, pages 331-339, 1995.
26. C.D. Manning, P. Raghavan, H. Schutze, and Ebooks Corporation. Introduction to Information Retrieval, volume 1. Cambridge University Press Cambridge, UK, 2008.
27. R. Meka, P. Jain, C. Caramanis, and I.S. Dhillon. Rank minimization via online learning. In Proceedings of the 25th International Conference on Machine learning, pages 656-663, 2008.
28. C.D. Meyer. Generalized inversion of modified matrices. SIAM Journal on Applied Mathematics, 24(3):315-323, 1973.
29. G. Meyer, S. Bonnabel, and R. Sepulchre. Regression on fixed-rank positive semidefinite matrices: a Riemannian approach. The Journal of Machine Learning Research, 12:593-625, 2011.
30. B.K. Natarajan. Sparse approximate solutions to linear systems. SIAM Journal on Computing, 24 (2):227-234, 1995.
31. Sahand Negahban and Martin J. Wainwright. Estimation of (near) low-rank matrices with noise and high-dimensional scaling. In Proceedings of the 27th International Conference on Machine Learning, pages 823-830, 2010.
32. C. Oberlin and S.J. Wright. Active set identification in nonlinear programming. SIAM Journal on Optimization, 17(2):577-605, 2007.
33. K.B. Petersen and M.S. Pedersen. The matrix cookbook, Oct. 2008.
34. B. Recht,M. Fazel, and P.A. Parrilo. Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Review, 52(3):471-501, 2010.
35. S. Shalev-Shwartz, Y. Singer, and A.Y. Ng. Online and batch learning of pseudo-metrics. In Proceedings of the Twenty-first International Conference on Machine Learning, page 94. ACM, 2004.
36. Uri Shalit, Daphna Weinshall, and Gal Chechik. Online learning in the manifold of low-rank matrices. In Advances in Neural Information Processing Systems 23, pages 2128-2136. MIT Press, 2010.
37. B. Vandereycken and S. Vandewalle. A Riemannian optimization approach for computing low-rank solutions of Lyapunov equations. SIAM Journal on Matrix Analysis and Applications, 31(5): 2553-2579, 2010.
38. B. Vandereycken, P.-A. Absil, and S. Vandewalle. Embedded geometry of the set of symmetric positive semidefinite matrices of fixed rank. In Statistical Signal Processing, 2009. SSP'09. IEEE/SP 15th Workshop on, pages 389-392. IEEE, 2009.
39. K.Q. Weinberger and L.K. Saul. Distance metric learning for large margin nearest neighbor classification. The Journal of Machine Learning Research, 10:207-244, 2009.
40. J. Weston, S. Bengio, and N. Usunier. Wsabie: Scaling up to large vocabulary image annotation. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI-11), 2011.
41. E.P. Xing, A.Y. Ng, M.I. Jordan, and S. Russell. Distance metric learning, with application to clustering with side-information. In Advances in Neural Information Processing Systems, volume 15, pages 505-512. MIT Press, 2002.
42. L. Yang. An overview of distance metric learning. Technical report, School of Computer Science, Carnegie Mellon University, 2007.
43. Y. Yang and J.O. Pedersen. A comparative study on feature selection in text categorization. In Proceedings of the 14th International Conference on Machine Learning, pages 412-420, 1997.