A fast augmented lagrangian algorithm for learning low-rank matrices

ICML 2010 - Proceedings, 27th International Conference on Machine Learning

Volumn , Issue , 2010, Pages 1087-1094

  Tomioka, Ryota ^a   Suzuki, Taiji ^a   Sugiyama, Masashi ^b   Kashima, Hisashi ^a  

^a UNIVERSITY OF TOKYO   (Japan)

^b TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

AUGMENTED LAGRANGIAN ALGORITHM; EFFICIENT ALGORITHM; GENERAL SPECTRAL; LOW-RANK MATRICES; MATRIX; MULTIPLE MATRICES; ORDERS OF MAGNITUDE; PROJECTED GRADIENT; SPEED-UPS;

BRAIN COMPUTER INTERFACE; GRADIENT METHODS; INTERFACES (COMPUTER); LAGRANGE MULTIPLIERS; LEARNING SYSTEMS;

LEARNING ALGORITHMS;

EID: 77956529188     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (31)

1. Abernethy, J., Bach, F., Evgeniou, T., and Vert, J. P. A new approach to collaborative filtering: Operator estimation with spectral regularization. The Journal of Machine Learning Research, 10:803-826, 2009.
2. Amit, Y., Fink, M., Srebro, N., and Ullman, S. Uncovering Shared Structures in Multiclass Classification. In Proc. ICML '07, pp. 17-24, New York, NY, USA, 2007. ACM Press.
3. Argyriou, A., Micchelli, C.A., and Pontil, M. Learning convex combinations of continuously parameterized basic kernels. In Auer, P. and Meir, R. (eds.), Proc. COLT2005. Springer, Berlin, Heidelberg, 2005.
4. Argyriou, A., Evgeniou, T., and Pontil, M. Multi-task feature learning. In Schölkopf, B., Platt, J., and Hoffman, T. (eds.), Advances in NIPS 19, pp. 41-48. MIT Press, Cambridge, MA, 2007.
5. Argyriou, A., Micchelli, C. A., Pontil, M., and Ying, Y. A spectral regularization framework for multi-task structure learning. In Platt, J.C., Koller, D., Singer, Y., and Roweis, S. (eds.), Advances in NIPS 20, pp. 25-32. MIT Press, Cambridge, MA, 2008.
6. Bertsekas, D. P. Constrained Optimization and Lagrange Multiplier Methods. Academic Press, 1982.
7. Blankertz, B., Müller, K.-R., Curio, G., Vaughan, T. M., Schalk, G., Wolpaw, J. R., Schlögl, A., Neuper, C, Pfurtscheller, G., Hinterberger, T., Schröder, M., and Birbaumer, N. The BCI competition 2003: Progress and perspectives in detection and discrimination of EEG single trials. IEEE Trans. Biomed. Eng., 51(6):1044-1051, 2004.
8. Cai, J.-F., Candes, E. J., and Shen, Z. A singular value thresholding algorithm for matrix completion. arXiv:0810.3286, 2008.
9. Candes, E. J. and Recht, B. Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 9(6):717-772, 2009.
10. Chen, S., Donoho, D., and Saunders, M. Atomic decomposition by basis pursuit. SIAM J. Sci. Comput., 20(1): 33-61, 1998.
11. Cortes, C. Can learning kernels help performance? Invited talk at International Conference on Machine Learning (ICML 2009). Montréal, Canada, 2009.
12. Fazel, M., Hindi, H., and Boyd, S. P. A Rank Minimization Heuristic with Application to Minimum Order System Approximation. In Proc. of the American Control Conference, 2001.
13. Figueiredo, M. A. T., Bioucas-Dias, J. M., and Nowak, R. D. Majorization-Minimization Algorithm for Wavelet-Based Image Restoration. IEEE Trans. Image Process., 16(12), 2007.
14. Hestenes, M. R. Multiplier and gradient methods. J. Op-tim. Theory Appl., 4:303-320, 1969.
15. Ji, S. and Ye, J. An accelerated gradient method for trace norm minimization. In Proc. ICML '09, pp. 457-464, New York, NY, 2009. ACM.
16. Kloft, M., Brefeld, U., Sonnenburg, S., Laskov, P., Müller, K.-R., and Zien, A. Efficient and accurate lp-norm mul-tiple kernel learning. In Advances in NIPS 22. 2010.
17. Lanckriet, G. R. G., Cristianini, N., Bartlett, P., El Ghaoui, L., and Jordan, M. I. Learning the Kernel Matrix with Semidefinite Programming. J. Machine Learning Research, 5:27-72, 2004.
18. Micchelli, C.A. and Pontil, M. Learning the kernel function via regularization. Journal of Machine Learning Research, 6:1099-1125, 2005.
19. Nocedal, J. and Wright, S. Numerical Optimization. Springer, 2nd edition, 2006.
20. Powell, M. J. D. A method for nonlinear constraints in minimization problems. In Fletcher, R. (ed.), Optimization, pp. 283-298. Academic Press, London, New York, 1969.
21. Rockafellar, R. T. Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. of Oper. Res., 1:97-116, 1976.
22. Simon, H. D. The lanczos algorithm with partial reorthogonalization. Math. Comput., 42(165):115-142, 1984.
23. Srebro, N., Rennie, J. D. M., and Jaakkola, T. S. Maximum-Margin Matrix Factorization. In Advances in NIPS 17, pp. 1329-1336. MIT Press, Cambridge, MA, 2005.
24. Tibshirani, R. Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. B, 58(1):267-288, 1996.
25. Tomioka, R. and Aihara, K. Classifying Matrices with a Spectral Regularization. In Proc. of the 24th international conference on Machine learning, pp. 895-902. ACM Press, 2007.
26. Tomioka, R. and Müller, K.-R. A regularized discriminative framework for EEG analysis with application to brain-computer interface. Neuroimage, 49(1):415-432, 2010.
27. Tomioka, R. and Sugiyama, M. Sparse learning with duality gap guarantee. In NIPS workshop OPT 2008 Optimization for Machine Learning, 2008.
28. Tomioka, R. and Sugiyama, M. Dual Augmented Lagrangian Method for Efficient Sparse Reconstruction. IEEE Signal Processing Letters, 16(12):1067-1070, 2009.
29. Tomioka, R., Suzuki, T., and Sugiyama, M. Super-Linear Convergence of Dual Augmented Lagrangian Algorithm for Sparse Learning, 2009. arXiv:0911.4046 [stat.ML].
30. Weimer, M., Karatzoglou, A., Le, Q., and Smola, A. Cofi rank - maximum margin matrix factorization for collab-orative ranking. In Platt, J.C., Koller, D., Singer, Y., and Roweis, S. (eds.), Advances in NIPS 20, pp. 1593-1600. MIT Press, Cambridge, MA, 2008.
31. Wipf, D. and Nagarajan, S. A new view of automatic relevance determination. In Advances in NIPS 20, pp. 1625-1632. 2008.