Spectral regularization algorithms for learning large incomplete matrices

Journal of Machine Learning Research

Volumn 11, Issue , 2010, Pages 2287-2322

  Mazumder, Rahul ^a   Hastie, Trevor ^a   Tibshirani, Robert ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

Collaborative filtering; Large scale convex optimization; Netflix prize; Nuclear norm; Spectral regularization

Indexed keywords

COLLABORATIVE FILTERING; LARGE-SCALE CONVEX OPTIMIZATION; NETFLIX PRIZE; NUCLEAR NORM; SPECTRAL REGULARIZATION;

APPROXIMATION ALGORITHMS; OPTIMIZATION; RELAXATION PROCESSES; SIGNAL FILTERING AND PREDICTION; SINGULAR VALUE DECOMPOSITION;

CONVEX OPTIMIZATION;

EID: 77956944781     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (39)

1. J. Abernethy, F. Bach, T. Evgeniou, and J.-P. Vert. A new approach to collaborative filtering: operator estimation with spectral regularization. Journal of Machine Learning Research, 10:803-826, 2009.
2. A. Argyriou, T. Evgeniou, and M. Pontil. Multi-task feature learning. In Advances in Neural Information Processing Systems 19. MIT Press, 2007.
3. A. Argyriou, T. Evgeniou, and M. Pontil. Convex multi-task feature learning. Machine Learning, 73(3):243-272, 2008.
4. F. Bach. Consistency of trace norm minimization. Journal of Machine Learning Research, 9:1019-1048, 2008.
5. R. M. Bell and Y. Koren. Lessons from the Netflix prize challenge. Technical report, AT&T Bell Laboratories, 2007.
6. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
7. S. Burer and R. D. C. Monteiro. Local minima and convergence in low-rank semidefinite programming. Mathematical Programming, 103(3):427-631, 2005.
8. J. Cai, E. J. Candes, and Z. Shen. A singular value thresholding algorithm for matrix completion, 2008. Available at http://www.citebase.org/ abstract?id=oai:arXiv.org:0810.3286.
9. E. Candes and B. Recht. Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 9:717-772, 2008.
10. E. J. Candes and T. Tao. The power of convex relaxation: near-optimal matrix completion. IEEE Transactions on Information Theory, 56(5):2053-2080, 2009.
11. D. DeCoste. Collaborative prediction using ensembles of maximum margin matrix factorizations. In Proceedings of the 23rd International Conference on Machine Learning, pages 249-256. ACM, 2006.
12. A. Dempster, N. Laird, and D. Rubin. Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society Series B, 39:1-38, 1977.
13. D. Donoho, I. Johnstone, G. Kerkyachairan, and D. Picard. Wavelet shrinkage; asymptopia? (with discussion). Journal of the Royal Statistical Society: Series B, 57:201-337, 1995.
14. M. Fazel. Matrix Rank Minimization with Applications. PhD thesis, Stanford University, 2002.
15. J. Friedman. Fast sparse regression and classification. Technical report, Department of Statistics, Stanford University, 2008.
16. J. Friedman, T. Hastie, H. Hoefling, and R. Tibshirani. Pathwise coordinate optimization. Annals of Applied Statistics, 2(1):302-332, 2007.
17. M. Grant and S. Boyd. CVX: Matlab software for disciplined convex programming, 2009. Web page and software available at http://Stanford.edu/~boyd/ cvx.
18. T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Prediction, Inference and Data Mining (Second Edition). Springer Verlag, New York, 2009.
19. S. Ji and J. Ye. An accelerated gradient method for trace norm minimization. In Proceedings of the 26th International Conference on Machine Learning, pages 457-464, 2009.
20. R. H. Keshavan, S. Oh, and A. Montanari. Matrix completion from a few entries. IEEE Transactions on Information Theory, 56(6):2980-2998, 2009.
21. R. M. Larsen. Lanczos bidiagonalization with partial reorthogonalization. Technical Report DAIMI PB-357, Department of Computer Science, Aarhus University, 1998.
22. R. M. Larsen. Propack-software for large and sparse svd calculations, 2004. Available at http://sun.Stanford.edu/~rmunk/PROPACK.
23. J. Liu, S. Ji, and J. Ye. SLEP: Sparse Learning with Efficient Projections. Arizona State University, 2009. Available at http://www.public.asu. edu/~jye02/Software/SLEP.
24. Z. Liu and L. Vandenberghe. Interior-point method for nuclear norm approximation with application to system identfication. SIAM Journal on Matrix Analysis and Applications, 31(3):1235-1256, 2009.
25. S. Ma, D. Goldfarb, and L. Chen. Fixed point and Bregman iterative methods for matrix rank minimization. Mathematical Programming Series A, forthcoming.
26. R. Mazumder, J. Friedman, and T. Hastie. Sparsenet: coordinate descent with non-convex penalties. Technical report, Stanford University, 2009.
27. Y. Nesterov. Introductory Lectures on Convex Optimization: Basic course. Kluwer, Boston, 2003.
28. Y. Nesterov. Gradient methods for minimizing composite objective function. Technical Report 76, Center for Operations Research and Econometrics (CORE), Catholic University of Louvain, 2007.
29. B. Recht, M. Fazel, and P. A. Parrilo. Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, 2007. Available at http://www.citebase.org/abstract?id=oai:arXiv.org:0706.4138.
30. J. Rennie and N. Srebro. Fast maximum margin matrix factorization for collaborative prediction. In Proceedings of the 22nd International Conference on Machine Learning, pages 713-719. ACM, 2005.
31. R. Salakhutdinov, A. Mnih, and G. E. Hinton. Restricted Boltzmann machines for collaborative filtering. In Proceedings of the 24th International Conference on Machine Learning, pages 791-798. AAAI Press, 2007.
32. ACM SIGKDD and Netfiix. Soft modelling by latent variables: the nonlinear iterative partial least squares (NIPALS) approach. In Proceedings of KDD Cup and Workshop, 2007. Available at http://www.cs.uic.edu/~liub/KDD-cup-2007/ proceedings.html.
33. N. Srebro and T. Jaakkola. Weighted low-rank approximations. In Proceedings of the 20th International Conference on Machine Learning, pages 720-727'. AAAI Press, 2003.
34. N. Srebro, N. Alon, and T. Jaakkola. Generalization error bounds for collaborative prediction with low-rank matrices. In Advances in Neural Information Processing Systems 17, pages 5-27. MIT Press, 2005a.
35. N. Srebro, J. Rennie, and T. Jaakkola. Maximum-margin matrix factorization. In Advances in Neural Information Processing Systems 17, pages 1329-1336. MIT Press, 2005b.
36. G. Takacs, I. Pilaszy, B. Nemeth, and D. Tikk. Scalable collaborative filtering approaches for large recommender systems. Journal of Machine Learning Research, 10:623-656, 2009.
37. R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 58:267-288, 1996.
38. O. Troyanskaya, M. Cantor, G. Sherlock, P. Brown, T. Hastie, R. Tibshirani, D. Botstein, and R. B. Altman. Missing value estimation methods for DNA microarrays. Bioinformatics, 17(6):520-525, 2001.
39. C. H. Zhang. Nearly unbiased variable selection under minimax concave penalty. Annals of Statistics, 38(2):894-942, 2010.