Lifted coordinate descent for learning with trace-norm regularization

Journal of Machine Learning Research

Volumn 22, Issue , 2012, Pages 327-336

  Dudík, Miroslav ^a   Harchaoui, Zaid ^b   Malick, Jérôme ^b  

^a YAHOO RESEARCH   (United States)

^b UNIV GRENOBLE ALPES   (France)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE;

CONVEX PROBLEMS; COORDINATE DESCENT; MATRIX SIZE; MULTITASK LEARNING; STATE OF THE ART; TRACE-NORMS; VARIATIONAL BOUNDS;

SINGULAR VALUE DECOMPOSITION;

EID: 84954222977     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (40)

1. A. Argyriou, T. Evgeniou, and M. Pontil. Convex multi-task feature learning. Machine Learning. 73(3):243-272, 2008.
2. F. Bach. Consistency of trace norm minimization. JMLR, 9:1019-1048, 2008.
3. A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1):183-202, 2009.
4. A. Berg, J. Deng, and F.-F. Li. ImageNet large scale visual recognition challenge, 2010. http://www.image-net.org/.
5. D. Bertsekas. Nonlinear Programming (2nd ed.). Athena Scientific, 2004.
6. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge UP, 2004.
7. V. Chandrasekaran. Convex Optimization Methods for Graphs and Statistical Modeling. PhD thesis, MIT, 2011.
8. O. Chapelle and Z. Harchaoui. A machine learning approach to conjoint analysis. In Adv. NIPS. 2005.
9. K. Chen. Matrix Preconditioning Techniques and Applications. Cambridge UP, 2005.
10. K. L. Clarkson. Coresets, sparse greedy approximation, and the Frank-wolfe algorithm. In Proc. SODA, 2008.
11. M. Collins, R. E. Schapire, and Y. Singer. Logistic regression, AdaBoost and Bregman distances. Machine Learning, 47, 2002.
12. A. Demiriz, K. P. Bennett, and J. Shawe-Taylor. Linear programming boosting via column generation. Machine Learning, 46, 2002.
13. V. Demyanov and A. Rubinov. Approximate Methods in Optimization Problems. American Elsevier, 1970.
14. M. Dudik, S. J. Phillips, and R. E. Schapire. Maximum entropy density estimation with generalized regularization and an application to species distribution modeling. JMLR, 8:1217-1260, 2007.
15. M. Fazel. Matrix rank minimization with applications. PhD thesis, Stanford, 2002.
16. M. Frank and P. Wolfe. An algorithm for quadratic programming. Naval Research Logistics Quarterly, 3:95-110, 1956.
17. J. Friedman, T. Hastie, and R. Tibshirani. Regu-larization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1), 2010.
18. T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning (2nd Ed.). Springer Series in Statistics. Springer, 2008.
19. E. Hazan. Sparse approximate solutions to semidefinite programs. In Proc. 8th Latin American Conf. Theor. Informatics, pages 306-316, 2008.
20. J.-B. Hiriart-Urruty and C. Lemarechal. Convex Analysis and Minimization Algorithms. Springer Verlag, Heidelberg, 1993. Two volumes.
21. E. J. C. J-F Cai and Z. Shen. A singular value thresholding algorithm for matrix completion. SIAM J. Optimization, 20(4):1956-1982, 2008.
22. M. Jaggi and M. Sulovsky. A simple algorithm for nuclear norm regularized problems. In ICML, 2010.
23. G. Jameson. Summing and nuclear norms in Ba-nach space theory. London Mathematical Society Student Texts, 8. Cambridge University Press. XI, 1987.
24. S. Ji and J. Ye. An accelerated gradient method for trace norm minimization. In ICML, 2009.
25. L. Mason, P. Bartlett, J. Baxter, and M. Frean. Functional gradient techniques for combining hypotheses. In B. Scholkopf, A. Smola, P. Bartlett, and D. Schuurmans, editors, Adv. Large Margin Classifiers. MIT Press, 2000.
26. Y. Nesterov. Gradient methods for minimizing composite objective function. Technical report, CORE, 2007.
27. Y. Nesterov. Efficiency of coordinate descent methods on huge-scale optimization problems. Technical report, CORE, 2010.
28. G. Obozinski, B. Taskar, and M. I. Jordan. Joint covariate selection and joint subspace selection for multiple classification problems. Statistics and Computing, 2010.
29. R. Phelps. Convex functions, monotone operators, and differentiability. Lecture notes in mathematics. Springer-Verlag, 1993.
30. T. K. Pong, S. J. Paul Tseng, and J. Ye. Trace norm regularization: Reformulations, algorithms, and multi-task learning. SIAM J. Optimization, 20(6):3465-3489, 2010.
31. B. Recht, M. Fazel, and P. Parrilo. Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Review, 52(3):471-501, 2010.
32. R. Rockafellar. Convex Analysis. Princeton University Press, 1970.
33. S. Shalev-Shwartz, A. Gonen, and O. Shamir. Large-scale convex minimization with a low-rank constraint. In ICML, 2011.
34. W. Shi, G. Wahba, S. Wright, K. Lee, R. Klein, and B. Klein. Lasso-patternsearch algorithm with application to ophthalmology and genomic data. ASA Proceedings of the Joint Statistical Meetings, 2006.
35. S. Sra, S. Nowozin, and S. J. Wright. Optimization for Machine Learning. The MIT Press, 2010.
36. N. Srebro, J. D. M. Rennie, and T. S. Jaakola. Maximum-margin matrix factorization. In Adv. NIPS, 2005.
37. A. Tewari, P. K. Ravikumar, and I. S. Dhillon. Greedy algorithms for structurally constrained high dimensional problems. In Adv. NIPS, 2011.
38. S. Wright. Accelerated block-coordinate relaxation for regularized optimization. Technical report, 2010.
39. G.-X. Yuan, K.-W. Chang, C.-J. Hsieh, and C.-J. Lin. A comparison of optimization methods and software for large-scale l1-regularized linear classification. JMLR, 11, December 2010.
40. T. Zhang. Sequential greedy approximation for certain convex optimization problems. IEEE Transaction on Information Theory, 49:682-691, 2003.