Efficient low-rank stochastic gradient descent methods for solving semidefinite programs

Journal of Machine Learning Research

Volumn 33, Issue , 2014, Pages 122-130

  Chen, Jianhui ^a   Yang, Tianbao ^b   Zhu, Shenghuo ^b  

^a GE GLOBAL RESEARCH   (United States)

^b NEC LABORATORIES AMERICA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMMUNICATION CHANNELS (INFORMATION THEORY); ITERATIVE METHODS; OPTIMIZATION; TIME VARYING NETWORKS;

COMPUTATIONAL ADVANTAGES; OPTIMAL CONVERGENCE; OPTIMAL SOLUTIONS; SEMI-DEFINITE PROGRAMMING; SEMIDEFINITE PROGRAMS; STOCHASTIC GRADIENT; STOCHASTIC GRADIENT DESCENT; STOCHASTIC GRADIENT DESCENT METHOD;

STOCHASTIC SYSTEMS;

EID: 84955457109     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (34)

1. A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro, "Robust stochastic approximation approach to stochastic programming," SIAM J. on Optimization, vol. 19, pp. 1574-1609, 2009.
2. K. Q. Weinberger and L. K. Saul, "Distance metric learning for large margin nearest neighbor classification," J. Mach. Learn. Res., vol. 10, pp. 207-244, 2009.
3. A. d'Aspremont, O. Banerjee, and L. El Ghaoui, "First-order methods for sparse covariance selection," SIAM J. Matrix Anal. Appl., vol. 30, pp. 56-66, 2008.
4. S. Nathan, D. M. R. Jason, and S. J. Tommi, "Maximum margin matrix factorization," in NIPS, 2005.
5. Y. Zhang and Y. Dit-Yan, "A convex formulation for learning task relationships in multi-task learning," in UAI, 2010.
6. J. Chen, L. Tang, J. Liu, and J. Ye, "A convex formulation for learning a shared predictive structure from multiple tasks," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, pp. 1025-1038, 2013.
7. A. d'Aspremont, L. El Ghaoui, M. I. Jordan, and G. R. G. Lanckriet, "A direct formulation for sparse pca using semidefinite programming," SIAM Rev., vol. 49, pp. 434-448, 2007.
8. W. Liu, X. Tian, D. Tao, and J. Liu, "Constrained metric learning via distance gap maximization." in AAAI, 2010.
9. S. C. Hoi, W. Liu, and S.-F. Chang, "Semi-supervised distance metric learning for collaborative image retrieval and clustering," ACM Trans. Multimedia Comput. Commun. Appl., vol. 6, pp. 18-1-18:26, 2010.
10. M. Mahdavi, T. Yang, R. Jin, S. Zhu, and J. Yi, "Stochastic gradient descent with only one projection," in NIPS, 2012.
11. E. Hazan and S. Kale, "Stochastic gradient descent with only one projection," in ICML, 2012.
12. M. Frank and P. Wolfe, "An algorithm for quadratic programming. naval research logistics," Naval Res. Logistics, vol. 3, pp. 95-110, 1956.
13. Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course. Springer Netherlands, 2003.
14. J. V. Davis, B. Kulis, P. Jain, S. Sra, and I. S. Dhillon, "Information-theoretic metric learning," in ICML, 2007.
15. S. Shalev-Shwartz, Y. Singer, and A. Y. Ng, "Online and batch learning of pseudo-metrics," in ICML, 2004.
16. P. Jain, B. Kulis, I. S. Dhillon, and K. Grauman, "Online metric learning and fast similarity search," in NIPS, 2008, pp. 761-768.
17. H. Avron, S. Kale, S. Kasiviswanathan, and V. Sindhwani, "Efficient and practical stochastic subgradient descent for nuclear norm regularization," in ICML, 2012.
18. S. Arora, E. Hazan, and S. Kale, "Fast algorithms for approximate semide.nite programming using the multiplicative weights update method," in FOCS, 2005.
19. E. Hazan, "Sparse approximate solutions to semidefinite programs," in LATIN, 2008.
20. A. Kleiner, A. Rahimi, and M. I. Jordan, "Random conic pursuit for semidefinite programming," in NIPS, 2010.
21. S. Laue, "A hybrid algorithm for convex semidefinite optimization," in ICML, 2012.
22. D. Garber and E. Hazan, "Approximating semidefinite programs in sublinear time," in NIPS, 2011.
23. Y. Nesterov, "Smoothing technique and its applications in semidefinite optimization," Math. Program., pp. 245-259, 2007.
24. Y. Ying and P. Li, "Distance metric learning with eigenvalue optimization," J. Mach. Learn. Res., pp. 1-26, 2012.
25. K. Weinberger and L. Saul, "Fast solvers and efficient implementations for distance metric learning," in ICML, 2008.
26. M. Journée, Y. Nesterov, P. Richtárik, and R. Sepulchre, "Generalized power method for sparse principal component analysis," J. Mach. Learn. Res., vol. 11, pp. 517-553, 2010.
27. A. d'Aspremont, F. Bach, and L. E. Ghaoui, "Optimal solutions for sparse principal component analysis," J. Mach. Learn. Res., vol. 9, pp. 1269-1294, 2008.
28. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge University Press, 2004.
29. Z. Wen, D. Goldfarb, and W. Yin, "Alternating direction augmented lagrangian methods for semidefinite programming," Mathematical Programming Computation, 2010.
30. H. Ouyang, N. He, L. Tran, and A. G. Gray, "Stochastic alternating direction method of multipliers," in ICML, 2013.
31. N. Srebro and A. Tewari, "Stochastic optimization for machine learning," in ICML Tutorial, 2010.
32. J. N. Franklin, Matrix Theory. Courier Dover Publications, 2000.
33. J. Duchi, S. Shalev-Shwartz, Y. Singer, and A. Tewari, "Composite objective mirror descent." in COLT, 2010.
34. P. Sen, G. M. Namata, M. Bilgic, L. Getoor, B. Gallagher, and T. Eliassi-Rad, "Collective classification in network data," AI Magazine, vol. 29, no. 3, 2008.