Block coordinate descent methods for semidefinite programming

International Series in Operations Research and Management Science

Volumn 166, Issue , 2012, Pages 533-564

  Wen, Zaiwen ^a   Goldfarb, Donald ^b   Scheinberg, Katya ^c  

^a SHANGHAI JIAO TONG UNIVERSITY   (China)

^b Columbia University ^*  (United States)

^c Lehigh University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84955061741     PISSN: 08848289     EISSN: None     Source Type: Book Series    
DOI: 10.1007/978-1-4614-0769-0_19     Document Type: Chapter

References (78)

1. Alizadeh, F., Goldfarb, D.: Second-order cone programming. Math. Programming 95(1, Ser. B), 3-51 (2003). ISMP 2000, Part 3 (Atlanta, GA)
2. Banerjee, O., El Ghaoui, L., d'Aspremont, A.: Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. J. Mach. Learn. Res. 9, 485-516 (2008)
3. Benson, S. J., Ye, Y., Zhang, X.: Solving large-scale sparse semidefinite programs for combinatorial optimization. SIAM J. Optim. 10(2), 443-461 (2000)
4. Bertsekas, D. P.: Necessary and sufficient condition for a penalty method to be exact. Math. Programming 9(1), 87-99 (1975)
5. Bertsekas, D. P.: Constrained optimization and Lagrange multipliermethods. Computer Science and Applied Mathematics. Academic Press Inc., New York (1982)
6. Bertsekas, D. P.: Nonlinear Programming. Athena Scientific (1999)
7. Bertsekas, D. P., Tsitsiklis, J. N.: Parallel and distributed computation: numerical methods. Prentice-Hall, Inc., Upper Saddle River, NJ, USA (1989)
8. Bo, L., Sminchisescu, C.: Greedy Block Coordinate Descent for Large Scale Gaussian Process Regression. In: Uncertainty in Artificial Intelligence (2008)
9. Bouman, C. A., Sauer, K.: A unified approach to statistical tomography using coordinate descent optimization. IEEE Transactions on Image Processing 5, 480-492 (1996)
10. Boyd, S., Vandenberghe, L.: Convex optimization. Cambridge University Press, Cambridge (2004)
11. Burer, S.: Optimizing a polyhedral-semidefinite relaxation of completely positive programs. Tech. rep., Department of Management Sciences, University of Iowa (2008)
12. Burer, S., Monteiro, R. D. C.: A projected gradient algorithm for solving the maxcut SDP relaxation. Optim. Methods Softw. 15(3-4), 175-200 (2001)
13. Burer, S., Monteiro, R. D. C.: A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization. Math. Program. 95(2, Ser. B), 329-357 (2003)
14. Burer, S., Monteiro, R. D. C.: Local minima and convergence in low-rank semidefinite programming. Math. Program. 103(3, Ser. A), 427-444 (2005)
15. Burer, S., Vandenbussche, D.: Solving lift-and-project relaxations of binary integer programs. SIAM J. Optim. 16(3), 726-750 (2006)
16. Candès, E. J., Recht, B.: Exact matrix completion via convex optimization. Foundations of Computational Mathematics (2009). DOI 10. 1007/s10208-009-9045-5
17. Chang, K. W., Hsieh, C. J., Lin, C. J.: Coordinate descent method for large-scale L2-loss linear support vector machines. J. Mach. Learn. Res. 9, 1369-1398 (2008)
18. Chen, G., Teboulle, M.: A proximal-based decomposition method for convex minimization problems. Math. Programming 64(1, Ser. A), 81-101 (1994)
19. Clarkson, K. L.: Coresets, sparse greedy approximation, and the Frank-Wolfe algorithm. In: SODA '08: Proceedings of the nineteenth annual ACM-SIAM symposium on Discrete algorithms, pp. 922-931 (2008)
20. Diachin, L. F., Knupp, P., Munson, T., Shontz, S.: A comparison of inexact newton and coordinate descent mesh optimization techniques. In: 13th International Meshing Roundtable, Williamsburg, VA, Sandia National Laboratories, pp. 243-254 (2004)
21. Eckstein, J., Bertsekas, D. P.: An alternating direction method for linear programming. LIDS-P, 1967. Laboratory for Information and Decision Systems, MIT
22. Eckstein, J., Bertsekas, D. P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximalmonotone operators. Math. Programming 55(3, Ser. A), 293-318 (1992)
23. Elad, M., Matalon, B., Zibulevsky, M.: Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization. Appl. Comput. Harmon. Anal. 23(3), 346-367 (2007)
24. Fessler, J. A.: Grouped coordinate descent algorithms for robust edge-preserving image restoration. In: in Proc. SPIE 3071, Im. Recon. and Restor. II, pp. 184-94 (1997)
25. Fletcher, R.: Practical methods of optimization, second edn. AWiley-Interscience Publication. John Wiley & Sons Ltd., Chichester (1987)
26. Friedman, J., Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9, 432-441 (2008)
27. Fujisawa, K., Kojima, M., Nakata, K.: Exploiting sparsity in primal-dual interior-point methods for semidefinite programming. Math. Programming 79(1-3, Ser. B), 235-253 (1997)
28. Goemans, M. X., Williamson, D. P.: Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J. Assoc. Comput. Mach. 42(6), 1115-1145 (1995)
29. Goldfarb, D., Iyengar, G.: Robust convex quadratically constrained programs. Math. Program. 97(3, Ser. B), 495-515 (2003). New trends in optimization and computational algorithms (NTOC 2001) (Kyoto)
30. Goldfarb, D., Iyengar, G.: Robust portfolio selection problems. Math. Oper. Res. 28(1), 1-38 (2003)
31. Goldfarb, D., Ma, S.: Fast alternating linearization methods for minimizing the sum of two convex functions. Tech. rep., IEOR, Columbia University (2009)
32. Goldfarb, D., Ma, S.: Fast multiple splitting algorithms for convex optimization. Tech. rep., IEOR, Columbia University (2009)
33. Grippo, L., Sciandrone, M.: Globally convergent block-coordinate techniques for unconstrained optimization. Optim. Methods Softw. 10(4), 587-637 (1999)
34. Grippo, L., Sciandrone, M.: On the convergence of the block nonlinear Gauss-Seidel method under convex constraints. Oper. Res. Lett. 26(3), 127-136 (2000)
35. Hackbusch, W.: Multigrid methods and applications, Springer Series in Computational Mathematics, vol. 4. Springer-Verlag, Berlin (1985)
36. He, B., Liao, L. Z., Han, D., Yang, H.: A new inexact alternating directions method for monotone variational inequalities. Math. Program. 92(1, Ser. A), 103-118 (2002)
37. He, B. S., Yang, H., Wang, S. L.: Alternating direction method with self-adaptive penalty parameters for monotone variational inequalities. J. Optim. Theory Appl. 106(2), 337-356 (2000)
38. Helmberg, C., Rendl, F.: A spectral bundle method for semidefinite programming. SIAM J. Optim. 10(3), 673-696 (2000)
39. Huang, F. L., Hsieh, C. J., Chang, K. W., Lin, C. J.: Iterative scaling and coordinate descent methods for maximum entropy. In: ACL-IJCNLP '09: Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pp. 285-288. Association for Computational Linguistics, Morristown, NJ, USA (2009)
40. Jian-Feng, C., Candes, E. J., Zuowei, S.: A singular value thresholding algorithm for matrix completion export. SIAM J. Optim. 20, 1956-1982 (2010)
41. Kiwiel, K. C., Rosa, C. H., Ruszczyński, A.: Proximal decomposition via alternating linearization. SIAM J. Optim. 9(3), 668-689 (1999)
42. Kolda, T. G., Lewis, R. M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45(3), 385-482 (2003)
43. Kontogiorgis, S., Meyer, R. R.: A variable-penalty alternating directions method for convex optimization. Math. Programming 83(1, Ser. A), 29-53 (1998)
44. Li, Y., Osher, S.: Coordinate descent optimization for _1 minimization with application to compressed sensing; a greedy algorithm. Inverse Probl. Imaging 3(3), 487-503 (2009)
45. Luo, Z. Q., Tseng, P.: On the convergence of the coordinate descent method for convex differentiable minimization. J. Optim. Theory Appl. 72(1), 7-35 (1992)
46. Ma, S., Goldfarb, D., Chen, L.: Fixed point and bregman iterative methods for matrix rank minimization. Mathematical Programming pp. 1-33 (2009)
47. Malick, J., Povh, J., Rendl, F., Wiegele, A.: Regularization methods for semidefinite programming. SIAM Journal on Optimization 20(1), 336-356 (2009)
48. Mazumder, R., Friedman, J., Hastie, T.: Sparsenet: Coordinate descent with non-convex penalties. Tech. rep., Stanford University (2009)
49. Nesterov, Y.: Efficiency of coordinate descent methods on huge-scale optimization problems. Tech. rep., CORE Discussion Paper (2010)
50. Povh, J., Rendl, F., Wiegele, A.: A boundary point method to solve semidefinite programs. Computing 78(3), 277-286 (2006)
51. Powell, M. J. D.: On search directions for minimization algorithms. Math. Programming 4, 193-201 (1973)
52. Rockafellar, R. T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1(2), 97-116 (1976)
53. Saad, Y.: Iterative methods for sparse linear systems, second edn. Society for Industrial and Applied Mathematics, Philadelphia, PA (2003)
54. Saha, A., Tewari, A.: On the finite time convergence of cyclic coordinate descent methods. Tech. rep., University of Chicago (2010)
55. Shalev-Shwartz, S., Tewari, A.: Stochastic methods for l1 regularized loss minimization. In: ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 929-936. ACM (2009)
56. Tai, X. C., Xu, J.: Global and uniform convergence of subspace correction methods for some convex optimization problems. Math. Comp. 71(237), 105-124 (2002)
57. Todd, M. J.: Semidefinite optimization. Acta Numer. 10, 515-560 (2001)
58. Tseng, P.: Dual coordinate ascent methods for non-strictly convex minimization. Math. Programming 59(2, Ser. A), 231-247 (1993)
59. Tseng, P.: Alternating projection-proximal methods for convex programming and variational inequalities. SIAM J. Optim. 7(4), 951-965 (1997)
60. Tseng, P.: Convergence of a block coordinate descent method for nondifferentiable minimization. J. Optim. Theory Appl. 109(3), 475-494 (2001)
61. Tseng, P., Yun, S.: A coordinate gradient descent method for linearly constrained smooth optimization and support vector machines training. Comput. Optim. Appl. (2007)
62. Tseng, P., Yun, S.: Block-coordinate gradient descent method for linearly constrained nonsmooth separable optimization. J. Optim. Theory Appl. 140(3), 513-535 (2009). DOI 10. 1007/ s10957-008-9458-3
63. Tseng, P., Yun, S.: A coordinate gradient descent method for nonsmooth separable minimization. Math. Program. 117(1-2, Ser. B), 387-423 (2009)
64. Vandenberghe, L., Boyd, S.: Semidefinite programming. SIAM Rev. 38(1), 49-95 (1996)
65. Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci. 1(3), 248-272 (2008)
66. Wen, Z.: First-order methods for semidefinite programming. Ph. D. thesis, Columbia University (2009)
67. Wen, Z., Goldfarb, D., Ma, S., Scheinberg, K.: Row by row methods for semidefinite programming. Tech. rep., Dept of IEOR, Columbia University (2009)
68. Wen, Z., Goldfarb, D., Yin, W.: Alternating direction augmented lagrangian methods for semidefinite programming. Mathematical Programming Computation 2(3-4), 203-230 (2010)
69. Wen, Z., Yin, W., Zhang, Y.: Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm. Tech. rep., Rice University (2010). CAAM Technical Report TR10-07
70. Wolkowicz, H., Saigal, R., Vandenberghe, L. (eds.): Handbook of semidefinite programming. Kluwer Academic Publishers, Boston, MA (2000)
71. Yang, J., Zhang, Y.: Alternating direction algorithms for l1-problems in compressive sensing. Tech. rep., Rice University (2009)
72. Ye, C., Yuan, X.: A descent method for structured monotone variational inequalities. Optimization Methods and Software 22, 329-338 (2007)
73. Ye, J. C., Webb, K. J., Bouman, C. A., Millane, R. P.: Optical diffusion tomography by iterativecoordinate-descent optimization in a bayesian framework. J. Opt. Soc. Am. A 16(10), 2400-2412 (1999)
74. Yu, Z.: Solving semidefinite programming problems via alternating direction methods. J. Comput. Appl. Math. 193(2), 437-445 (2006)
75. Yun, S., Toh, K. C.: A coordinate gradient descent method for l1-regularized convex minimization. Comput. Optim. Appl. (2009)
76. Zhang, F. (ed.): The Schur complement and its applications, Numerical Methods and Algorithms, vol. 4. Springer-Verlag, New York (2005)
77. Zhang, Y.: User's guide for YALL1: Your algorithms for l1 optimization. Tech. rep., Rice University (2009)
78. Zhao, X., Sun, D., Toh, K.: A Newton-CG augmented lagrangian method for semidefinite programming. SIAM Journal on Optimization 20, 1737-1765 (2010)