A feasible method for optimization with orthogonality constraints

Mathematical Programming

Volumn 142, Issue 1-2, 2013, Pages 397-434

  Wen, Zaiwen ^a   Yin, Wotao ^b  

^a SHANGHAI JIAO TONG UNIVERSITY   (China)

^b Rice University   (United States)

Author keywords

Cayley transformation; Curvilinear search; Eigenvalue and eigenvector; Invariant subspace; Maxcut SDP; Nearest correlation matrix; Orthogonality constraint; Polynomial optimization; Quadratic assignment problem; Spherical constraint; Stiefel manifold

Indexed keywords

CAYLEY TRANSFORMATION; CORRELATION MATRIX; CURVILINEAR SEARCH; EIGEN-VALUE; INVARIANT SUBSPACE; MAX-CUT; ORTHOGONALITY CONSTRAINTS; POLYNOMIAL OPTIMIZATION; QUADRATIC ASSIGNMENT PROBLEMS; SPHERICAL CONSTRAINT; STIEFEL MANIFOLD;

ALGORITHMS; COMBINATORIAL OPTIMIZATION; LINEAR TRANSFORMATIONS; MATHEMATICAL TRANSFORMATIONS; MATRIX ALGEBRA; POLYNOMIALS;

EIGENVALUES AND EIGENFUNCTIONS;

EID: 84879905842     PISSN: 00255610     EISSN: 14364646     Source Type: Journal    
DOI: 10.1007/s10107-012-0584-1     Document Type: Article

References (63)

1. Absil, P.-A.; Baker, C.G.; Gallivan, K.A.: Trust-region methods on Riemannian manifolds. Found. Comput. Math. 7, 303-330 (2007)
2. Absil, P.-A.; Mahony, R.; Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008)
3. Adler, R.L.; Dedieu, J.-P.; Margulies, J.Y.; Martens, M.; Shub, M.: Newton's method on Riemannian manifolds and a geometric model for the human spine. IMA J. Numer. Anal. 22, 359-390 (2002)
4. Baker, C.G.; Absil, P.-A.; Gallivan, K.A.: An implicit trust-region method on Riemannian manifolds. IMA J. Numer. Anal. 28, 665-689 (2008)
5. Barzilai, J.; Borwein, J.M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8, 141-148 (1988)
6. Bencteux, G.; Cancés, E.; Hager, W.W.; Le Bris, C.: Analysis of a quadratic programming decomposition algorithm. SIAM J. Numer. Anal. 47, 4517-4539 (2010)
7. Benson, S.J.; Ye, Y.; Zhang, X.: Solving large-scale sparse semidefinite programs for combinatorial optimization. SIAM J. Optim. 10, 443-461 (2000)
8. Boufounos, P.; Baraniuk, R.: 1-bit Compressive Sensing. Conf. on. Info. Science and Systems (CISS), Princeton (2008)
9. Brace, I.; Manton, J.H.: An improved BFGS-on-manifold algorithm for computing weighted low rank approximations. In Proceedings of the 17th International Symposium on Mathematical Theory of Networks and Systems, pp. 1735-1738 (2006)
10. Burer, S.; Monteiro, R.D.C.: A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization. Math. Program. 95, 329-357 (2003)
11. Burkard, R.E.; Karisch, S.E.; Rendl, F.: QAPLIB: a quadratic assignment problem library. J. Global Optim. 10, 391-403 (1997)
12. Cancès, E.; Le Bris, C.; Lions, P.-L.: Molecular simulation and related topics: some open mathematical problems. Nonlinearity 21, T165-T176 (2008)
13. Chang, G.J.; Huang, L.-H.; Yeh, H.-G.: On the rank of a cograph. Linear Algebra Appl. 429, 601-605 (2008)
14. Dai, Y.-H.; Fletcher, R.: Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming. Numer. Math. 100, 21-47 (2005)
15. d'Aspremont, A.; El Ghaoui, L.; Jordan, M.; Lanckriet, G.R.: A direct formulation for sparse PCA using semidefinite programming. SIAM Rev. 49, 434-448 (2007)
16. Davis, T.A.: The University of Florida Sparse Matrix Collection, Technical Report. University of Florida, Florida (2010)
17. Dolan, E.D.; Moré, J.J.; Munson, T.S.: Benchmarking Optimization Software with cops 3.0, Technical Report. Mathematics and Computer Science Division, Argonne National Laboratory (February 2004)
18. Edelman, A.; Arias, T.A.; Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20, 303-353 (1999)
19. Fletcher, R.: Practical Methods of Optimization. John Wiley & Sons (2000)
20. Francisco, J.B.; Martínez, J.M.; Martínez, L.; Pisnitchenko, F.: Inexact restoration method for minimization problems arising in electronic structure calculations. Comput. Opt. Appl. 10(3), 555-590 (2011)
21. Friedland, S.; Nocedal, J.; Overton, M.L.: The formulation and analysis of numerical methods for inverse eigenvalue problems. SIAM J. Numer. Anal. 24, 634-667 (1987)
22. Gao, Y.; Sun, D.: A Majorized Penalty Approach for Calibrating Rank Constrained Correlation Matrix Problems, Technical Report. National University of Singapore (2010)
23. Goldfarb, D.; Wen, Z.; Yin, W.: A curvilinear search method for the p-harmonic flow on spheres. SIAM J. Imaging Sci. 2, 84-109 (2009)
24. Golub, G.H.; Van Loan, C.F.: Matrix Computations. Johns Hopkins Studies in the Mathematical Sciences, Johns Hopkins University Press, Baltimore (1996)
25. Grippo, L.; Palagi, L.; Piccialli, V.: An unconstrained minimization method for solving low-rank sdp relaxations of the maxcut problem. Math. Program.; 1-28 (2009). doi: 10.1007/s10107-009-0275-8
26. Grubišić, I.; Pietersz, R.: Efficient rank reduction of correlation matrices. Linear Algebra Appl. 422, 629-653 (2007)
27. He, S.; Li, Z.; Zhang, S.: Approximation algorithms for homogeneous polynomial optimization with quadratic constraints. Math. Program. 125, 353-383 (2010)
28. Helmberg, C.; Rendl, F.: A spectral bundle method for semidefinite programming. SIAM J. Optim. 10, 673-696 (2000)
29. Helmke, U.; Moore, J.B.: Optimization and dynamical systems. In: Brockett, R. (ed.) Communications and Control Engineering Series. Springer, London (1994)
30. Horn, R.A.; Johnson, C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1985)
31. Kokiopoulou, E.; Chen, J.; Saad, Y.: Trace Optimization and Eigenproblems in Dimension Reduction Methods, Technical report. University of Minnesota (2010)
32. Kružík, M.; Prohl, A.: Recent developments in the modeling, analysis, and numerics of ferromagnetism. SIAM Rev. 48, 439-483 (2006)
33. Laska, J.N.; Wen, Z.; Yin, W.; Baraniuk, R.G.: Trust, but Verify: Fast and Accurate Signal Recovering from 1-bit Compressive Measurements, Technical report. Rice University (2010)
34. Lu, Z.; Zhang, Y.: An Augmented Lagrangian Approach for Sparse Principal Component, Analysis, arXiv:0907.2079 (2009)
35. Malick, J.; Povh, J.; Rendl, F.; Wiegele, A.: Regularization methods for semidefinite programming. SIAM J. Optim. 20, 336-356 (2009)
36. Moré, J.J.; Thuente, D.J.: Line search algorithms with guaranteed sufficient decrease. ACM Trans. Math. Softw. 20, 286-307 (1994)
37. Nemirovski, A.: Sums of random symmetric matrices and quadratic optimization under orthogonality constraints. Math. Program. 109, 283-317 (2007)
38. Nie, J.: Regularization Methods for Sum of Squares Relaxations in Large Scale Polynomial Optimization, Technical report. Department of Mathematics, UCSD (2009)
39. Nishimori, Y.; Akaho, S.: Learning algorithms utilizing quasi-geodesic flows on the stiefel manifold. Neurocomputing 67, 106-135 (2005)
40. Nocedal, J.; Wright, S.J.: Numerical Optimization. Springer Series in Operations Research and Financial Engineering, 2nd edn. Springer, New York (2006)
41. Owren, B.; Welfert, B.: The Newton iteration on Lie groups. BIT 40, 121-145 (2000)
42. Pietersz, R.; Groenen, P.J.F.: Rank reduction of correlation matrices by majorization. Quant. Finance 4, 649-662 (2004)
43. Qi, C.; Gallivan, K.A.; Absil, P.-A.: Riemannian BFGS algorithm with applications. In: Diehl, M.; Glineur, F.; Jarlebring, E.; Michiels, W. (eds.) Recent Advances in Optimization and its Applications in Engineering. Springer, Berlin (2010)
44. Raydan, M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7, 26-33 (1997)
45. Rebonato, R.; Jäckel, P.: The most general methodology to create a valid correlation matrix for risk management and option pricing purposes. J. Risk 2, 17-27 (1999)
46. Roy, R.; Kailath, T.: Esprit - estimation of signal parameters via rotational invariance techniques. IEEE Trans. Acoust, Speech, Signal Process. 37(7), 984-995 (1989)
47. Schneider, R.; Rohwedder, T.; Neelov, A.; Blauert, J.: Direct minimization for calculating invariant subspaces in density functional computations of the electronic structure. J. Comput. Math. 27, 360-387 (2009)
48. Shub, M.: Some remarks on dynamical systems and numerical analysis. In Dynamical Systems and Partial Differential Equations (Caracas, 1984), pp. 69-91. Univ. Simon Bolivar, Caracas (1986)
49. Simon, D.; Abell, J.: A majorization algorithm for constrained correlation matrix approximation. Linear Algebra Appl. 432, 1152-1164 (2010)
50. Smith, S.T.: Geometric Optimization Methods for Adaptive Filtering. ProQuest LLC, Ann Arbor, MI. PhD Thesis, Harvard University (1993)
51. Smith, S.T.: Optimization techniques on Riemannian manifolds. In Hamiltonian and Gradient Flows, Algorithms and Control, vol. 3, pp. 113-136. Fields Inst. Commun.; Amer. Math. Soc.; Providence, RI (1994)
52. Sun, W.; Yuan, Y.-X.: Optimization Theory and Methods, vol. 1 of Springer Optimization and its Applications. Springer, New York (2006)
53. Udrişte, C.: Convex Functions and Optimization Methods on Riemannian Manifolds, vol. 297 of Mathematics and its Applications. Kluwer Academic Publishers Group, Dordrecht (1994)
54. Vese, L.A.; Osher, S.J.: Numerical methods for p -harmonic flows and applications to image processing. SIAM J. Numer. Anal. 40, 2085-2104 (2002)
55. Wächter, A.; Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106, 25-57 (2006)
56. Ward, J.: Space-Time Adaptive Processing for Airborne Radar, Technical report of MIT (1994)
57. Weber, V.; VandeVondele, J.; Hütter, J.; Niklasson, A.M.: Direct energy functional minimization under orthogonality constraints. J. Chem. Phys. 128(8), 084113 (2008)
58. Witten, D.M.; Tibshirani, R.; Hastie, T.: A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. Biostatistics 10, 515-534 (2009)
59. Yang, B.: Projection approximation subspace tracking. IEEE Trans. Signal Process. 43, 95-107 (1995)
60. Yang, C.; Meza, J.C.; Lee, B.; Wang, L.-W.: Kssolv-a matlab toolbox for solving the kohn-sham equations. ACM Trans. Math. Softw. 36, 1-35 (2009)
61. Yang, C.; Meza, J.C.; Wang, L.-W.: A constrained optimization algorithm for total energy minimization in electronic structure calculations. J. Comput. Phys. 217, 709-721 (2006)
62. Zhang, H.; Hager, W.W.: A nonmonotone line search technique and its application to unconstrained optimization. SIAM J. Optim. 14, 1043-1056 (2004)
63. Zhao, X.; Sun, D.; Toh, K.: A Newton-CG augmented lagrangian method for semidefinite programming. SIAM J. Optim. 20, 1737-1765 (2010)