Interior-point method for nuclear norm approximation with application to system identification

SIAM Journal on Matrix Analysis and Applications

Volumn 31, Issue 3, 2009, Pages 1235-1256

  Zhang, Liu ^a   Lieven, Vandenberghe ^a  

^a Engineering IV   (United States)

Author keywords

Interior point methods; Matrix rank minimization; Nuclear norm approximation; Semidefinite programming; Subspace algorithm; System identification

Indexed keywords

INTERIOR-POINT METHOD; MATRIX RANK; SEMI-DEFINITE PROGRAMMING; SUB-SPACE ALGORITHMS; SYSTEM IDENTIFICATIONS;

COST REDUCTION; HEURISTIC METHODS; MATHEMATICAL PROGRAMMING; SIGNAL PROCESSING; SINGULAR VALUE DECOMPOSITION;

APPROXIMATION ALGORITHMS;

EID: 72549110327     PISSN: 08954798     EISSN: 10957162     Source Type: Journal    
DOI: 10.1137/090755436     Document Type: Article

References (49)

1. S. J. BENSON AND Y. YE, Algorithm 875: DSDP5: Software for semidefinite programming, ACM Trans. Math. Software, 34 (2008), pp. 1-20.
2. S. J. BENSON AND Y. YE, DSDP5 User Guide-Software for Semidefinite Programming, Technical report ANL/MCS-TM-277, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL, 2005.
3. B. BORCHERS, CSDP, a C library for semidefinite programming, Optim. Methods Softw., 11 (1999), pp. 613-623.
4. S. BOYD AND L. VANDENBERGHE, Convex Optimization, Cambridge University Press, Cambridge, UK, 2004. Also available online at www.stanford.edu/∼boyd/ cvxbook.
5. J.-F. CAI, E. J. CANDÈS, AND Z. SHEN, A Singular Value Thresholding Algorithm for Matrix Completion, http://arxiv.org/abs/0810.3286, 2008.
6. E. J. CANDÈS AND Y. PLAN, Matrix completion with noise, Proc. IEEE, submitted.
7. E. J. CANDÈS AND B. RECHT, Exact Matrix Completion via Convex Optimization, Foundations of Computational Mathematics, to appear.
8. E. J. CANDÈS, J. ROMBERG, AND T. TAO, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory, 52 (2006), pp. 489-509.
9. E. J. CANDÈS, J. K. ROMBERG, AND T. TAO, Stable signal recovery from incomplete and inaccurate measurements, Comm. Pure Appl. Math., 59 (2006), pp. 1207-1223.
10. E. J. CANDÈS AND T. TAO, Decoding by linear programming, IEEE Trans. Inform. Theory, 51 (2005), pp. 4203-4215.
11. E. J. CANDÈS AND T. TAO, The power of convex relaxation: Near-optimal matrix completion, IEEE Trans. Inform. Theory, to appear.
12. S. S. CHEN, D. L. DONOHO, AND M. A. SAUNDERS, Atomic decomposition by basis pursuit, SIAM Rev., 43 (2001), pp. 129-159.
13. J. DAHL AND L. VANDENBERGHE, CVXOPT: A Python Package for Convex Optimization, available online at http://www.abel.ee.ucla.edu/cvxopt, 2008.
14. B. DE MOOR, Total least squares for affinely structured matrices and the noisy realization problem, IEEE Trans. Signal Process., 42 (1994), pp. 3104-3113.
15. B. DE MOOR, P. DE GERSEM, B. DE SCHUTTER, AND W. FAVOREEL, DAISY: A database for the identification of systems, Journal A, 38 (1997), pp. 4-5.
16. B. DE MOOR, M. MOONEN, L. VANDENBERGHE, AND J. VANDEWALLE, A geometrical approach for the identification of state space models with the singular valuedecomposition, in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, IEEE, Washington, DC, 1988, pp. 2244-2247.
17. D. L. DONOHO, Compressed sensing, IEEE Trans. Inform. Theory, 52 (2006), pp. 1289-1306.
18. L. ELDÉN, Matrix Methods in Data Mining and Pattern Recognition, Fundamentals of Algorithms 4, SIAM, Philadelphia, 2007.
19. M. FAZEL, Matrix Rank Minimization with Applications, Ph.D. thesis, Stanford University, Stanford, CA, 2002.
20. M. FAZEL, H. HINDI, AND S. BOYD, A rank minimization heuristic with application to minimum order system approximation, in Proceedings of the American Control Conference, Vol. 6, Arlington, VA, 2001, pp. 4734-4739.
21. M. FAZEL, H. HINDI, AND S. BOYD, Rank minimization and applications in system theory, in Proceedings of American Control Conference, Boston, MA, 2004, pp. 3273-3278.
22. K. FUJISAWA, M. KOJIMA, AND K. NAKATA, Exploiting sparsity in primal-dual interior-point methods for semidefinite programming, Math. Programming, 79 (1997), pp. 235-253.
23. Y. GENIN, Y. HACHEZ, YU. NESTEROV, AND P. VAN DOOREN, Optimization problems over positive pseudopolynomial matrices, SIAM J. Matrix Anal. Appl., 25 (2003), pp. 57-79.
24. T. HASTIE, R. TIBSHIRANI, AND J. FRIEDMAN, The Elements of Statistical Learning. Data Mining, Inference, and Prediction, Springer-Verlag, New York, 2001.
25. J.-B. HIRIART-URRUTY AND C. LEMARÉCHAL, Convex Analysis and Minimization Algorithms II: Advanced Theory and Bundle Methods, Grundlehren Math. Wiss. 306, Springer-Verlag, New York, 1993.
26. W. E. LARIMORE, Canonical variate analysis in identification, filtering, and adaptive control, in Proceedings of the 29th IEEE Conference on Decision and Control, Vol. 2, IEEE Control Systems Society, Piscataway, NJ, 1990, pp. 596-604.
27. L. LJUNG, System Identification: Theory for the User, 2nd ed., Prentice-Hall, Upper Saddle River, NJ, 1999.
28. S. MA, D. GOLDFARB, AND L. CHEN, Fixed point and Bregman iterative methods for matrix rank minimization, Math. Program. Ser. A, to appear.
29. I. MARKOVSKY, J. C. WILLEMS, S. VAN HUFFEL, B. DE MOOR, AND R. PINTELON, Application of structured total least squares for system identification and model reduction, IEEE Trans. Automat. Control, 50 (2005), pp. 1490-1500. (Pubitemid 41555599)
30. T. MORITA AND T. KANADE, A sequential factorization method for recovering shape and motion from image streams, IEEE Trans. Pattern Analysis and Machine Intelligence, 19 (1997), pp. 858-867. (Pubitemid 127762277)
31. Y. NESTEROV, Introductory Lectures on Convex Optimization, Kluwer Academic Publishers, Dordrecht, The Netherlands, 2004.
32. YU. NESTEROV, Smooth minimization of non-smooth functions, Math. Program. Ser. A, 103 (2005), pp. 127-152.
33. YU. E. NESTEROV AND M. J. TODD, Self-scaled barriers and interior-point methods for convex programming, Math. Oper. Res., 22 (1997), pp. 1-42.
34. P. VAN OVERSCHEE AND B. DE MOOR, N4SID: Subspace algorithms for the identification of combined deterministic-stochastic systems, Automatica J. IFAC, 30 (1994), pp. 75-93.
35. B. RECHT, M. FAZEL, AND P. A. PARRILO, Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, SIAM Rev., to appear.
36. B. RECHT, W. XU, AND B. HASSIBI, Necessary and sufficient conditions for success of the nuclear norm heuristic for rank minimization, in Proceedings of the 47th IEEE Conference on Decision and Control, IEEE Control Systems Society, Piscataway, NJ, 2008, pp. 3065-3070.
37. T. ROH AND L. VANDENBERGHE, Discrete transforms, semidefinite programming, and sum-ofsquares representations of nonnegative polynomials, SIAM J. Optim., 16 (2006), pp. 939-964.
38. N. Z. SHOR, Minimization Methods for Nondifferentiable Functions, Springer Ser. Comput. Math., Springer-Verlag, Berlin, 1985.
39. N. SREBRO, Learning with Matrix Factorizations, Ph.D. thesis, MIT, Cambridge, MA, 2004.
40. J. F. STURM, Using SEDUMI 1.02, a MATLAB toolbox for optimization over symmetric cones, Optim. Methods Softw., 11/12 (1999), pp. 625-653.
41. R. TIBSHIRANI, Regression shrinkage and selection via the lasso, J. Roy. Statist. Soc. Ser. B, 58 (1996), pp. 267-288.
42. M. J. TODD, K. C. TOH, AND R. H. TÜTÜNCÜ, On the Nesterov-Todd direction in semidefinite programming, SIAM J. Optim., 8 (1998), pp. 769-796.
43. C. TOMASI AND T. KANADE, Shape and motion from image streams under orthography: A factorization method, Internat. J. Computer Vision, 9 (1992), pp. 137-154.
44. J. A. TROPP, Just relax: Convex programming methods for identifying sparse signals in noise, IEEE Trans. Inform. Theory, 52 (2006), pp. 1030-1051.
45. P. TSENG, On accelerated proximal gradient methods for convex-concave optimization, manuscript, 2008.
46. R. H. TÜTÜNCÜ, K. C. TOH, AND M. J. TODD, Solving semidefinite-quadratic-linear programs using SDPT3, Math. Program. Ser. B, 95 (2003), pp. 189-217.
47. M. VERHAEGEN, Identification of the deterministic part of MIMO state space models given in innovations form from input-output data, Automatica J. IFAC, 30 (1994), pp. 61-74.
48. M. VIBERG, Subspace-based methods for the identification of linear time-invariant systems, Automatica J. IFAC, 31 (1995), pp. 1835-1851.
49. M. YAMASHITA, K. FUJISAWA, AND M. KOJIMA, Implementation and evaluation of SDPA 6.0 (Semidefinite Programming Algorithm 6.0), Optim. Methods Softw., 18 (2003), pp. 491-505.