An inexact interior point method for L1-regularized sparse covariance selection

Mathematical Programming Computation

Volumn 2, Issue 3-4, 2010, Pages 291-315

  Li, Lu ^a   Toh, Kim Chuan ^a,b  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^b SINGAPORE MIT ALLIANCE   (Singapore)

Author keywords

Inexact interior point method; Inexact search direction; Iterative solver; Log determinant semidefinite programming; Sparse inverse covariance selection

Indexed keywords

INEXACT SEARCH DIRECTION; INTERIOR POINT METHODS; INVERSE COVARIANCE; ITERATIVE SOLVERS; SEMI-DEFINITE PROGRAMMING;

ALGORITHMS; LINEAR SYSTEMS; MATHEMATICAL PROGRAMMING;

ITERATIVE METHODS;

EID: 84860633052     PISSN: 18672949     EISSN: 18672957     Source Type: Journal    
DOI: 10.1007/s12532-010-0020-6     Document Type: Article

References (48)

1. Banerjee, O., El Ghaoui, L., d'Aspremont, A.: Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. J. Mach. Learn. Res. 9, 485-516 (2008)
2. Bilmes, J.A.: Natural statistical models for automatic speech recognition. PhD thesis, University of California, Berkeley (1999)
3. Burer, S., Monteiro, R.D.C., Zhang, Y.: A computational study of a gradient-based log-barrier algorithm for a class of large-scale SDPs. Math. Program. 95, 359-379 (2003)
4. Chen, S.S., Gopinath, R.A.: Model selection in acoustic modeling. In: Proc. EUROSPEECH'99, pp. 1087-1090, Budapest, Hungary (1999)
5. Dahl, J., Vandenberghe, L., Roychowdhury, V.: Covariance selection for nonchordal graphs via chordal embedding. Optim. Methods Softw. 23, 501-520 (2008)
6. d'Aspremont, A.: Identifying small mean reverting portfolios. Quant. Finance (2010, to appear)
7. d'Aspremont, A., Banerjee, O., El Ghaoui, L.: First-order methods for sparse covariance selection. SIAM J. Matrix Anal. Appl. 30, 56-66 (2008)
8. Dempster, A.P.: Covariance selection. Biometrics 28, 157-175 (1972)
9. Dobra, A.: Variable selection and dependency networks for genomewide data. Biostatistics 10, 621-639 (2009)
10. Edwards, D.: Introduction to graphical modelling, 2nd edn. Springer, New York (2000)
11. Fan, J., Feng, Y., Wu, Y.: Network exploration via the adaptive LASSO and SCAD penalties. Ann. Appl. Stat. 3, 521-541 (2009)
12. Freund, R., Nachtigal, N.: A new Krylov-subspace method for symmetric indefinite linear system. In: Proceedings of the 14th IMACSWorld Congress on Computational and Applied Mathematics, Atlanta, USA, pp. 1253-1256 (1994)
13. Friedman, J., Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9, 432-441 (2008)
14. Golub, T.R., Slonim, D.K., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J.P., Coller, H., Loh, M.L., Downing, J.R., Caligiuri, M.A., Bloomfield, C.D.: Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286, 531-537 (1999)
15. Hedenfalk, I., Duggan, D., Chen, Y., Radmacher, M., Bittner, M., Simon, R., Meltzer, P., Gusterson, B., Esteller, M., Raffeld, M., Yakhini, Z., Ben-Dor, A., Dougherty, E., Kononen, J., Bubendorf, L., Fehrle, W., Pittaluga, S., Gruvberger, S., Loman, N., Johannsson, O., Olsson, H., Wilfond, B., Sauter, G., Kallioniemi, O.-P., Borg, A., Trent, J.: Gene-expression profiles in hereditary breast cancer. N. Engl. J. Med. 344, 539-548 (2001)
16. Jarre, F., Rendl, F.: An augmented primal-dual method for linear conic programs. SIAM J. Optim. 19, 808-823 (2008)
17. Krishnamurthy, V., d'Aspremont, A.: A pathwise algorithm for covariance selection. Preprint (2009)
18. Lan, G., Lu, Z., Monterio, R.D.: Primal-dual first-order methods withO(1/ε) iteration-complexity for cone programming. Math. Program. (2010, to appear)
19. Lauritzen, S.L.: Graphical models. In: Oxford Statistical Science Series, vol. 17. The Clarendon Press/Oxford University Press/Oxford Science Publications, New York (1996)
20. Lu, Z.: Smooth optimization approach for sparse covariance selection. SIAM J. Optim. 19, 1807-1827 (2008)
21. Lu, Z.: Adaptive first-order methods for general sparse inverse covariance selection. SIAM J. Matrix Anal. Appl. 31, 2000-2016 (2010)
22. Meinshausen, N., Bühlmann, P.: High-dimensional graphs and variable selection with the lasso. Ann. Stat. 34, 1436-1462 (2006)
23. Nesterov, Y.: Smooth minimization of non-smooth functions. Math. Program. 103, 127-152 (2005)
24. Nesterov, Y., Todd, M.J.: Primal-dual interior-point methods for self-scaled cones. SIAM J. Optim. 8, 324-364 (1998)
25. Pittman, J., Huang, E., Dressman, H., Horng, C.-F., Cheng, S.H., Tsou, M.-H., Chen, C.-M., Bild, A., Iversen, E.S., Huang, A.T., Nevins, J.R., West, M.: Integrated modeling of clinical and gene expression information for personalized prediction of disease outcomes. Proc. Natl. Acad. Sci. USA 101(22), 8431-8436 (2004)
26. Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1, 97-116 (1976)
27. Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. Society for Industrial and Applied Mathematics, Philadelphia (2003)
28. Sachs, K., Perez, O., Pe'er, D., Lauffenburger, D.A., Nolan, G.P.: Causal protein-signaling networks derived from multiparameter single-cell data. Science 308, 523-529 (2005)
29. Scheinberg, K., Rish, I.: Learning sparse Gaussian Markov networks using a greedy coordinate ascent approach. In: Balcázar, J., Bonchi, F., Gionis, A., Sebag, M. (eds.) Machine Learning and Knowledge Discovery in Databases, Lecture Notes in Computer Science 6323. pp. 196-212 (2010)
30. Storey, J.D., Tibshirani, R.: Statistical significance for genome-wide studies. Proc. Natl. Acad. Sci. USA 100(16), 9440-9445 (2003)
31. Sturm, J.F.: Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones. Optim. Methods Softw. 11/12, 625-653 (1999)
32. Toh, K.-C.: Solving large scale semidefinite programs via an iterative solver on the augmented systems. SIAM J. Optim. 14, 670-698 (2003)
33. Toh, K.-C.: An inexact primal-dual path following algorithm for convex quadratic SDP. Math. Program. 112, 221-254 (2008)
34. Toh, K.-C., Todd, M.J., Tütüncü, R.H.: SDPT3-a MATLAB software package for semidefinite programming, version 1.3. Optim. Methods Softw. 11/12, 545-581 (1999)
35. Tsuchiya, T., Xia, Y.: An extension of the standard polynomial-time primal-dual path-following algorithm to the weighted determinant maximization problem with semidefinite constraints. Pac. J. Optim. 3, 165-182 (2007)
36. Ueno, U., Tsuchiya, T.: Covariance regularization in inverse space. Q. J. R. Meteorol. Soc. 135, 1133-1156 (2009)
37. Vandenberghe, L., Boyd, S., Wu, S.-P.: Determinant maximization with linear matrix inequality constraints. SIAM J. Matrix Anal. Appl. 19, 499-533 (1998)
38. Wang, C., Sun, D., Toh, K.-C.: Solving log-determinant optimization problems by a newton-cg proximal point algorithm. SIAM J. Optim. 20, 2994-3013 (2010)
39. Whittaker, J.: Graphical models in applied multivariate statistics. In: Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. Wiley, Chichester (1990)
40. Wille, A., Zimmermann, P., Vranová, E., Fürholz, A., Laule, O., Bleuler, S., Hennig, L., Prelíc, A., von Rohr, P., Thiele, L., Zitzler, E., Gruissem, W., Bühlmann, P.: Sparse graphical gaussian modeling of the isoprenoid gene network in Arabidopsis thaliana. Genome Biol. 5, R92 (2004)
41. Wong, F., Carter, C.K., Kohn, R.: Efficient estimation of covariance selection models. Biometrika 90, 809-830 (2003)
42. Wu, W.B., Pourahmadi, M.: Nonparameteric estimation of large covariance matrices of longitudinal data. Biometrika 90, 831-844 (2003)
43. Yeung, K.Y., Bumgarner, R.E., Raftery, A.E.: Bayesian model averaging: development of an improved multi-class, gene selection and classification tool for microarray data. Bioinformatics 21, 2394-2402 (2005)
44. Yuan, M., Lin, Y.: Model selection and estimation in the Gaussian graphical model. Biometrika 94, 19-35 (2007)
45. Yuan, X.: Alternating direction methods for sparse covariance selection. Preprint (2009)
46. Zhang, Y.: On extending some primal-dual interior-point algorithms from linear programming to semidefinite programming. SIAM J. Optim. 8, 365-386 (1998)
47. Zhao, X.Y., Sun, D., Toh, K.-C.: A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim. 20, 1737-1765 (2010)
48. Zhou, G., Toh, K.-C.: Polynomiality of an inexact infeasible interior point algorithm for semidefinite programming. Math. Program. 99, 261-282 (2004)