Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses

Annals of Statistics

Volumn 41, Issue 6, 2013, Pages 3022-3049

  Loh, Po Ling ^a   Wainwright, Martin J ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Exponential families; Graphical models; High dimensional statistics; Inverse covariance estimation; Legendre duality; Markov random fields; Model selection

Indexed keywords

EID: 84891906007     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/13-AOS1162     Document Type: Article

References (40)

1. AGARWAL, A., NEGAHBAN, S. and WAINWRIGHT, M. J. (2012). Fast global convergence of gradient methods for high-dimensional statistical recovery. Ann. Statist. 40 2452-2482. MR3097609
2. ANANDKUMAR, A., TAN, V. Y. F., HUANG, F. and WILLSKY, A. S. (2012). Highdimensional structure estimation in Ising models: Local separation criterion. Ann. Statist. 40 1346-1375. MR3015028
3. BANERJEE, O., EL GHAOUI, L. and D'ASPREMONT, A. (2008). Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. J. Mach. Learn. Res. 9 485-516. MR2417243
4. BARNDORFF-NIELSON, O. E. (1978). Information and Exponential Families. Wiley, Chichester.
5. BESAG, J. (1974). Spatial interaction and the statistical analysis of lattice systems. J. R. Stat. Soc. Ser. B Stat. Methodol. 36 192-236. MR0373208
6. BRESLER, G., MOSSEL, E. and SLY, A. (2008). Reconstruction of Markov random fields from samples: Some observations and algorithms. In Approximation, Randomization and Combinatorial Optimization. 343-356. Springer, Berlin. MR2538799
7. BROWN, L. D. (1986). Fundamentals of Statistical Exponential Families. IMS, Hayward, CA. MR0882001
8. CAI, T., LIU, W. and LUO, X. (2011). A constrained ℓ1 minimization approach to sparse precision matrix estimation. J. Amer. Statist. Assoc. 106 594-607. MR2847973
9. CARROLL, R. J., RUPPERT, D. and STEFANSKI, L. A. (1995). Measurement Error in Nonlinear Models. Chapman & Hall, London. MR1630517
10. CHOW, C. I. and LIU, C. N. (1968). Approximating discrete probability distributions with dependence trees. IEEE Trans. Inform. Theory 14 462-467.
11. DARROCH, J. N. and SPEED, T. P. (1983). Additive and multiplicative models and interactions. Ann. Statist. 11 724-738. MR0707924
12. D'ASPREMONT, A., BANERJEE, O. and EL GHAOUI, L. (2008). First-order methods for sparse covariance selection. SIAM J. Matrix Anal. Appl. 30 56-66. MR2399568
13. DEMPSTER, A. P., LAIRD, N. M. and RUBIN, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B Stat. Methodol. 39 1-38. MR0501537
14. FRIEDMAN, J., HASTIE, T. and TIBSHIRANI, R. (2008). Sparse inverse covariance estimation with the graphical Lasso. Biostatistics 9 432-441.
15. GRIMMETT, G. R. (1973). A theorem about random fields. Bull. Lond. Math. Soc. 5 81-84. MR0329039
16. HORN, R. A. and JOHNSON, C. R. (1990). Matrix Analysis. Cambridge Univ. Press, Cambridge. MR1084815
17. IBRAHIM, J. G., CHEN, M.-H., LIPSITZ, S. R. and HERRING, A. H. (2005). Missing-data methods for generalized linear models: A comparative review. J. Amer. Statist. Assoc. 100 332-346. MR2166072
18. JACOB, L., OBOZINSKI, G. and VERT, J. P. (2009). Group Lasso with overlap and graph Lasso. In International Conference on Machine Learning (ICML) 433-440. ACM, New York.
19. JALALI, A., RAVIKUMAR, P. D., VASUKI, V. and SANGHAVI, S. (2011). On learning discrete graphical models using group-sparse regularization. Journal of Machine Learning Research-Proceedings Track 15 378-387.
20. KOLLER, D. and FRIEDMAN, N. (2009). Probabilistic Graphical Models: Principles and Techniques. MIT Press, Cambridge. MR2778120
21. LAURITZEN, S. L. (1996). Graphical Models. Oxford Univ. Press, New York. MR1419991
22. LAURITZEN, S. L. and SPIEGELHALTER, D. J. (1988). Local computations with probabilities on graphical structures and their application to expert systems. J. R. Stat. Soc. Ser. B Stat. Methodol. 50 157-224. MR0964177
23. LIU, H., HAN, F., YUAN, M., LAFFERTY, J. and WASSERMAN, L. (2012). Highdimensional semiparametric Gaussian copula graphical models. Ann. Statist. 40 2293-2326. MR3059084
24. LIU, H., LAFFERTY, J. and WASSERMAN, L. (2009). The nonparanormal: Semiparametric estimation of high dimensional undirected graphs. J. Mach. Learn. Res. 10 2295-2328. MR2563983
25. LOH, P.-L. and WAINWRIGHT, M. J. (2012). High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity. Ann. Statist. 40 1637-1664. MR3015038
26. LOH, P. and WAINWRIGHT, M. J. (2013). Supplement to "Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses." DOI:10.1214/13-AOS1162SUPP.
27. MEINSHAUSEN, N. and BÜHLMANN, P. (2006). High-dimensional graphs and variable selection with the lasso. Ann. Statist. 34 1436-1462. MR2278363
28. NEWMAN, M. E. J. and WATTS, D. J. (1999). Scaling and percolation in the small-world network model. Phys. Rev. E (3) 60 7332-7342.
29. OBOZINSKI, G., WAINWRIGHT, M. J. and JORDAN, M. I. (2011). Support union recovery in high-dimensional multivariate regression. Ann. Statist. 39 1-47. MR2797839
30. RAVIKUMAR, P., WAINWRIGHT, M. J. and LAFFERTY, J. D. (2010). High-dimensional Ising model selection using ℓ1-regularized logistic regression. Ann. Statist. 38 1287-1319. MR2662343
31. RAVIKUMAR, P., WAINWRIGHT, M. J., RASKUTTI, G. and YU, B. (2011). High-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence. Electron. J. Stat. 5 935-980. MR2836766
32. ROCKAFELLAR, R. T. (1970). Convex Analysis. Princeton Univ. Press, Princeton, NJ. MR0274683
33. ROTHMAN, A. J., BICKEL, P. J., LEVINA, E. and ZHU, J. (2008). Sparse permutation invariant covariance estimation. Electron. J. Stat. 2 494-515. MR2417391
34. RUBIN, D. B. (1987). Multiple Imputation for Nonresponse in Surveys. Wiley, New York. MR0899519
35. SANTHANAM, N. P. and WAINWRIGHT, M. J. (2012). Information-theoretic limits of selecting binary graphical models in high dimensions. IEEE Trans. Inform. Theory 58 4117-4134. MR2943079
36. WAINWRIGHT, M. J. and JORDAN, M. I. (2008). Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn. 1 1-305. ISSN 1935-8237.
37. XUE, L. and ZOU, H. (2012). Regularized rank-based estimation of high-dimensional nonparanormal graphical models. Ann. Statist. 40 2541-2571. MR3097612
38. YUAN, M. (2010). High dimensional inverse covariance matrix estimation via linear programming. J. Mach. Learn. Res. 11 2261-2286. MR2719856
39. YUAN, M. and LIN, Y. (2007). Model selection and estimation in the Gaussian graphical model. Biometrika 94 19-35. MR2367824
40. ZHAO, P. and YU, B. (2006). On model selection consistency of Lasso. J. Mach. Learn. Res. 7 2541-2563. MR2274449