Distributed covariance estimation in Gaussian graphical models

IEEE Transactions on Signal Processing

Volumn 60, Issue 1, 2012, Pages 211-220

  Wiesel, Ami ^a   Hero III, Alfred O ^b  

^a HEBREW UNIVERSITY OF JERUSALEM   (Israel)

^b UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Covariance estimation; distributed signal processing; graphical models

Indexed keywords

BELIEF PROPAGATION; COMMUNICATION COST; COVARIANCE ESTIMATION; DISTRIBUTED ESTIMATION; DISTRIBUTED SIGNAL PROCESSING; GAUSSIANS; GRAPHICAL MODEL; INVERSE COVARIANCE; LOCAL COVARIANCE; LOCAL MAXIMUM LIKELIHOOD; MATRIX; MAXIMUM LIKELIHOOD APPROACHES; MULTIVARIATE DISTRIBUTIONS; NUMERICAL EXPERIMENTS; POTENTIAL FUNCTION; PSEUDO-LIKELIHOOD; REAL WORLD DATA; SYMMETRY CONSTRAINTS; SYNTHETIC DATA;

COVARIANCE MATRIX; ESTIMATION; GRAPHIC METHODS; MESSAGE PASSING; NUMERICAL METHODS; PROCESSING; SIGNAL PROCESSING;

MAXIMUM LIKELIHOOD ESTIMATION;

EID: 84555218018     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2011.2172430     Document Type: Article

References (43)

1. H. Krim and M. Viberg, "Two decades of array signal processing research: the parametric approach," IEEE Signal Process. Mag., vol. 13, no. 4, pp. 67-94, Jul. 1996.
2. E. Dougherty, A. Datta, and C. Sima, "Research issues in genomic signal processing," IEEE Signal Process. Mag., vol. 22, no. 6, pp. 46-68, Nov. 2005.
3. M. Cetin, L. Chen, J. W. Fisher, A. T. Ihler, R. L. Moses, M. J. Wainwright, and A. S. Willsky, "Distributed fusion in sensor networks: A graphical models perspective," IEEE Signal Process. Mag., vol. 23, no. 4, pp. 42-55, Jul. 2006.
4. A. Wiesel and A. O. Hero, "Decomposable principal component analysis," IEEE Trans. Signal Process., vol. 57, no. 11, pp. 4369-4377, Nov. 2009.
5. V. Delouille, R. Neelamani, and R. Baraniuk, "Robust distributed estimation using the embedded subgraphs algorithm," IEEE Trans. Signal Process., vol. 54, no. 8, pp. 2998-3010, 2006.
6. Y. Weiss and W. Freeman, "Correctness of belief propagation in Gaussian graphical models of arbitrary topology," Neural Computat., vol. 13, no. 10, pp. 2173-2200, 2001.
7. V. Chandrasekaran, J. Johnson, and A. Willsky, "Estimation in Gaussian graphical models using tractable subgraphs: A walk-sum analysis," IEEE Trans. Signal Process., vol. 56, no. 5, p. 1916, 2008.
8. J. Johnson, D. Malioutov, and A. Willsky, "Walk-sum interpretation and analysis of Gaussian belief propagation," Adv. in Neural Inf. Process. Syst., vol. 18, p. 579, 2006.
9. E. Sudderth, M. Wainwright, and A. Willsky, "Embedded trees: Estimation of Gaussian processes on graphs with cycles," IEEE Trans. Signal Process., vol. 52, no. 11, pp. 3136-3150, Nov. 2004.
10. D. M. Malioutov, J. Johnson, and A. Willsky, "Walk-sums and belief propagation in Gaussian graphical models," J. Mach. Learn. Res., vol. 7, pp. 2031-2064, Oct. 2006.
11. S. L. Lauritzen, Graphical models. New York: Oxford Statist. Sci. Ser., 1996, vol. 17.
12. A. P. Dempster, "Covariance selection," Biometrics, vol. 28, pp. 157-175, 1972.
13. A. Wiesel, Y. C. Eldar, and A. O. Hero, "Covariance estimation in decomposable Gaussian graphical models," IEEE Trans. Signal Process., vol. 58, no. 3, pp. 1482-1492, Mar. 2010.
14. J. Dahl, L. Vandenberghe, and V. Roychowdhury, "Covariance selection for nonchordal graphs via chordal embedding," Optimiz. Meth. Softw., vol. 23, no. 4, pp. 501-520, 2008.
15. L. Vandenberghe, S. Boyd, and S. P. Wu, "Determinant maximization with linear matrix inequality constraints," SIAM J. Matrix Anal. Appl., vol. 19, no. 2, pp. 499-533, 1998.
16. Y. Teh and M. Welling, "On improving the efficiency of the iterative proportional fitting procedure," in Proc. Int. Workshop on Artif. Intell. Statist., 2003, vol. 9.
17. D.Koller and N. Friedman, Probabilistic Graphical Models: Principles and Techniques. Cambridge, MA: The MIT Press, 2009.
18. D. Malioutov, J. Johnson, M. J. Choi, and A. Willsky, "Low-rank variance approximation in GMRF models: Single and multiscale approaches," IEEE Trans. Signal Process., vol. 56, no. 10, pp. 4621-4634, Oct. 2008.
19. P. Liang and M. Jordan, "An asymptotic analysis of generative, discriminative, and pseudolikelihood estimators," in Proc. 25th Int. Conf. Mach. Learn., 2008, pp. 584-591.
20. J. Besag, "Statistical analysis of non-lattice data," in The Statistician, 1975, pp. 179-195.
21. J. V. Dillon and G. Lebanon, Statistical and Computational Tradeoffs in Stochastic Composite Likelihood, 2010 [Online]. Available: http:// arxiv.org/abs/1003.0691
22. B. Lindsay, "Composite likelihood methods," Contemp. Math., vol. 80, no. 1, pp. 221-39, 1988.
23. M. Wainwright, "Estimating the wrong graphical model: Benefits in the computation-limited setting," J. Mach. Learn. Res., vol. 7, pp. 1829-1859, 2006.
24. G. Rocha, P. Zhao, and B. Yu, A Path Following Algorithm for Sparse Pseudo-Likelihood Inverse Covariance Estimation (Splice), 2008 [Online]. Available: http://arxiv.org/abs/0807.3734
25. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers, Stanford Univ., Stanford, CA, 2010.
26. D. Bertsekas and J. Tsitsiklis, Parallel and distributed computation: numerical methods. New York: Athena Scientific, 1997.
27. Q. Ling and Z. Tian, "Decentralized sparse signal recovery for compressive sleeping wireless sensor networks," IEEE Trans. Signal Process., vol. PP, no. 99, p. 1, 2010.
28. I. Schizas, G. Mateos, and G. Giannakis, "Distributed LMS for consensus- based in-network adaptive processing," IEEE Trans. Signal Process., vol. 57, no. 6, pp. 2365-2382, Jun. 2009.
29. I. D. Schizas, G. B. Giannakis, and Z. Q. Luo, "Distributed estimation using reduced-dimensionality sensor observations," IEEE Trans. Signal Process., vol. 55, no. 8, pp. 4284-4299, Aug. 2007.
30. I. Schizas, A. Ribeiro, and G. Giannakis, "Consensus in ad hoc WSNs with noisy links-Part I: Distributed estimation of deterministic signals," IEEE Trans. Signal Process., vol. 56, no. 1, pp. 350-364, 2008.
31. G. Mateos, J. Bazerque, and G. Giannakis, "Distributed sparse linear regression," IEEE Trans. Signal Process., vol. 58, no. 10, pp. 5262-5276, 2010.
32. C. Guestrin, P. Bodik, R. Thibaux, M. Paskin, and S. Madden, "Distributed regression: An efficient framework for modeling sensor network data," in Proc. 3rd Int. Symp. Inf. Process. Sens. Netw., 2004, pp. 1-10.
33. A. Deshpande, C. Guestrin, S. Madden, J. Hellerstein, and W. Hong, "Model-driven data acquisition in sensor networks," in Proc. 30th Int. Conf. Very large Data Bases, 2004, vol. 30, pp. 588-599.
34. N. Meinshausen and P. Buhlmann, "High-dimensional graphs and variable selection with the Lasso," Ann. Statist., vol. 34, no. 3, pp. 1436-1462, 2006.
35. A. Rothman, P. Bickel, E. Levina, and J. Zhu, "Sparse permutation invariant covariance estimation," Electron. J. Statist., vol. 2, pp. 494-515, 2008.
36. P. Ravikumar, M. Wainwright, G. Raskutti, and B. Yu, "High- dimensional covariance estimation by minimizing L1-penalized log-determinant divergence," in Adv. Neural Inf. Process. Syst., 2008.
37. C. Lam and J. Fan, "Sparsistency and rates of convergence in large covariance matrix estimation," Ann. Statist., vol. 37, no. 6B, pp. 4254-4278, 2009.
38. P. Bickel and E. Levina, "Regularized estimation of large covariance matrices," Ann. Statist., vol. 36, no. 1, pp. 199-227, 2008.
39. J. Johnson, "Fisher information in Gaussian graphical models," Mass. Inst. of Technol., Cambridge, MA, Tech. Rep., 2006, unpublished.
40. S. J. Press, Applied Multivariate Analysis. New York: Holt, Rinehart and Winston, 1972.
41. P. Huber and E. Ronchetti, Robust Statistics. New York:Wiley, 2009.
42. J. Gorman and A. Hero, "Lower bounds for parametric estimation with constraints," IEEE Trans. Inf. Theory, vol. 36, no. 6, pp. 1285-1301, 1990.
43. K.Werner,M. Jansson, and P. Stoica, "On estimation of covariance matrices with Kronecker product structure," IEEE Trans. Signal Process., vol. 56, no. 2, pp. 478-491, Feb. 2008.