Bilinear generalized approximate message passing - Part I: Derivation

IEEE Transactions on Signal Processing

Volumn 62, Issue 22, 2014, Pages 5839-5853

  Parker, Jason T ^a   Schniter, Philip ^b   Cevher, Volkan ^c  

^a AIR FORCE RESEARCH LABORATORY   (United States)

^b OHIO STATE UNIVERSITY   (United States)

^c EPFL   (Switzerland)

Author keywords

Approximate message passing; belief propagation; bilinear estimation; dictionary learning; matrix completion; matrix factorization; robust principal components analysis

Indexed keywords

BELIEF PROPAGATION; DICTIONARY LEARNING; MATRIX COMPLETION; MATRIX FACTORIZATIONS; ROBUST PRINCIPAL COMPONENTS;

EID: 84908439875     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2014.2357776     Document Type: Article

References (57)

1. J. T. Parker, P. Schniter, and V. Cevher, "Bilinear generalized approximate message passing-Part II: Applications," IEEE Trans. Signal Process., vol. 62, no. 22, pp. 5854-5867, 2014.
2. P. Schniter and V. Cevher, "Approximate message passing for bilinear models," in Proc. Workshop Signal Process. Adapt. Sparse Struct. Repr. (SPARS), Edinburgh, Scotland, Jun. 2011, pp. 68-68.
3. P. Schniter, J. T. Parker, and V. Cevher, "Bilinear generalized approximate message passing (BiG-AMP) for matrix recovery problems," presented at the Inf. Theory Appl. Workshop (ITA), La Jolla, CA, USA, Feb. 2012.
4. J. T. Parker and P. Schniter, "Bilinear generalized approximate message passing (BiG-AMP) for matrix completion," presented at the Asilomar Conf., Pacific Grove, CA, USA, Nov. 2012.
5. J. Cai, E. Candès, and Z. Shen, "A singular value thresholding algorithm for matrix completion," SIAM J. Optim., vol. 20, no. 4, pp. 1956-1982, 2010 [Online]. Available: http://epubs.siam.org/doi/ abs/10.1137/080738970
6. S. Ma, D. Goldfarb, and L. Chen, "Fixed point and bregman iterative methods for matrix rank minimization," Math. Programm., vol. 128, no. 1-2, pp. 321-353, 2011 [Online]. Available: http://dx.doi.org/10. 1007/s10107-009-0306-5
7. Z. Zhou, X. Li, J.Wright, E. Candès, and Y.Ma, "Stable principal component pursuit," in Proc. IEEE Int. Symp. Inf. Theory Process. (ISIT), Jun. 2010, pp. 1518-1522.
8. B. Recht, M. Fazel, and P. Parrilo, "Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization," SIAM Rev., vol. 52, no. 3, pp. 471-501, 2010 [Online]. Available: http:// epubs.siam.org/doi/abs/10.1137/070697835
9. Z. Lin, M. Chen, L. Wu, and Y. Ma, "The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices," arXiv preprint, arXiv:1009.5055 [Online]. Available: http://arxiv.org/abs/1009.5055
10. M. Tao and X. Yuan, "Recovering low-rank and sparse components of matrices from incomplete and noisy observations," SIAM J. Optim., vol. 21, no. 1, pp. 57-81, 2011.
11. T. Zhou and D. Tao, L. Getoor and T. Scheffer, Eds., "Godec: Randomized low-rank and sparse matrix decomposition in noisy case," in Proc. 28th Int. Conf. Mach. Learn. (ICML-11), Ser. ICML'11, ACM, New York, NY, USA, Jun. 2011, pp. 33-40.
12. H. Ghasemi, M. Malek-Mohammadi, M. Babaei-Zadeh, and C. Jutten, "SRF:Matrix completion based on smoothed rank function," presented at the IEEE Int. Conf. Acoust. Speech, Signal Process., Prague, Czech Republic, May 2011.
13. Z. Wen, W. Yin, and Y. Zhang, "Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm," Math. Programm. Comput., vol. 4, pp. 333-361, 2012 [Online]. Available: http://dx.doi.org/10.1007/s12532-012-0044-1
14. A. Kyrillidis and V. Cevher, "Matrix recipes for hard thresholding methods," J. Math. Imag. Vis., vol. 48, pp. 235-265, 2014.
15. G.Marjanovic and V. Solo, "On optimization and matrix completion," IEEE Trans. Signal Process., vol. 60, no. 11, pp. 5714-5724, 2012.
16. J. Haldar and D. Hernando, "Rank-constrained solutions to linear matrix equations using Power Factorization," IEEE Signal Process. Lett., vol. 16, no. 7, pp. 584-587, Jul. 2009.
17. L. Balzano, R. Nowak, and B. Recht, "Online identification and tracking of subspaces from highly incomplete information," in Proc. Allerton Conf. Commun. Control Comput., Sep. 2010, pp. 704-711.
18. R. Keshavan, A.Montanari, and S. Oh, "Matrix completion from a few entries," IEEE Trans. Inf. Theory, vol. 56, no. 6, pp. 2980-2998, Jun. 2010.
19. W. Dai and O. Milenkovic, "SET: An algorithm for consistent matrix completion," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing (ICASSP), 2010, pp. 3646-3649.
20. J. He, L. Balzano, and J. Lui, Online robust subspace tracking from partial information arXiv:1109.3827 2011.
21. R. Salakhutdinov and A. Mnih, "Probabilistic matrix factorization," Adv. Neural Inf. Process. Syst., vol. 20, pp. 1257-1264, 2008.
22. X. Ding, L. He, and L. Carin, "Bayesian robust principal component analysis," IEEE Trans. Image Process., vol. 20, no. 12, pp. 3419-3430, Dec. 2011.
23. Y. Lim and Y. Teh, "Variational Bayesian approach to movie rating prediction," in Proc. KDD Cup Workshop, 2007, pp. 15-21, Citeseer.
24. S. D. Babacan, M. Luessi, R. Molina, and A. K. Katsaggelos, "Sparse Bayesian methods for low-rankmatrix estimation," IEEE Trans. Signal Process., vol. 60, no. 8, pp. 3964-3977, Aug. 2012.
25. M. Tipping and C. Bishop, "Probabilistic principal component analysis," J. Roy. Statist. Soc., Series B (Statist. Methodol.), vol. 61, no. 3, pp. 611-622, 1999.
26. F. Léger, G. Yu, and G. Sapiro, "Efficient matrix completion with Gaussian models," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2011, pp. 1113-1116.
27. N.Wang, T. Yao, J.Wang, and D. Yeung, A. Fitzgibbon, Ed., "A probabilistic approach to robust matrix factorization," in Proc. Eur. Conf. Comput. Vis., 2012, vol. VII, pp. 126-139.
28. B.-H. Kim, A. Yedla, and H. Pfister, "Imp: A message-passing algorithm for matrix completion," in Proc. 6th Int. Symp. Turbo Codes Iterative Inf. Process. (ISTC), 2010, pp. 462-466.
29. B. J. Frey and D. J. C. MacKay, "A revolution: Belief propagation in graphs with cycles," in Proc. Neural Inf. Process. Syst. Conf., Denver, CO, USA, 1997, pp. 479-485.
30. D. L. Donoho, A. Maleki, and A. Montanari, "Message passing algorithms for compressed sensing," Proc. Nat. Acad. Sci., vol. 106, no. 45, pp. 18914-18919, Nov. 2009.
31. D. L. Donoho, A. Maleki, and A. Montanari, "Message passing algorithms for compressed sensing: I. motivation and construction," in Proc. Inf. Theory Workshop, Cairo, Egypt, Jan. 2010, pp. 1-5.
32. A. Montanari, "Graphical models concepts in compressed sensing," in Compressed Sensing: Theory and Applicant, Y. C. Eldar and G. Kutyniok, Eds. Cambridge, U.K.: Cambridge Univ. Press, 2012.
33. S. Rangan, "Generalized approximate message passing for estimation with randomlinearmixing," in Proc. IEEE Int. Symp. Inf. Theory, Saint Petersburg, Russia, Aug. 2011, pp. 2168-2172.
34. M. Bayati and A. Montanari, "The dynamics of message passing on dense graphs, with applications to compressed sensing," IEEE Trans. Inf. Theory, vol. 57, no. 2, pp. 764-785, Feb. 2011.
35. A. Javanmard and A. Montanari, "State evolution for general approximate message passing algorithms, with applications to spatial coupling," Inf. Inference, vol. 2, no. 2, pp. 115-144, 2013.
36. S. Rangan, P. Schniter, and A. Fletcher, "On the convergence of generalized approximate message passing with arbitrary matrices," presented at the IEEE Int. Symp. Inf. Theory, Honolulu, HI, USA, Jul. 2014.
37. A. Dempster, N. M. Laird, and D. B. Rubin, "Maximum-likelihood from incomplete data via the EM algorithm," J. Roy. Statist. Soc., vol. 39, pp. 1-17, 1977.
38. J. P. Vila and P. Schniter, "Expectation-maximization Gaussian-mixture approximate message passing," IEEE Trans. Signal Process., vol. 61, no. 19, pp. 4658-4672, Oct. 2013.
39. P. Schniter, "Turbo reconstruction of structured sparse signals," in Proc. Conf. Inf. Sci. Syst., Princeton, NJ, USA, Mar. 2010, pp. 1-6.
40. J. Vila, P. Schniter, and J. Meola, "Hyperspectral image unmixing via bilinear generalized approximate message passing," Proc. SPIE, vol. 8743, no. 87430Y, p. 9, 2013.
41. S. Rangan and A. K. Fletcher, "Iterative estimation of constrained rank-one matrices in noise," in Proc. IEEE Int. Symp. Inf. Theory, 2012, pp. 1246-1250.
42. F. Krzakala, M. Mézard, and L. Zdeborová, "Phase diagram and approximate message passing for blind calibration and dictionary learning," in Proc. IEEE Int. Symp. Inf. Theory, 2013, pp. 659-663.
43. J. Pearl, Probabilistic Reasoning in Intelligent Systems. SanMateo, CA, USA: Morgan Kaufmann, 1988.
44. F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, "Factor graphs and the sum-product algorithm," IEEE Trans. Inf. Theory, vol. 47, pp. 498-519, Feb. 2001.
45. G. F. Cooper, "The computational complexity of probabilistic inference using Bayesian belief networks," Artif. Intell., vol. 42, pp. 393-405, 1990.
46. W. T. Freeman, E. C. Pasztor, and O. T. Carmichael, "Learning lowlevel vision," Intl. J. Comput. Vis., vol. 40, no. 1, pp. 25-47, Oct. 2000.
47. R. J. McEliece, D. J. C. MacKay, and J.-F. Cheng, "Turbo decoding as an instance of pearl's 'belief propagation' algorithm," IEEE J. Sel. Areas Commun., vol. 16, no. 2, pp. 140-152, Feb. 1998.
48. D. J. C. MacKay, Information Theory, Inference, and Learning Algorithms. New York, NY, USA: Cambridge Univ. Press, 2003.
49. J. Boutros and G. Caire, "Iterative multiuser joint decoding: Unified framework and asymptotic analysis," IEEE Trans. Inf. Theory, vol. 48, no. 7, pp. 1772-1793, Jul. 2002.
50. J. T. Parker, V. Cevher, and P. Schniter, "Compressive sensing under matrix uncertainties: An approximate message passing approach," in Proc. Asilomar Conf. Signals, Syst., Comput., Pacific Grove, CA, USA, Nov. 2011, pp. 804-808.
51. P. Schniter and S. Rangan, "Compressive phase retrieval via generalized approximate message passing," presented at the Allerton Conf. Commun., Control, Comput., Monticello, IL, USA, Oct. 2012.
52. S. Rangan et al., GAMPmatlab [Online]. Available: https://sourceforge. net/projects/gampmatlab/
53. S. Wright, R. Nowak, and M. Figueiredo, "Sparse reconstruction by separable approximation," IEEE Trans. Signal Process., vol. 57, no. 7, pp. 2479-2493, Jul. 2009.
54. S. Rangan, P. Schniter, E. Riegler, A. Fletcher, and V. Cevher, "Fixed points of generalized approximate message passing with arbitrary matrices," in Proc. IEEE Int. Symp. Inf. Theory, Istanbul, Turkey, Jul. 2013, pp. 664-668.
55. R. Neal and G. Hinton, "A view of the EM algorithm that justifies incremental, sparse, and other variants," in Learning in Graphical Models, M. I. Jordan, Ed. Cambridge, MA, USA:MIT Press, 1999, pp. 355-368.
56. P. Stoica andY. Selen, "Model-order selection: a review of information criterion rules," IEEE Signal Process. Mag., vol. 21, no. 4, pp. 36-47, Jul. 2004.
57. S. Rangan, "Generalized approximate message passing for estimation with random linear mixing," Aug. 2012, arXiv:1010.5141v2.