Merging belief propagation and the mean field approximation: A free energy approach

IEEE Transactions on Information Theory

Volumn 59, Issue 1, 2013, Pages 588-602

  Riegler, Erwin ^a   Kirkelund, Gunvor Elisabeth ^b   Manchón, Carles Navarro ^b   Badiu, Mihai Alin ^c   Fleury, Bernard Henri ^b  

^a VIENNA UNIVERSITY OF TECHNOLOGY   (Austria)

^b AALBORG UNIVERSITY   (Denmark)

^c TECHNICAL UNIVERSITY OF CLUJ NAPOCA   (Romania)

Author keywords

Belief propagation (BP); iterative algorithms; iterative decoding; message passing; parameter estimation

Indexed keywords

APPROXIMATION METHODS; BELIEF PROPAGATION; ENERGY APPROACH; FACTOR GRAPHS; FIXED POINT EQUATION; HARD CONSTRAINTS; ITERATIVE ALGORITHM; ITERATIVE CHANNEL ESTIMATION; MEAN FIELD APPROXIMATION; ORTHOGONAL FREQUENCY DIVISION MULTIPLEXING SYSTEMS; REGION-BASED; STATIONARY POINTS; TECHNICAL CONDITIONS; UNDERLYING FACTORS;

ALGORITHMS; APPROXIMATION THEORY; FREE ENERGY; MESSAGE PASSING; MULTIPLEXING EQUIPMENT; ORTHOGONAL FREQUENCY DIVISION MULTIPLEXING; PARAMETER ESTIMATION;

ITERATIVE DECODING;

EID: 84871805061     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2012.2218573     Document Type: Article

References (40)

1. E. Riegler, G. E. Kirkelund, C. N. Manchón, and B. H. Fleury, "Merging belief propagation and the mean field approximation: A free energy approach," in Proc. IEEE Symp. Turbo Codes, Brest, France, Sep. 2010, pp. 256-260.
2. P. Parisi, Statistical Field Theory. Oxford, U.K.: Perseus, 1988.
3. Z. Ghahramani and M. J. Beal, Graphical Models and Variational Methods, D. Saad and M. Opper, Eds. Cambridge, MA: MIT Press, 2000.
4. M. I. Jordan, Z. Ghahramani, T. S. Jaakola, and L. K. Saul, "An introduction to variational methods for graphical models," J. Mach. Learn., vol. 37, no. 2, pp. 183-233, 1999.
5. J.Winn and C. Bishop, "Variationalmessage passing," J. Mach. Learn., vol. 6, no. 6, pp. 661-694, 2005.
6. C. M. Bishop, Pattern Recognition and Machine Learning. New York: Springer-Verlag, 2006.
7. G. E.Kirkelund, C.N. Manchón, L. P. B. Christensen, E. Riegler, and B. H. Fleury, "Variational message-passing for joint channel estimation and decoding in MIMO-OFDM," in Proc. IEEE Global Telecom. Conf., Miami, FL, Dec. 2010, pp. 1-6.
8. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Francisco, CA: Morgan Kaufmann, 1998.
9. J. S. Yedidia,W. T. Freeman, andY.Weiss, "Constructing free-energy approximations and generalized belief propagation algorithms," IEEE Trans. Inf. Theory, vol. 51, no. 7, pp. 2282-2312, Jul. 2005.
10. F. R. Kschischang, J. F. Brendan, and H.-A. Loeliger, "Factor graphs and the sum-product algorithm," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 498-519, Feb. 2001.
11. M. J. Wainwright, T. S. Jaakkola, and A. S. Willsky, "A new class of upper bounds on the log partition function," IEEE Trans. Inf. Theory, vol. 51, no. 7, pp. 2313-2335, Jul. 2005.
12. H. Wymeersch, F. Penna, and V. Savic, "Uniformly reweighted belief propagation: A factor graph approach," in Proc. IEEE Int. Symp. Inf. Theory, St. Petersburg, Russia, Jul./Aug. 2011, pp. 2000-2004.
13. S. Ikeda, T. Tanaka, and S.-I. Amari, "Information geometry of turbo and low-density parity-check codes," IEEE Trans. Inf. Theory, vol. 50, no. 6, pp. 1097-1114, Jun. 2004.
14. S.-I. Amari and H. Nagaoka, Methods of Information Geometry. Oxford, U.K.: Oxford Univ. Press, 2000.
15. M. Moher and T. A. Gulliver, "Cross-entropy and iterative decoding," IEEE Trans. Inf. Theory, vol. 44, no. 7, pp. 3097-3104, Nov. 1998.
16. J. M. Walsh, C. R. Johnson, and P. A. Regalia, "A refined information geometric interpretation of turbo decoding," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Philadelphia, PA, Mar. 2005, pp. 481-484.
17. T. Minka, Divergence measures and message passing 2005, Microsoft Res. Tech. Rep. MSR-TR-2005-173.
18. X. Wang and H. V. Poor, "Iterative (turbo) soft interference cancellation and decoding for coded CDMA," IEEE Trans. Commun., vol. 47, no. 7, pp. 1046-1061, Jul. 1999.
19. J. Wehinger and C. F. Mecklenbrauker, "Iterative CDMA multiuser receiver with soft decision-directed channel estimation," IEEE Trans. Signal Process., vol. 54, no. 10, pp. 3922-3934, Oct. 2006.
20. S. P. Rossi and R. R. Müller, "Joint twofold-iterative channel estimation and multiuser detection for MIMO-OFDM systems," IEEE Trans. Wireless Commun., vol. 7, no. 11, pp. 4719-4729, Nov. 2008.
21. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum-likelihood from incomplete data via the EM algorithm," J. Roy. Statist. Soc., vol. 39, no. 1, pp. 1-38, 1977.
22. S. Kullback, Information Theory and Statistics. New York: Wiley, 1978.
23. C. N. Manchón, G. E. Kirkelund, E. Riegler, L. Christensen, and B. H. Fleury, Receiver architectures forMIMO-OFDMbased on a combined VMP-SP algorithm 2011, arXiv: 1111.5848.
24. M.-A Badiu, C. N. Manchón, V. Bota, and B. H. Fleury, "Distributed iterative processing for interference channels with receiver cooperation," in Proc. IEEE Symp. Turbo Codes, Gothenburg, Sweden, Aug. 2012, pp. 1-5.
25. T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. New York: Wiley, 2006.
26. H. A. Bethe, "Statistical theory of superlattices," Proc. Roy. Soc. London, ser. A, vol. 150, no. 871, pp. 552-575, 1935.
27. T. Heskes, "Stable fixed points of loopy belief propagation are minima of the Bethe free energy," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2003, pp. 343-350.
28. D. P. Bertsekas, Nonlinear Programming, 2nd ed. Belmont, MA: Athena Scientific, 2003.
29. J. Dauwels, "On variational message passing on factor graphs," in Proc. IEEE Int. Symp. Inf. Theory, Nice, France, Jun. 2007, pp. 2546-2550.
30. T. Tanaka, "Information geometry of mean-field approximation," Neural Comput., vol. 12, pp. 1951-1968, 2000.
31. A. W. Eckford, "The factor graph EM algorithm: Applications for LDPC codes," in Proc. IEEE SPAWC, New York, Jun. 2005, pp. 910-914.
32. J. Dauwels, S. Korl, and H.-A. Loeliger, "Expectation maximization as message passing," in Proc. IEEE Int. Symp. Inf. Theory, Adelaide, Australia, Sep. 2005, pp. 1-4.
33. B. Hu, "A variational Bayesian framework divergence minimization and its application in CDMA receivers," Ph.D. dissertation, Dept. Electron. Syst., Aalborg Univ., Aalborg, Denmark, 2010.
34. C. Knievel, P. A. Hoeher, A. Tyrrell, and G. Auer, "Multi- dimensional graph-based soft iterative receiver for MIMO-OFDM," IEEE Trans. Commun., vol. 60, no. 6, pp. 1599-1609, Jun. 2012.
35. X. Xu and R. Mathar, "Low complexity joint channel estimation and detection for LDPC codedMIMO-OFDMsystems," in Proc. IEEE Veh. Technol. Conf., Budapest, Hungary, May 2011, pp. 1-5.
36. M.-A. Badiu, G. E. Kirkelund, C. N. Manchón, E. Riegler, and B. H. Fleury, "Message-passing algorithms for channel estimation and decoding using approximate inference," in Proc. IEEE Int. Symp. Inf. Theory, Cambridge, MA, Jul. 2012, pp. 1-5.
37. B. D. Craven, "A generalization of Lagrange multipliers," Bull. Austral. Math. Soc., vol. 3, pp. 353-362, 1970.
38. D. G. Luenberger, Optimization by Vector Space Methods. New York: Wiley, 1997.
39. H. Wymeersch, J. Lien, and M. Z. Win, "Cooperative localization in wireless networks," Proc. IEEE, vol. 97, no. 2, pp. 427-450, Feb. 2009.
40. C. Pedersen, T. Pedersen, and B. H. Fleury, "A variational message passing algorithm for sensor self-localization in wireless networks," in Proc. IEEE Int. Symp. Inf. Theory, St. Petersburg, Russia, Jul.-Aug. 2011, pp. 2158-2162.