Turbo decoding as an instance of Pearl's "belief propagation" algorithm

IEEE Journal on Selected Areas in Communications

Volumn 16, Issue 2, 1998, Pages 140-152

  McEliece, Robert J ^{a,b,e,f,g,h,i}   MacKay, David J C ^c,e,f,j   Cheng, Jung Fu ^d,e,k,l  

^a IEEE   (United States)

^b California Institute of Technology   (United States)

^c UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^d Salomon Brothers Inc   (United States)

^e CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^f TRINITY COLLEGE   (United Kingdom)

^g CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^h Information Processing Group   (United States)

ⁱ University of Illinois at Urbana Champaign   (United States)

^j Darwin College   (United Kingdom)

^k NATIONAL TAIWAN UNIVERSITY   (Taiwan)

^l Solomon Brothers Inc   (United States)

more..

Author keywords

Belief propagation; Error correcting codes; Iterative decoding; Pearl's Algorithm; Probabilistic inference; Turbo codes

Indexed keywords

ALGORITHMS; CODES (SYMBOLS); CODING ERRORS; PROBABILISTIC LOGICS;

ERROR CORRECTING CODES; INTERACTIVE DECODING; PEARL BELIEF PROPAGATION ALGORITHM; TURBO CODES;

DECODING;

EID: 0032001728     PISSN: 07338716     EISSN: None     Source Type: Journal    
DOI: 10.1109/49.661103     Document Type: Article

References (68)

1. S. Aji and R. J. McEliece, "A general algorithm for distributing information on a graph," in Proc. 1997 IEEE Int. Symp. Inform. Theory, Ulm, Germany, June 1997, p. 6.
2. _, "The generalized distributive law," in Proc. 4th Int. Symp. Commun. Theory Appl., Ambleside, U.K., July 1997, pp. 135-146. Revised version available from http://www.systems.caltech.edu/EE/Faculty/rjm.
3. J. Andersen, "The TURBO coding scheme," unpublished manuscript distributed at 1994 IEEE Int. Symp. Inform. Theory, Trondheim, Norway, June 1994.
4. L. R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, "Optimal decoding of linear codes for minimizing symbol error rate," IEEE Trans. Inform. Theory, vol. IT-20, pp. 284-287, Mar. 1974.
5. A. S. Barbulescu and S. S. Pietrobon, "Interleaver design for three dimensional turbo-codes," in Proc. 1995 IEEE Int. Symp. Inform. Theory, Whistler, B.C., Canada, Sept. 1995, p. 37.
6. L. E. Baum and T. Petrie, "Statistical inference for probabilistic functions of finite state Markov chains," Ann. Math. Statist., vol. 37, pp. 1554-1563, 1966.
7. S. Benedetto and G. Montorsi, "Unveiling turbo codes: Some results on parallel concatenated coding schemes," IEEE Trans. Inform. Theory, vol. 42, pp. 409-428, Mar. 1996.
8. _, "Serial concatenation of block and convolutional codes," Electron. Lett., vol. 32, pp. 887-888, May 1996.
9. S. Benedetto, G. Montorsi, D. Divsalar, and F. Pollara, "Serial concatenation of interleaved codes: Performance analysis, design, and iterative decoding," JPL TDA Progr. Rep., vol. 42-126, Aug. 1996.
10. G. Berrou, A. Glavieux, and P. Thitimajshima, "Near Shannon limit error-correcting coding: Turbo codes," in Proc. 1993 Int. Conf. Commun., Geneva, Switzerland, May 1993, pp. 1064-1070.
11. R. W. Chang and J. C. Hancock, "On receiver structures for channels having memory," IEEE Trans. Inform. Theory, vol. IT-12, pp. 463-468, Oct. 1966.
12. J.-F. Cheng and R. J. McEliece, "Unit memory Hamming turbo codes," in Proc. 1995 IEEE Int. Symp. Inform. Theory, Whistler, B.C., Canada, Sept. 1995, p. 33.
13. _, "Near capacity codecs for the Gaussian channel based on low-density generator matrices," submitted to 1996 Allerton Conf.
14. J.-F. Cheng, "On the construction of efficient multilevel coded modulations," submitted to the 1997 IEEE Int. Symp. Inform. Theory.
15. _, "Iterative decoding," Ph.D dissertation, Caltech, Pasadena. CA, Mar. 1997.
16. G. Cooper, "The computational complexity of probabilistic inference using Bayesian belief networks," Artif. Intell., vol. 42, pp. 393-405, 1990.
17. P. Dagum and M. Luby, "Approximating probabilistic inference in Bayesian belief networks is NP-hard," Artif. Intell., vol. 60, pp. 141-153, 1993.
18. D. Divsalar and F. Pollara, "Turbo codes for deep-space communications," TDA Progr. Rep., vol. 42-120, pp. 29-39, Feb. 15, 1995.
19. _, "Multiple turbo codes for deep-space communications," TDA Progr. Rep. vol. 42-121, pp. 66-77, May 15, 1995.
20. D. Divsalar, S. Dolinar, R. J. McEliece, and F. Pollara, "Transfer function bounds on the performance of turbo codes," TDA Progr. Rep. vol. 42-122, pp, 44-55, July 15, 1995.
21. D. Divsalar and R. J. McEliece, "Effective free distance of turbo-codes," Electron. Lett., vol. 32, pp. 445-446, Feb. 1996.
22. G. D. Forney, Jr., Concatenated Codes. Cambridge, MA: MIT Press, 1966.
23. _, "The Viterbi algorithm," Proc. IEEE, vol. 63, pp. 268-278, Mar. 1973.
24. _, "The forward-backward algorithm" in Proc. 34th Allerton Conf. Commun., Contr., Computing, Allerton, IL, Oct. 1996.
25. R. G. Gallager, "Low-density parity-check codes," IRE Trans. Inform. Theory, vol. IT-8, pp. 21-28, Jan. 1962.
26. _, Low-Density Parity-Check Codes. Cambridge, MA: MIT Press, 1963.
27. J. Hagenauer, E. Offer, and L. Papke, "Iterative decoding of binary block and convolutional codes," IEEE Trans. Inform. Theory vol. 42, pp. 429-445, Mar. 1996.
28. D. Heckerman and M. P. Wellman, "Bayesian networks," Commun. ACM, vol. 38, pp. 27-30, 1995.
29. F. V. Jensen, S. L. Lauritzen, and K. G. Olesen, "Bayesian updating in recursive graphical models by local compulations," Computational Statist. Quart., vol. 4, pp. 269-282, 1990.
30. F. V. Jensen, An Introduction to Bayesian Networks. New York: Springer-Verlag, 1996.
31. J. H. Kim and J. Pearl, "A computational model for combined causal and diagnostic reasoning in inference systems," in Proc. 8th Int. Joint Conf. AI (IJCA183), Karlsruhe, Germany, pp. 190-193.
32. R. Kindermann and J. L. Snell, Markov Random Fields and their Applications. Providence, RI: American Mathematical Society, 1980.
33. F. R. Kschischang and B. J. Frey, "Iterative decoding of compound codes by probability propagation in graphical models, this issue, pp. 219-230.
34. S. L. Lauritzen and D. J. Spiegelhalter, "Local computations with probabilities on graphical structures and their application to expert systems," J. Roy. Statist. Soc., Ser. B, vol. 50, pp. 157-224, 1988.
35. S. L. Lauritzen, A. P. Dawid, B. N. Larsen, and H.-G. Leimer, "Independence properties of directed Markov fields," Networks, vol. 20, pp. 491-505, 1990.
36. D. J. C. MacKay, "A free energy minimization framework for inference problems in modulo 2 arithmetic," in Fast Software Encryption, B. Preneel, Ed. Berlin, Germany: Springer-Verlag Lecture Notes in Computer Science, vol. 1008, 1995, pp. 179-195.
37. D. J. C. MacKay and R. Neal, "Good codes based on very sparse matrices," in Proc. 5th IMA Conf. Cryptography and Coding, C. Boyd, Ed. Berlin, Germany: Springer Lecture Notes in Computer Science, vol. 1025, 1995, pp. 100-111.
38. D. J. C. MacKay, "Good error-correcting codes based on very sparse matrices," submitted to IEEE Trans. Inform. Theory. Preprint available from http://wol.ra.phy.cam.ac.uk.
39. D. J. C. MacKay and R. M. Neal, "Near Shannon limit performance of low density parity check codes," Electron. Lett., vol. 32, pp. 1645-1646, Aug. 1996. Reprinted in Electron. Lett., vol. 33, pp. 457-458, Mar. 1997.
40. D. J. C. MacKay and R. M. Neal, "Near Shannon limit performance of low density parity check codes," Electron. Lett., vol. 32, pp. 1645-1646, Aug. 1996. Reprinted in Electron. Lett., vol. 33, pp. 457-458, Mar. 1997.
41. P. L. McAdam, L. Welch, and C. Weber, "M.A.P. bit decoding of convolutional codes," in Abstr. Papers, 1972 IEEE Int. Symp. Inform. Theory, Asilomar, CA, Jan. 1972, p. 90.
42. R. J. McEliece, E. R. Rodemich, and J-F. Cheng, "The turbo decision algorithm," in Proc. 33rd Allerton Conf. Commun., Contr., Computing, Oct. 1995, pp. 366-379.
43. J. Pearl, "Reverend Bayes on inference engines: A distributed hierarchical approach," in Proc. Conf. Nat. Conf. A1, Pittsburgh, PA, 1982, pp. 133-136.
44. _, "Fusion, propagation, and structuring in belief networks," Artif. Intell., vol. 29, pp. 241-288, 1986.
45. _, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA: Morgan Kaufmann, 1988.
46. L. C. Perez, J. Seghers, and D. J. Costello, Jr., " A distance spectrum interpretation of turbo codes," IEEE Trans. Inform. Theory, vol. 42, pp. 1698-1709, Nov. 1996.
47. A. Picart and R. Pyndiah, "Performance of turbo decoded product codes used in multilevel coding," in Proc. IEEE ICC'96, Dallas, TX, June 1996.
48. A. M. Poritz, "Hidden Markov models: A guided tour," in Proc. 1988 IEEE Int. Conf. Acoust., Speech, Signal Processing. New York: IEEE Press, vol. 1, pp. 7-13.
49. R. Pyndiah, A. Glavieux, A. Picart, and S. Jacq, "Near optimum decoding of product codes," in Proc. IEEE GLOBECOM'94, San Francisco, CA, Nov. 1994, vol. 1, pp. 339-343.
50. L. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, pp. 257-285, 1989.
51. P. Robertson, "Illuminating the structure of code and decoder of parallel and concatenated recursive systematic (Turbo) codes," in Proc. IEEE GLOBECOM 1994, pp. 1298-1303.
52. R. D. Shachter, "Probabilistic inference and influence diagrams," Oper. Res., vol. 36, pp. 589-604, 1988.
53. G. R. Shafer and P. P. Shenoy, "Probability propagation," Ann. Mat. Artif. Intell., vol. 2, pp. 327-352, 1990.
54. S. E. Shimony, "Finding MAPS for belief networks is NP-hard," Artif. Intell., vol. 68, pp. 399-410, 1994.
55. J. Seghers, "On the free distance of turbo codes and related product codes," Ph.D. dissertation, Swiss Fed. Inst. Technol., Zurich, Switzerland, Aug. 1995, Final Rep., Diploma Project SS 1995.
56. P. Smyth, D. Heckerman, and M. Jordan, "Probabilistic independence networks for hidden Markov probability models," Neural Computation, accepted for publication.
57. G. Solomon and H. C. A. van Tilborg, "A connection between block and convolutional codes," SIAM J. Appl. Math., vol. 37, pp. 358-369, Oct. 1979.
58. M. Sipser and D. A. Spielman, "Expander codes," IEEE Trans. Inform. Theory, vol. 42, pp. 1710-1722, Nov. 1996.
59. D. J. Spiegelhalter and S. L. Lauritzen, "Sequential updating of conditional probabilities on directed graphical structures," Networks, vol. 20, pp. 579-605, 1990.
60. D. J. Spiegelhalter, A. P. Dawid, S. L. Lauritzen, and R. G. Cowell, "Bayesian analysis in expert systems," Statist. Sci., vol. 8, pp. 219-283, 1993.
61. D. A. Spielman, "Linear-time encodable and decodable error-corecting codes." IEEE Trans. Inform. Theory, vol. 42, pp. 1723-1731, Nov. 1996.
62. R. M. Tanner, "A recursive approach to low complexity codes," IEEE Trans. Inform. Theory, vol. IT-27, pp. 533-547, Sept. 1981.
63. G. Ungerboeck, "Nonlinear equalization of binary signals in Gaussian noise," IEEE Trans. Commun. Technol., vol. COM-19, pp. 1128, Dec. 1971.
64. S. Verdu and H. V. Poor, "Abstract dynamic programming models under commutativity conditions," SIAM J. Contr. Optimiz., vol. 25, pp. 990-1006, July 1987.
65. A. J. Viterbi, "Error bounds for convolutional codes and an asymptotically optimum decoding algorithm," IEEE Trans. Inform. Theory, vol. IT-13, pp. 260-269, Apr. 1967.
66. J. Whittaker, Graphical Models in Applied Multivariate Statistics. Chichester, U.K.: Wiley, 1990.
67. N. Wiberg, H.-A. Loeliger, and R. Kötter, "Codes and iterative decoding on general graphs," Europ. Trans. Telecommun. vol. 6, pp. 513-526, Sept.-Oct. 1995.
68. N. Wiberg, "Codes and decoding on general graphs," Linköping Studies in Sci. and Technol., dissertations no. 440. Linköping, Sweden, 1996.