Maxwell construction: The hidden bridge between iterative and maximum a Posteriori decoding

IEEE Transactions on Information Theory

Volumn 54, Issue 12, 2008, Pages 5277-5307

  Méasson, Cyril ^a   Montanari, Andrea ^b   Urbanke, Rüdiger ^c  

^a QUALCOMM INC   (United States)

^b Stanford University   (United States)

^c EPFL   (Switzerland)

Author keywords

Area theorem; Belief propagation (BP); Entropy; EXIT curve; Maximum a posteriori (MAP); Maximum likelihood; Maxwell construction; Phase transition; Threshold

Indexed keywords

CONFORMAL MAPPING; ITERATIVE DECODING; MAXIMUM LIKELIHOOD ESTIMATION; MAXWELL EQUATIONS; PHASE TRANSITIONS;

AREA THEOREM; BELIEF PROPAGATION (BP); EXIT CURVE; MAXIMUM A POSTERIORI (MAP); MAXWELL CONSTRUCTION; THRESHOLD;

DECODING;

EID: 57449105742     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2008.2006466     Document Type: Article

References (54)

1. C. Méasson and R. Urbanke, "An upper-bound for the ML threshold of iterative coding systems over the BEC," in Proc. 41st Allerton Conf. Communications, Control and Computing, Monticello, IL, Oct. 2003.
2. A. Montanari, "Why 'practical' decoding algorithms are not as good as 'ideal' ones?," in Proc. DIMACS Workshop on Codes and Complexity, Piscataway, NJ, Dec. 2001, pp. 63-66.
3. C. Méasson, A. Montanari, and R. Urbanke, "Maxwell's construction: The hidden bridge between maximum-likelihood and iterative decoding," in Proc. IEEE Int. Symp. Inform. Theory, Chicago, IL, 2004.
4. C. Méasson, A. Montanari, T. Richardson, and R. Urbanke, "Life above threshold: From list decoding to area theorem and MSE," in Proc. IEEE Information Theory Workshop, San Antonio, TX, Oct. 2004.
5. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA: Morgan Kaufmann, 1988.
6. N. Wiberg, "Codes and Decoding on General Graphs," Ph.D. dissertation, Linköping University, Linköping, Sweden, 1996, S-581 83.
7. S. M. Aji and R. J. McEliece, "The generalized distributive law," IEEE Trans. Inf. Theory, vol. 46, no. 2, pp. 325-343, Mar. 2000.
8. F. Kschischang, B. Frey, and H.-A. Loeliger, "Factor graphs and the sum-product algorithm," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 498-519, Feb. 2001.
9. T. Etzion, A. Trachtenberg, and A. Vardy, "Which codes have cycle-free Tanner graphs?," IEEE Trans. Inf. Theory, vol. 45, no. 6, pp. 2173-2181, Sep. 1999.
10. T. Richardson and R. Urbanke, Modern Coding Theory. Cambridge, U.K.: Cambridge Univ. Press, 2008.
11. S. ten Brink, "Convergence behavior of iteratively decoded parallel concatenated codes," IEEE Trans. Commun., vol. 49, no. 10, pp. 1727-1737, Oct. 2001.
12. A. Ashikhmin, G. Kramer, and S. ten Brink, "Code rate and the area under extrinsic information transfer curves," in Proc. IEEE Int. Symp. Information Theory, Lausanne, Switzerland, Jun./Jul. 2002, p. 115.
13. H. Pishro-Nik and F. Fekri, "On decoding of low-density parity-check codes over the binary erasure channel," IEEE Trans. Inf. Theory, vol. 50, no. 3, pp. 439-454, Mar. 2004.
14. C. Méasson, A. Montanari, T. Richardson, and R. Urbanke, "The generalized area theorem and some of its consequences," IEEE Trans. Inf. Theory, submitted for publication.
15. R. G. Gallager, Low-Density Parity-Check Codes. Cambridge, MA: MIT Press, 1963.
16. M. Luby, M. Mitzenmacher, A. Shokrollahi, and D. A. Spielman, "Efficient erasure correcting codes," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 569-584, Feb. 2001.
17. M. Luby, M. Mitzenmacher, A. Shokrollahi, and D. A. Spielman, "Improved low-density parity-check codes using irregular graphs," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 585-598, Feb. 2001.
18. T. Richardson and R. Urbanke, "The capacity of low-density parity check codes under message-passing decoding," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 599-618, Feb. 2001.
19. T. Richardson, A. Shokrollahi, and R. Urbanke, "Design of capacityapproaching irregular low-density parity-check codes," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 619-637, Feb. 2001.
20. R. G. Gallager, "Low-density parity-check codes," IRE Trans. Inf. Theory, vol. IT-8, no. 1, pp. 21-28, Jan. 1962.
21. T. Murayama, Y. Kabashima, D. Saad, and R. Vicente, "Statistical physics of regular low-density parity-check error-correcting codes," Phys. Rev. E, vol. 62, no. 026705, pp. 1577-1591, 2000.
22. A. Montanari, "The glassy phase of Gallager codes," Europ. Phys. J. vOl. 23, no. 121, 2001 [Online]. Available: ArXiv:cond-math/cond-mat/0104079
23. J. van Mourik, D. Saad, and Y. Kabashima, "Critical noise levels for LDPC decoding," Phys. Rev. E, vol. 66, no. 026705, 2002.
24. S. Franz, M. Leone, A. Montanari, and F. Ricci-Tersenghi, "The dynamic phase transition for decoding algorithms," Phys. Rev. E, vol. 66, no. 046120, 2002.
25. C. Berrou, A. Glavieux, and P. Thitimajshima, "Near Shannon limit error-correcting coding and decoding," in Proc. ICC'93, Geneva, Switzerland, May 1993, pp. 1064-1070.
26. J. Hagenauer, E. Offer, and L. Papke, "Iterative decoding of binary block and convolutional codes," IEEE Trans. Inf. Theory, vol. 42, no. 2, pp. 429-445, Mar. 1996.
27. A. Montanari, "Tight bounds for LDPC and LDGM codes under MAP decoding," IEEE Trans. Inf. Theory, vol. 51, no. 9, pp. 3247-3261, Sep. 2005.
28. G. Zémor and G. Cohen, "The threshold probability of a code," IEEE Trans. Inf. Theory, vol. 41, no. 2, pp. 469-477, Mar. 1995.
29. L. Bazzi, T. Richardson, and R. Urbanke, "Exact thresholds and optimal codes for the binary-symmetric channel and Gallager's decoding algorithm A," IEEE Trans. Inf. Theory, vol. 50, no. 9, pp. 2010-2021, Sep. 2004.
30. B. Bollobás, Random Graphs. Cambridge, U.K.: Cambridge Univ. Press, 2001.
31. C. Di, A. Montanari, and R. Urbanke, "Weight distributions of LDPC code ensembles: Combinatorics meets statistical physics," in Proc. IEEE Int. Symp. Information Theory, Chicago, IL, Jun./Jul. 2004, p. 102.
32. G. Miller and D. Burshtein, "Asymptotic enumeration method for analyzing LDPC codes," IEEE Trans. Inf. Theory, vol. 50, no. 6, pp. 1115-1131, Jun. 2004.
33. S. L. Litsyn and V. S. Shevelev, "On ensembles of low-density parity-check codes: Asymptotic distance distributions," IEEE Trans. Inf. Theory, vol. 48, no. 4, pp. 887-908, Apr. 2002.
34. S. L. Litsyn and V. S. Shevelev, "Distance distribution in ensembles of irregular low-density parity-check codes," IEEE Trans. Inf. Theory vol. 49, no. 12, pp. 3140-3159, Dec. 2003.
35. C. Di, T. Richardson, and R. Urbanke, "Weight distribution of iterative coding systems: How deviant can you be?," in Proc. IEEE Int. Symp. Information Theory, Washington, DC, Jun. 2001, p. 50.
36. C. Di, T. Richardson, and R. Urbanke, "Weight distribution of low-density parity-check codes," IEEE Trans. Inf. Theory, vol. 52, no. 11, pp. 4839-4855, Nov. 2006.
37. C. Di, "Asymptotic and Finite-Length Analysis of Low-Density Parity-Check Codes," Ph.D. dissertation, EPFL, Lausanne, Switzerland, 2004, thesis no. 3072.
38. R. Sedgewick and P. Flajolet, An Introduction to Analysis of Algorithms. Reading, MA: Addison-Wesley, 1996.
39. P. Flajolet and R. Sedgewick, "The Average Case Analysis of Algorithms: Saddle Point Asymptotics" Tech. Rep. RR 2376, 1994.
40. C. Kittel and H. Kroemer, Thermal Physics, 2nd ed. New York: Freeman, Mar. 2002.
41. D. J. C. MacKay, "Good error correcting codes based on very sparse matrices," IEEE Trans. Inf. Theory, vol. 45, no. 2, pp. 399-431, Mar. 1999.
42. A. Montanari, "The glassy phase of gallager codes," Europ. Phys. J. B, vol. 23, pp. 121-136, 2001.
43. C. Méasson, "Conservation Laws for Coding," Ph.D. dissertation, EPFL, Lausanne, Switzerland, 2006, thesis no. 3485.
44. A. Ashikhmin, G. Kramer, and S. ten Brink, "Extrinsic information transfer functions: Model and erasure channel property," IEEE Trans. Inf. Theory, vol. 50, no. 11, pp. 2657-2673, Nov. 2004.
45. C. Méasson and R. Urbanke, "Further analytic properties of EXIT-like curves and applications," in Proc. IEEE Int. Symp. Information Theory, Yokohama, Japan, Jun./Jul. 2003, p. 266.
46. R. M. Tanner, "A recursive approach to low complexity codes," IEEE Trans. Inf. Theory, vol. IT-27, no. 5, pp. 533-547, Sep. 1981.
47. J. Boutros, O. Pothier, and G. Zémor, "Generalized low-density (Tanner) codes," in Proc. ICC'99, Vancouver, BC, Canada, Jun. 1999, pp. 441-445.
48. M. Lentmaier and K. Zigangirov, "Iterative decoding of generalized low-density parity-check codes," in Proc. IEEE Int. Symp. Information Theory, Cambridge, MA, Aug. 1998, pp. 441-445.
49. M. Mézard, F. Ricci-Tersenghi, and R. Zecchina, "Alternative solutions to diluted p-spin models and XORSAT problems," J. Stat. POys. vol. 111, p. 505, 2003 [Online]. Available: ArXiv:cond-math/ cond-mat/0207140
50. S. Cocco, O. Dubois, J. Mandler, and R. Monasson, "Rigorous decimation-based construction of ground pure states for spin glass models on random lattices," Phys. Rev. Lett. vol. 90, no. 047205, 2O03 [Online]. Available: ArXiv:cond-math/cond-mat/0206239
51. A. Braunstein, M. Leone, F. Ricci-Tersenghi, and R. Zecchina, "Complexity transitions in global algorithms for sparse linear systems over finite fields," J. Stat. Phys. vol. 35, p. 7559, 2002 [Online]. Available: ArXiv:cond-math/cond-mat/0203613
52. R. Rockafellar, Convex Analysis. Princeton, NJ: Princeton Univ. Press, 1970.
53. T. Richardson and R. Urbanke, "Efficient encoding of low-density parity-check codes," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 638-656, Feb. 2001.
54. A. Shokrollahi, "Capacity-Achieving Sequences," in Codes, Systems, and Graphical Models, ser. IMA Volumes in Mathematics and its Applications, B. Marcus and J. Rosenthal, Eds. New York: Springer-Verlag, 2000, vol. 123, pp. 153-166.