Statistical mechanics of low-density parity-check codes

Journal of Physics A: Mathematical and General

Volumn 37, Issue 6, 2004, Pages

  Kabashima, Yoshiyuki ^a   Saad, David ^b  

^a Tokyo Institute of Technology   (Japan)

^b ASTON UNIVERSITY   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 1342265651     PISSN: 03054470     EISSN: None     Source Type: Journal    
DOI: 10.1088/0305-4470/37/6/R01     Document Type: Review

References (117)

1. [ADU03] Amraoui A, Dusad S and Urbanke R 2003 Achieving general points in the 2-user Gaussian MAC without time-sharing or rate-splitting by means of iterative coding Proc. 2002 IEEE Int. Symp. on Information Theory (Lausanne, Switzerland) p 334
2. [AJK01] Aji S, Jin H, Khandekar A, MacKay D and McEliece R 2001 ESC thresholds for code ensembles based on 'typical pairs' decoding Codes, Systems and Graphical Models ed B Marcus and J Rosenthal (New York: Springer) pp 195-210
3. [AL95] Amic C D E and Luck J 1995 Zero-temperature error-correcting code for a binary symmetric channel J. Phys. A: Math. Gen. 28 135-47
4. [Bar03] Barg A 2003 More on the reliability function of the BSC Proc. 2003 IEEE Int. Symp. on Information Theory (Yokohama, Japan) p 115
5. [BGT93] Berrou C, Glavieux A and Thitimajshima P 1993 Near Shannon limit error-correcting coding and decoding: turbo codes Proc. IEEE Int. Conf. Commun. (ICC) (Geneva, Switzerland) pp 1064-70
6. [BL82] Bowman D and Levin K 1982 Spin-glass in the Bethe approximation: insights and problems Phys. Rev. B 25 3438-41
7. [Chu00] Chung S-Y 2000 On the construction of some capacity-approaching coding schemes PhD Thesis Massachusetts Institute of Technology
8. [CRU01] Chung S, Richardson T and Urbanke R 2001 Analysis of sum-product decoding of low-density parity-check codes using a Gaussian approximation IEEE Trans. Inform. Theory 47 657-70
9. [CT91] Cover T and Thomas J 1991 Elements of Information Theory (New York: Wiley)
10. [Dav98] Davey M 1998 Record-breaking correction using low-density parity-check codes Hamilton Prize essay Gonville and Caius College, Cambridge
11. [Dav99] Davey M 1999 Error-correction using low-density parity-check codes PhD Thesis University of Cambridge
12. [dBD02] de Baynast A and Declercq D 2002 Random code division multiple access (RCDMA) with Gallager codes Tech. Report unpublished
13. [dBD03] de Baynast A and Declercq D 2003 Gallager codes for multiple user applications Proc. 2002 IEEE Int. Symp. on Information Theory (Lausanne, Switzerland) p 335
14. [Der81] Derrida B 1981 Random energy model: an exactly solvable model of disordered systems Phys. Rev. B 24238-51
15. [DH76] Diffie W and Hellman M E 1976 New directions in cryptography IEEE Trans. Inform. Theory 22 644-54
16. [ElG85] ElGammal T 1985 A public key cryptosystem and a signature scheme based on discrete logarithms IEEE Trans. Inform. Theory 31 469-72
17. [FLMRT02] Franz S, Leone M, Montanari A and Ricci-Tersenghi F 2002 Dynamic phase transition for decoding algorithms Phys. Rev. E 66 046120
18. [FM98] Frey B and MacKay D 1998 A revolution: belief propagation in graphs with cycles Advances in Neural Information Processing Systems vol 10 ed M Jordan, M Kearns and S Solla (Cambridge, MA: The MIT Press) pp 479-85
19. [FP95] Franz S and Parisi G 1995 Recipes for metastable states in spin glasses J. Physique I 5 1401
20. [Fre98] Frey B 1998 Graphical Models for Machine Learning and Digital Communication (Cambridge, MA: MIT Press)
21. [Gal62] Gallager R 1962 Low density parity check codes IRE Trans. Inform. Theory 821-8
22. [Gal63] Gallager R 1963 Low-Density Parity-Check codes (Research Monograph Series no 21) (Cambridge, MA: MIT Press) p 21
23. [Gal68] Gallager R 1968 Information Theory and Reliable Communication (New York: Wiley)
24. [Gil52] Gilbert E N 1952 A comparison of signaling alphabets Bell Syst. Tech. J. 31 504-22
25. [GJ79] Garey M R and Johnson D S 1979 Computers and Intractability vol 251 (San Francisco, CA: Freeman)
26. [Gol91] Goldschmidt Y 1991 Spin glass on the finite-connectivity lattice: the replica solution without replicas Phys. Rev. B 43 8148-52
27. [Gue03] Guerra F 2003 Replica broken bounds in the mean field spin glass model Commun. Math. Phys. 233 1-12
28. [Guj95] Gujrati P 1995 Bethe or Bethe-like lattice calculations are more reliable than conventional mean-field calculations Phys. Rev. Lett. 74 809-12
29. [HKN02] Hosaka T, Kabashima Y and Nishimori H 2002 Statistical mechanics of lossy data compression using a nonmonotonic perception Phys. Rev. E 66 066126
30. [Iba99] Iba Y 1999 The Nishimori line and Bayesian statistics J. Phys. A: Math. Gen. 32 3875-88
31. [Jel68] Jelinek F 1968 Probabilistic Information Theory (New York: McGraw-Hill)
32. [KF98] Kschischang F and Frey B 1998 Iterative decoding of compound codes by probability propagation in graphical models IEEE J. Sel. Areas Commun. 2 153-9
33. [KMS00a] Kabashima Y, Murayama T and Saad D 2000 Cryptographical properties of Ising spin systems Phys. Rev. Lett. 84 2030-3
34. [KMS00b] Kabashima Y, Murayama T and Saad D 2000 Typical performance of Gallager-type error-correcting codes Phys. Rev. Lett. 84 1355-8
35. [KMSV00] Kabashima Y, Murayama T, Saad D and Vicente R 2000 Regular and irregular Gallager-type error-correcting codes Advances in Neural Information Processing Systems vol 12 ed S Solla, T Leen and K Müler (Cambridge, MA: MIT Press) pp 272-8
36. [KNvM02] Kabashima Y, Nakamura K and van Mourik J 2002 Statistical mechanics of typical set decoding Phys. Rev. E 66 036125
37. [KS78] Kirkpatrick S and Sherrington D 1978 Infinite-ranged models of spin-glasses Phys. Rev. B 17 4384-403
38. [KS87] Kanter I and Sompolinsky H 1987 Mean-field theory of spin-glasses with finite coordination number Phys. Rev. Lett. 58 164-7
39. [KS98] Kabashima Y and Saad D 1998 Belief propagation vs. TAP for decoding corrupted messages Europhys. Lett. 44 668-74
40. [KS99a] Kabashima Y and Saad D 1999 Statistical physics of error-correcting codes Europhys. Lett. 45 97-103
41. [KS99b] Kanter I and Saad D 1999 Error-correcting codes that nearly saturate Shannon's bound Phys. Rev. Lett. 83 2660-3
42. [KS00a] Kanter I and Saad D 2000 Cascading parity-check error-correcting codes Phys. Rev. E 61 2137-40
43. [KS00b] Kanter I and Saad D 2000 Finite-size effects and error-free communication in Gaussian channels J. Phys. A: Math. Gen. 33 1675-81
44. [KSNS01] Kabashima Y, Sazuka N, Nakamura K and Saad D 2001 Tighter decoding reliability bound for Gallager's error-correcting codes Phys. Rev. E 64 046113
45. [LMSS01] Luby M, Mitzenmacher M, Shokrollahi A and Spielman D 2001 Improved low-density parity-check codes using irregular graphs and belief propagation IEEE Trans. Inform. Theory 47 585-98
46. [Mac99] MacKay D 1999 Good error-correcting codes based on very sparse matrices IEEE Trans. Inform. Theory 45 399-431
47. [MB01] Miller G and Burshtein D 2001 Bounds on the maximum-likelihood decoding error probability of low-density parity-check codes IEEE Trans. Inform. Theory 47 2696-710
48. [McE78] McEliece R 1978 A public-key cryptosystem based on algebraic coding theory Tech. Report DSN Progress Report 42-44 JPL-Caltech, CA
49. [McEon] McEliece R 2002 Theory of Information & Coding 2nd edn (Cambridge, MA: Cambridge University Press)
50. [MKSV00] Murayama T, Kabashima Y, Saad D and Vicente R 2000 Statistical physics of regular low-density parity-check error-correcting codes Phys. Rev. E 62 1577-91
51. [MN95] MacKay D and Neal R 1995 Good codes based on very sparse matrices Lecture Notes in Computer Science vol 1025 (Berlin: Springer) pp 100-11
52. [MN00] MacKay D and Nakamura K 2000 Thresholds of low-density parity check codes (discussion document) available online at http://www.inference.phy.cam.ac.uk/mackay/Discussion.html
53. [MO03] Murayama T and Okada M 2003 One step RSB scheme for the rate distortion function J. Phys. A: Math. Gen. 36 11123-30
54. [Mon95] Monasson R 1995 The structural glass transition and the entropy of the metastable states Phys. Rev. Lett. 75 2847-50
55. [Mon00] Montanari A 2000 Turbo codes: the phase transition Eur. Phys. J. B 18 121-36
56. [Mon01] Montanari A 2001 The glassy phase of Gallager codes Eur. Phys. J. B 23 121-36
57. [MPV87] Mézard M, Parisi G and Virasoro M 1987 Spin Class Theory and Beyond (Singapore: World Scientific)
58. [MPZ02] Mézard M, Parisi G and Zecchina R 2002 Analytic and algorithmic solution of random satisfiability problems Science 297 812-5
59. [MS77] MacWilliams F and Sloane N 1977 The Theory of Error-Correcting Codes (Amsterdam: North-Holland)
60. [MS00] Montanari A and Sourlas N 2000 The statistical mechanics of turbo codes Eur. Phys. J. B 18 107-19
61. [Mur02] Murayama T 2002 Statistical mechanics of the data compression theorem J. Phys. A: Math. Gen. 35 L95-100
62. [MY02] Matsunaga Y and Yamamoto Y 2002 A coding theorem for lossy data compression by LDPC codes Proc. 2002 IEEE Int. Symp. on Information Theory (Lausanne, Switzerland) p 461
63. [Nis80] Nishimori H 1980 Exact results and critical properties of the Ising model with competing interactions J. Phys. C: Solid State Phys. 13 4071-6
64. [Nis93] Nishimori H 1993 Optimal decoding for error-correcting codes J. Phys. Soc. Japan 62 2973-5
65. [Nis01] Nishimori H 2001 Statistical Physics of Spin Glasses and Information Processing (Oxford: Oxford University Press)
66. [NKMZS03] Nakamura K, Kabashima Y, Morelos-Zaragoza R and Saad D 2003 Statistical mechanics of broadcast channels using low density parity check codes Phys. Rev. E 67 036703
67. [NKS01] Nakamura K, Kabashima Y and Saad D 2001 Statistical mechanics of low-density parity check error-correcting codes over Galois fields Eurphys. Lett. 56 610-6
68. [NS01] Nishimori H and Sherrington D 2001 Absence of replica symmetry breaking in a region of the phase diagram of the Ising spin glass Disordered and Complex Systems ed P Sollich, A Coolen, L Hughston and R Streater (New York: AIP) p 67-72
69. [NW99] Nishimori H and Wong M 1999 Statistical mechanics of image restoration and error-correcting codes Phys. Rev. E 60 132-44
70. [Pea88] Pearl J 1988 Probabilistic Reasoning in Intelligent Systems (San Francisco, CA: Morgan Kaufmann)
71. [RCGV02] Roumy A, Caire G, Guemghar S and Verdú S 2002 Maximizing the spectral efficiency of LDPC-encoded CDMA Tech. Report unpublished
72. [RK92] Rieger H and Kirkpatrick T 1992 Disordered p-spin interaction models on Husimi trees Phys. Rev. B 45 9772-7
73. [RSA78] Rivest R I, Shamir A and Adleman L 1978 A method for obtaining digital signatures and public key cryptosystems Commun. ACM 21 120-6
74. [RSU01] Richardson T, Shokrollahi A and Urbanke R 2001 Design of capacity-approaching irregular low-density parity-check codes IEEE Trans. Inform. Theory 47 619-37
75. [RU01a] Richardson T and Urbanke R 2001 The capacity of low-density parity check codes under message-passing decoding IEEE Trans. Inform. Theory 47 599-618
76. [RU01b] Richardson T and Urbanke R 2001 Efficient decoding of low-density parity-check codes IEEE Trans. Inform. Theory 47 638-56
77. [Ruj93] Ruján P 1993 Finite temperature error-correcting codes Phys. Rev. Lett. 70 2968-71
78. [Saa98] Saakian D 1998 Diluted generalized random energy model JETP Lett. 67 440-4
79. [Sha48] Shannon C 1948 Mathematical theory of communication: I Bell. Syst. Tech. J. 27 379-423 Shannon C 1948 Mathematical theory of communication: II Bell. Syst. Tech. J. 27 623-56
80. [Sha48] Shannon C 1948 Mathematical theory of communication: I Bell. Syst. Tech. J. 27 379-423 Shannon C 1948 Mathematical theory of communication: II Bell. Syst. Tech. J. 27 623-56
81. [Sha49] Shannon C E 1949 The Mathematical Theory of Information (Urbana, IL: University of Illinois Press)
82. [Sha59] Shannon C E 1959 Coding theorems for a discrete source with a fidelity criterion IRE National Convention Record vol 4 pp 142-63
83. [SK75] Sherrington D and Kirkpatrick S 1975 Solvable model of a spin-glass Phys. Rev. Lett. 35 1792-6
84. [SKM01] Saad D, Kabashima Y and Murayama T 2001 Public key cryptography and error correcting codes as Ising models Disordered and Complex Systems ed P Sollich, A Coolen, L Hughston and R Streater (New York: AIP) pp 89-94
85. [Sou89] Sourlas N 1989 Spin-glass models as error-correcting codes Nature 339 693-5
86. [Sou94] Sourlas N 1994 Spin-glasses, error-correcting codes and finite-temperature decoding Europhys. Lett. 25 159-64
87. [SSK03] Skantzos N, Saad D and Kabashima Y 2003 Analysis of common attacks in public-key cryptosystems based on low-density parity-check codes Phys. Rev. E 68 at press
88. [SST92] Seung H S, Sompolinsky H and Tishby N 1992 Statistical mechanics of learning from examples Phys. Rev. A 45 6056-91
89. [SU03] Sason I and Urbanke R 2003 Parity-check density versus performance of binary linear block codes over memoryless symmertic channels IEEE Trans. Inform. Theory 49 1611-35
90. [SvMS03] Skantzos N, van Mourik J and Saad D 2003 Magnetization enumerator of real-valued symmetric channels in gallager error-correcting codes Phys. Rev. E 67 037101
91. [SvMSK03] Skantzos N, van Mourik J, Saad D and Kabashima Y 2003 Average and reliability error exponents in low-density parity-check codes J. Phys. A: Math. Gen. 36 11131-41
92. [SW73] Slepian D and Wolf J K 1973 Noiseless coding of correlated information sources IEEE Trans. Inform. Theory 19 471-80
93. [Tal03] Talagrand M 2003 The generalised Parisi formula Tech. Report unpublished
94. [Tan02] Tanaka T 2002 A statistical-mechanics approach to large-system analysis of CDMA multiuser detectors IEEE Trans. Inform. Theory 11 2888-910
95. [TS03a] Tanaka T and Saad D 2003 A statistical-mechanical analysis of coded CDMA with regular LDPC codes Proc. 2003 IEEE Int. Symp. on Information Theory (Yokohama, Japan) p 444
96. [TS03b] Tanaka T and Saad D 2003 Statistical-mechanical analysis of LDPC-coded CDMA Tech. Report unpublished
97. [TS03c] Tanaka T and Saad D 2003 Typical performance of low-density parity-check codes over general symmetric channels J. Phys. A: Math. Gen. 36 11143-57
98. [Var57] Varshamov R R 1957 Estimate of the number of signals in error-correcting codes Dokl. Akad. Nauk. SSSR 117 5 739-41
99. [Ver98] Verdú S 1998 Multiuser Detection (Cambridge: Cambridge University Press)
100. [vG83] van Gils W 1983 Two topics on linear unequal error protection codes: bounds on their length and cyclic code classes IEEE Trans. Inform. Theory 29 866-76
101. [vG84] van Gils W 1984 Linear unequal error protection codes from shorter codes (corresp.) IEEE Trans. Inform. Theory 30 544-46
102. [vMK03] van Mourik J and Kabashima Y 2003 The polynomial error probability for LDPC codes Preprint cond-mat/0310177
103. [vMSK01] van Mourik J, Saad D and Kabashima Y 2001 Weight vs. magnetization enumerator for Gallager codes Cryptography and Coding: 8th IMA Int. Conf. (Berlin: Germany) ed B Honary (Berlin: Springer) pp 148-57
104. [vMSK02] van Mourik J, Saad D and Kabashima Y 2002 Critical noise levels for LDPC decoding Phys. Rev. E 66 026705
105. [vMWB00] van Mourik J, Wong K Y M and Bollé D 2000 From shrinking to percolation in an optimization model J. Phys. A: Math. Gen. 33 L53
106. [VO79] Viterbi A and Omura J 1979 Principles of Digital Communication and Coding (Singapore: McGraw-Hill)
107. [VSK99] Vicente R, Saad D and Kabashima Y 1999 Finite-connectivity systems as error-correcting codes Phys. Rev. E 60 5352-66
108. [VSK00a] Vicente R, Saad D and Kabashima Y 2000 Error-correcting code on a cactus: a solvable model Europhys. Lett. 51 698-704
109. [VSK00b] Vicente R, Saad D and Kabashima Y 2000 Statistical mechanics of irregular low-density parity-check codes J. Phys. A: Math. Gen. 33 6527-42
110. [VSK01] Vicente R, Saad D and Kabashima Y 2001 Error-correcting codes on a Bethe-like lattice Advances in Neural Information Processing Systems vol 13 ed T Leen, T Dietterich and V Tresp (Cambridge, MA: The MIT Press) pp 322-8
111. [VSK02] Vicente R, Saad D and Kabashima Y 2002 Low density parity check codes: a statistical physics perspective Advances in Imaging and Electron Physics vol 125 ed P Hawkes (New York: Academic) p 232-353
112. [Wib96] Wiberg N 1996 Codes and decoding on general graphs PhD Thesis Department of Electrical Engineering, Linköping University
113. [WS87a] Wong K and Sherrington D 1987 Graph bipartitioning and spin glasses on a random network of fixed finite valence J. Phys. A: Math. Gen. 20 L793-9
114. [WS87b] Wong K and Sherrington D 1987 Graph bipartitioning and the Bethe spin-glass J. Phys. A: Math. Gen. 20 L785-91
115. [YFW02] Yedidia J S, Freeman W T and Weiss Y 2002 Constructing free energy approximations and generalised belief propagation algorithms Tech. Report TR2002-35 Mitsubishi Electric Research Laboratories
116. [YZB97] Yang E, Zhang Z and Berger T 1997 Fixed-slope universal lossy data compression IEEE Trans. Inform. Theory 43 1465-76
117. [ZL77] Ziv J and Lempel A 1977 A universal algorithm for sequential data compression IEEE Trans. Inform. Theory 23 337-43