Good error-correcting codes based on very sparse matrices

IEEE Transactions on Information Theory

Volumn 45, Issue 2, 1999, Pages 399-431

  MacKay, David J C ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

Error correction codes; Iterative probabilistic decoding; Low complexity decoding; Nonlinear codes; Shannon limit

Indexed keywords

ALGORITHMS; COMMUNICATION CHANNELS (INFORMATION THEORY); COMPUTATIONAL COMPLEXITY; DECODING; ERROR CORRECTION; MATRIX ALGEBRA; PROBABILITY;

BINARY SYMMETRIC CHANNELS (BSC); ERROR CORRECTION CODES; SHANNON LIMIT;

CODES (SYMBOLS);

EID: 0033099611     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.748992     Document Type: Article

References (73)

1. R. Ahlswede and G. Dueck, "Good codes can be produced by a few permutations," IEEE Trans. Inform. Theory, vol. IT-28, pp. 43043, May 1982.
2. S.V. B. Aiyer, M. Niranjan, and F. Fallside, "A theoretical investigation into the performance of the Hopfield model," IEEE Trans. Neural Networks, vol. 1, no. 2, pp. 204-215, 1990.
3. R.J. Anderson, "Searching for the optimum correlation attack," in Fast Software Encryption-Proceedings of 1994 K.U. Leuven Workshop on Cryptographic Algorithms, B. Preneel, Ed., Lecture Notes in Computer Science. Springer-Verlag, 1995, pp. 179-195.
4. S. Andreassen, M. Woldbye, B. Falck, and S. Andersen, "MUNIN-A causal probabilistic network for the interpretation of electromyographic findings," in Proc. 10th Nat. Conf. on AI (AAAI: Menlo Park CA., 1987), pp. 121-123.
5. L.R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, "Optimal decoding of linear codes for minimizing symbol error rate," IEEE Trans. Inform. Theory, vol. IT-20, pp. 284-287, 1974.
6. G. Battail, "We can think of good codes, and even decode them," in Eurocode'92 26-30 October, P. Camion, P. Charpin, and S. Harari, Eds. (Udine, Italy), in CISM Courses and Lectures, no 339. Wien, Austria: Springer, 1993, pp. 353-368.
7. L.E. Baum and T. Pétrie, "Statistical inference for probabilistic functions of finite-state Markov chains," Ann. Math. Stat., vol. 37, pp. 1559-1563, 1966.
8. F. Bein, "Construction of telephone networks," Notices Amer. Math. Soc., vol. 36, Jan. 1989.
9. E.R. Berlekamp, Algebraic Coding Theory. New York: McGraw-Hill, 1968.
10. E.R. Berlekamp, R. J. McEliece, and H. C. A. van Tilborg, "On the intractability of certain coding problems," IEEE Trans. Inform. Theory, vol. IT-24, pp. 384-386, May 1978.
11. C. Berrou and A. Glavieux, "Near optimum error correcting coding and decoding: Turbo-codes," IEEE Trans. Commun., vol. 44, pp. 1261-1271, Oct. 1996.
12. C. Berrou, A. Glavieux, and P. Thitimajshima, "Near Shannon limit error-correcting coding and decoding: Turbo-codes," in Proc. 1993 IEEE Int. Conf. on Communications (Geneva, Switzerland, 1993), pp. 1064-1070.
13. N.H. Bingham, "Fluctuation theory for the Ehrenfest urn," Adv. Appl. Probab., vol. 23, pp. 598-611, 1991.
14. V. Chepyzhov and B. Smeets, "On a fast correlation attack on certain stream ciphers," in Lecture Notes in Computer Science, vol. 547. New York: Springer-Verlag, 1991, pp. 176-185.
15. J.T. Coffey and R. M. Goodman, "Any code of which we cannot think is good," IEEE Trans. Inform. Theory, vol. 36, pp. 1453-1461, Nov. 1990.
16. T.M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
17. M.C. Davey and D. J. C. MacKay, "Low density parity check codes over GF(q)," IEEE Commun. Lett., June 1998.
18. "Low density parity check codes over GF(q)," in Proc. 1998 IEEE Information Theory Workshop, June 1998.
19. P. Delsarte and P. Piret, "Algebraic constructions of Shannon codes for regular channels," IEEE Trans. Inform. Theory, vol. IT-28, pp. 593-599, July 1982.
20. P. Diaconis, R. L. Graham, and J. A. Morrison, "Asymptotic analysis of a random walk on a hypercube with many dimensions," Random Struct. Algor., vol. 1, pp. 51-72, 1990.
21. D. Divsilar and F. Pollara, "On the design of turbo codes," Tech. Rep. TDA 42-123, Jet Propulsion Lab., Pasadena, CA, Nov. 1995.
22. G.-L. Feng and T. R. N. Rao, "Decoding algebraic-geometric codes up to the designed minimum distance," IEEE Trans. Inform. Theory, vol. 39, pp. 375, Jan. 1993.
23. G.D. Forney Jr., Concatenated Codes. Cambridge, MA: MIT Press, 1966.
24. B.J. Frey, Graphical Models for Machine Learning and Digital Communication. Cambridge MA: MIT Press, 1998.
25. P. Gacs, "Reliable computation with cellular automata," J. Comp. Syst. Sei, vol. 32, no. 1, pp. 15-78, 1986.
26. R.G. Gallager, "Low density parity check codes," IRE Trans. Inform. Theory, vol. IT-8, pp. 21-28, Jan. 1962.
27. Low Density Parity Check Codes, no. 21 in Research Monograph Series. Cambridge, MA: MIT Press, 1963.
28. "Finding parity in a simple broadcast network," IEEE Trans. Inform. Theory, vol. 34, pp. 176-180, Mar. 1988.
29. S.W. Golomb, R. E. Peile, and R. A. Scholtz, Basic Concepts in Information Theory and Coding: The Adventures of Secret Agent 00111. New York: Plenum 1994.
30. J.J. Hopfield and D. W. Tank, "Neural computation of decisions in optimization problems," Biol. Cybern., vol. 52, pp. 1-25, 1985.
31. M. Kac, "Random walk and the theory of Brownian motion," Amer. Math. Monthly, vol. 54, pp. 369-391, 1947.
32. S. Karlin, B. Lindqvist, and Y.-C. Yao, "Markov chains on hypercubes: Spectral representations and several majorization relations," Random Structures and Algorithms, vol. 4, pp. 1-36, 1993.
33. S.L. Lauritzen and D. J. Spiegelhalter, "Local computations with probabilities on graphical structures and their application to expert systems," J. Roy. Statist. Soc. B, vol. 50, pp. 157-224, 1988.
34. G. Letac and L. Takacs, "Random walk on the m-dimensional cube," J. Reine Angew. Math., vol. 310, pp. 187-195, 1979.
35. A. Lubotsky, "Ramanujan graphs," Combinatorica, vol. 8, 1988.
36. M.G. Luby, M. Mitzenmacher, M. A. Shokrollahi, and D. A. Spielman, "Improved low-density parity-check codes using irregular graphs and belief propagation," presented at the 1998 International Symposium on Information Theory, ISIT98, Cambridge, MA, Aug. 16-21, 1998. Also in Proc. Int. Symp. Information Theory, 1998, p. 117.
37. D.J. C. MacKay, "Free energy minimization algorithm for decoding and cryptanalysis," Electron. Lett., vol. 31, no. 6, pp. 446-447, 1995.
38. D.J. C. MacKay, "A free energy minimization framework for inference problems in modulo 2 arithmetic," in Fast Software Encryption-Proc. 1994 K.U. Leuven Workshop on Cryptographic Algorithms, B. Preneel, Ed., Lecture Notes in Computer Science, no. 1008. New York: Springer-Verlag, 1995, pp. 179-195.
39. "Iterative probabilistic decoding of low density parity check codes," Animations. [Online]. Available WWW: http://wol.ra.phy.cam. ac.uk/mackay/codes/gifs/, 1997..
40. D.J. C. MacKay and R. M. Neal, "Good codes based on very sparse matrices," in Cryptography and Coding 5th IMA Conf., C. Boyd, Ed., Lecture Notes in Computer Science, no. 1025. Berlin, Germany: Springer, 1995, pp. 100-111.
41. "Near Shannon limit performance of low density parity check codes," Electron. Lett., vol. 32, no. 18, pp. 1645-1646, Aug. 1996. Reprinted Electron. Lett., vol. 33, no. 6, pp. 457158, Mar. 1997.
42. D.J. C. MacKay, S. T. Wilson, and M. C. Davey, "Comparison of constructions of irregular Gallager codes," in Proc. 36th Allerton Conf. Commun. Contr., and Comput., Sept. 1998.
43. F.J. MacWilliams and N. J. A. Sloane, The Theory of Error-Correcting Codes. Amsterdam, The Netherlands: North-Holland, 1977.
44. G.A. Margulis, "Explicit constructions of graphs without short cycles and low-density codes," Combinatorica, vol. 2, no. 1, pp. 71-78, 1982.
45. R.J. McEliece, The Theory of Information and Coding: A Mathematical Framework for Communication. Reading, MA: Addison-Wesley, 1977.
46. R.J. McEliece, D. J. C. MacKay, and J.-F. Cheng, "Turbo decoding as an instance of Pearl's 'belief propagation' algorithm," IEEE J. Select. Areas Commun., vol. 16, no. 2, pp. 140-152, 1998.
47. W. Meier and O. Staffelbach, "Fast correlation attacks on certain stream ciphers," J. Cryptol, vol. 1, pp. 159-176, 1989.
48. M.J. Mihaljevic and J. D. Golic, "A comparison of cryptanalytic principles based on iterative error-correction," in Lecture Notes in Computer Science, vol. 547. New York: Springer-Verlag, 1991, pp. 527-531.
49. "A fast iterative algorithm for a shift register initial state reconstruction given the noisy output sequence," in Advances in Cryptology-AUSCRYPT90, vol. 453. New York: Springer-Verlag, 1992, pp. 165-175.
50. "Convergence of a Bayesian iterative error-correction procedure on a noisy shift register sequence," in Advances in Cryptology-EUROCRYPT'92, vol. 658.. New York: Springer-Verlag, 1993, pp. 124-137.
51. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA: Morgan Kaufmann, 1988.
52. W.W. Peterson and E. J. Weldon Jr.," Error-Correcting Codes, 2nd ed. Cambridge, MA: MIT Press, 1972.
53. N. Pippenger, "The expected capacity of concentrators," SIAM J. Discr. Math., vol. 4, no. 1, pp. 121-129, 1991.
54. N. Pippenger, G. D. Stamoulis, and J. N. Tsitsiklis, "On a lower bound for the redundancy of reliable networks with noisy gates," IEEE Trans. Inform. Theory, vol. 37, pp. 639-643, May 1991.
55. B. Radosavljevic, E. Arikan, and B. Hajek, "Sequential-decoding of low-density parity-check codes by adaptive reordering of parity checks," IEEE Trans. Inform. Theory, vol. 38, pp. 1833-1839, Sept. 1992.
56. T.R. N. Rao and E. Fujiwara, Error-Control Coding for Computer Systems. Englewood Cliffs, NJ: Prentice-Hall, 1989.
57. J. Rissanen and G. G. Langdon, "Arithmetic coding," IBM J. Res. Devet., vol. 23, pp. 149-162, 1979.
58. C.E. Shannon, "A mathematical theory of communication," Bell Syst. Tech. J., vol. 27, pp. 37923, 623-656, 1948.
59. A.M. Slinko, "Design of experiments to detect nonnegligible variables in a linear model," Cybernetics, vol. 27, no. 3, pp. 433142, 1991.
60. D.A. Spielman, "Linear-time encodable and decodable error-correcting codes," IEEE Trans. Inform. Theory, vol. 42, pp. 1723-1731, Nov. 1996.
61. L. Swanson, "A new code for Galileo," in Proc. 1988 IEEE Int. Symp. Information Theory, 1988, pp. 94-95.
62. L. Takacs, "On an urn problem of Paul and Tatiana Ehrenfest," Math. Proc. Camb. Phil. Soc., vol. 86, pp. 127-130, 1979.
63. R.M. Tanner, "A recursive approach to low complexity codes," IEEE Trans. Inform. Theory, vol. IT-27, pp. 533-547, Sept. 1981.
64. C. Thommesen, "Error-correcting capabilities of concatenated codes with MDS outer codes on memoryless channels with maximum-likelihood decoding," IEEE Trans. Inform. Theory, vol. IT-33, pp. 632-640, Sept. 1987.
65. D.J. Thouless, P. W. Anderson, and R. G. Palmer, "Solutions of 'solvable models of a spin glass'," Philosophical Mag., vol. 35, no. 3, pp. 593-601, 1977.
66. M.A. Tsfasman, "Algebraic-geometric codes and asymptotic problems," Discr. Appl. Math., vol. 33, nos. 1-3, pp. 241-256, 1991.
67. A.J. Viterbi, "Error bounds for convolutional codes and an asymptot-ically optimum decoding algorithm," IEEE Trans. Inform. Theory, vol. IT-13, pp. 260-269, 1967.
68. N. Wiberg, "Codes and decoding on general graphs," Ph.D. dissertation, Dept. Elec. Eng., Linköping Univ., Linköping, Sweden (Linköping Studies in Science and Technology, Dissertation 440), 1996.
69. N. Wiberg, H.-A. Loeliger, and R. Kotier, "Codes and iterative decoding on general graphs," Euro. Trans. Telecommun., vol. 6, pp. 513-525, 1995.
70. I.H. Witten, R. M. Neal, and J. G. Cleary, "Arithmetic coding for data compression," Commun. Assoc. Comp. Mach., vol. 30, no. 6, pp. 520-540, 1987.
71. G. Zemor and G. D. Cohen, "The threshold probability of a code," IEEE Trans. Inform. Theory, vol. 41, pp. 46977, Mar. 1995.
72. M. Zivkovic, "On two probabilistic decoding algorithms for binary linear codes," IEEE Trans. Inform. Theory, vol. 37, pp. 1707-1716, Nov. 1991.
73. V.V. Zyablov and M. S. Pinsker, "Estimation of the error-correction complexity for Gallager low-density codes," Probl. Pered. Inform., vol. 11, no. 1, pp. 23-36, 1975.