Lossy source compression using low-density generator matrix codes: Analysis and algorithms

IEEE Transactions on Information Theory

Volumn 56, Issue 3, 2010, Pages 1351-1368

  Wainwright, Martin J ^a   Maneva, Elitza ^b   Martinian, Emin ^c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF BARCELONA   (Spain)

^c Tilda Consulting   (United States)

Author keywords

Belief propagation; Graphical codes, low density generator matrix (LDGM) codes; Lossy source coding; MAX XORSAT; Message passing; Satisfiability problems; Sum product; Survey propagation

Indexed keywords

BELIEF PROPAGATION; LOSSY SOURCE CODING; LOW-DENSITY GENERATOR MATRIX CODES; SATISFIABILITY PROBLEMS; SUM PRODUCT; SURVEY PROPAGATION;

ALGORITHMS; COMMUNICATION CHANNELS (INFORMATION THEORY); SURVEYS;

MESSAGE PASSING;

EID: 77949529481     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2009.2039160     Document Type: Article

References (48)

1. D. Achlioptas and Y. Peres, "The threshold for random k-SAT is 2klog2 - o(k)," J. AMS, vol.17, pp. 947-973, 2004.
2. N. Alon and J. Spencer, The Probabilistic Method. New York:Wiley Interscience, 2000.
3. C. Berroux and A. Glavieux, "Near optimum error correcting coding and decoding: Turbo codes," IEEE Trans. Commun., vol.44, pp. 1261-1271, Oct. 1996.
4. D. P. Bertsekas, Nonlinear Programming. Belmont, MA: Athena Scientific, 1995.
5. S.-Y. Chung, G. D. Forney, T. Richardson, and R. Urbanke, "On the design of low-density parity-check codes within 0.0045 dB of the Shannon limit," IEEE Commun. Lett., vol.5, no.2, pp. 58-60, Feb. 2001.
6. S. Ciliberti and M. Mézard, "The theoretical capacity of the parity source coder," J. Stat. Mechan., p. P10003, 2006.
7. S. Ciliberti, M. Mézard, and R. Zecchina, "Message-passing algorithms for non-linear nodes and data compression," Complexus, vol.3, p. 58, 2006.
8. S. Cocco, O. Dubois, J. Mandler, and R. Monasson, "Rigorous decimation- based construction of ground pure states for spin-glass models on random lattices," Phys. Rev. Lett., vol.90, no.4, Jan. 2003.
9. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
10. N. Creignou, H. Daudé, and O. Dubois, "Approximating the satisfiability threshold of random XOR formulas," Combin., Probab. Comput., vol.12, pp. 113-126, 2003.
11. A. G. Dimakis, M. J.Wainwright, and K. Ramchandran, Lower Bounds on the Rate-Distortion Function of LDGM Codes Aug. 2008, Tech. Rep., Univ. California, Berkeley.
12. O. Dubois and J. Mandler, "The 3-XORSAT threshold," in Proc. 43rd Symp. FOCS, 2002, pp. 769-778.
13. J. Feldman, M. J. Wainwright, and D. R. Karger, "Using linear programming to decode binary linear codes," IEEE Trans. Inf. Theory, vol.51, pp. 954-972, Mar. 2005.
14. T. Filler, "Minimizing Embedding Impact in Steganography Using Low Density Codes," Master's thesis, SUNY Binghamton, , Jul. 2007.
15. T. Filler and J. Fridrich, "Binary quantization using belief propagation with decimation over factor graphs of LDGM codes," in Proc. Allerton Conf. Commun. Contr. Computing, Sept. 2007.
16. J. Fridrich and T. Filler, "Practical methods for minimizing embedding impact in steganography," in Proc. SPIE Electron. Imag., Jan. 2007, vol.66, pp. 1-15.
17. R. G. Gallager, Low-Density Parity Check Codes. Cambridge, MA: MIT Press, 1963.
18. J. Garcia-Frias and Y. Zhao, "Compression of binary memoryless sources using punctured turbo codes," IEEE Commun. Lett., vol.6, no.9, pp. 394-396, Sept. 2002.
19. T. Goblick, "Coding for a Discrete Information Source With a Distortion Measure" Ph.D. dissertation, MIT, , 1962 [Online]. Available: http://hdl.handle.net/1721.1/39984
20. G. R. Grimmett and D. R. Stirzaker, Probability and Random Processes. Oxford, U.K.: Oxford Science, Clarendon, 1992.
21. A. Gupta and S. Verdu, "Nonlinear sparse graph codes for lossy compression," in Proc. Int. Symp. Inf. Theory Workshop, Lake Tahoe, NV, 2007.
22. J. Hiriart-Urruty and C. Lemaréchal, Convex Analysis and Minimization Algorithms Volume1. New York: Springer-Verlag, 1993.
23. F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, "Factor graphs and the sum-product algorithm," IEEE Trans. Inf. Theory, vol.47, no.2, pp. 498-519, Feb. 2001.
24. S. Kudekar and R. Urbanke, Lower Bounds on the Rate-Distortion Function of Individual LDGM Codes Tech. Rep. arXiv:cs.IT:0804. 1697, Apr. 2008, EPFL.
25. H. A. Loeliger, "An introduction to factor graphs," IEEE Signal Processing Mag., vol.21, pp. 28-41, 2004.
26. M. Luby,M.Mitzenmacher,M. A. Shokrollahi, and D. Spielman, "Improved low-density parity check codes using irregular graphs," IEEE Trans. Inf. Theory, vol.47, pp. 585-598, Feb. 2001.
27. E. Maneva, E. Mossel, and M. J. Wainwright, "A new look at survey propagation and its generalizations," J. ACM, vol.54, no.4, pp. 2-41, 2007.
28. M. W. Marcellin and T. R. Fischer, "Trellis coded quantization of memoryless and Gauss-Markov sources," IEEE Trans. Commun., vol.38, no.1, pp. 82-93, 1990.
29. E. Martinian and M. J. Wainwright, "Analysis of LDGM and compound codes for lossy compression and binning," in Proc. Workshop Inf. Theory Applicat. (ITA), Feb. 2006, pp. 229-233, Available at arxiv:cs.IT/0602046.
30. E. Martinian and M. J. Wainwright, "Low density codes achieve the rate-distortion bound," in Proc. Data Compress. Conf., Mar. 2006, vol.1, pp. 153-162, Available at arxiv:cs.IT/061123.
31. E. Martinian and M. J. Wainwright, "Low density codes can achieve the Wyner-Ziv and Gelfand-Pinsker bounds," in Proc. Int. Symp. Inf. Theory, July 2006, pp. 484-488, Available at arxiv:cs.IT/0605091.
32. E. Martinian and J. S. Yedidia, "Iterative quantization using codes on graphs," in Proc. Allerton Conf. Contr., Comput. Commun., Oct. 2003.
33. Y. Matsunaga and H. Yamamoto, "A coding theorem for lossy data compression by LDPC codes," IEEE Trans. Inf. Theory, vol.49, pp. 2225-2229, 2003.
34. M. Mézard, G. Parisi, and R. Zecchina, "Analytic and algorithmic solution of random satisfiability problems," Science, 2002.
35. M. Mézard, F. Ricci-Tersenghi, and R. Zecchina, "Alternative solutions to diluted p-spin models and XORSAT problems," J. Statist. Phys., vol.111, no.105, 2002.
36. M. Mézard and R. Zecchina, "Random k-satisfiability: From an analytic solution to an efficient algorithm," Phys. Rev. E, vol.66, no.056126, 2002.
37. R. Motwani and P. Raghavan, Randomized Algorithms. Cambridge, U.K.: Cambridge University Press, 1995.
38. T. Murayama, "Thouless-Anderson-Palmer approach for lossy compression," Phys. Rev. E, vol.69, pp. 035105(1)-035105(4), 2004.
39. T. Murayama and M. Okada, "One step RSB} scheme for the rate distortion function," J. Phys. A: Math. Gen., vol.65, pp. 11123-11130, 2003.
40. J. Pearl, Probabilistic Reasoning in Intelligent Systems. San Mateo, CA: Morgan Kaufman, 1988.
41. T. Richardson, A. Shokrollahi, and R. Urbanke, "Design of capacityapproaching irregular low-density parity check codes," IEEE Trans. Inf. Theory, vol.47, pp. 619-637, Feb. 2001.
42. T. Richardson and R. Urbanke, Modern Coding Theory. New York: Cambridge University Press, 2008.
43. D. Schonberg, S. S. Pradhan, and K. Ramchandran, "LDPC codes can approach the Slepian-Wolf bound for general binary sources," in Proc. 40th Annu. Allerton Conf. Contr., Commun., Comput., Oct. 2002, pp. 576-585.
44. R. M. Tanner, "A recursive approach to low complexity codes," IEEE Trans. Inf. Theory, vol.IT-27, pp. 533-547, Sep. 1980.
45. A. J. Viterbi and J. K. Omura, "Trellis encoding of memoryless discrete- time sources with a fidelity criterion," IEEE Trans. Inf. Theory, vol.IT-20, no.3, pp. 325-332, 1974.
46. M. J. Wainwright and E. Maneva, "Lossy source coding by messagepassing and decimation over generalized codewords of LDGM codes," in Proc. Int. Symp. Inf. Theory, Adelaide, Australia, Sep. 2005, Available at arxiv:cs.IT/0508068.
47. M. J. Wainwright and E. Martinian, "Low-density codes that are optimal for binning and coding with side information," IEEE Trans. Inf. Theory, vol.55, no.3, pp. 1061-1079, Mar. 2009.
48. A. G. Dimakis, M. J. Wainwright, and K. Ramchandran, "Lower bounds on the rate-distortion function of LDGM codes," presented at the Int. Symp. Inf. Theory Workshop (ITW) 2007, Lake Tahoe, NV, 2007.